System

Abstract

A system (700) comprising: a processor (702) configured to: determine a first image of a first recording area, which has at least one first pixel to which first pixel coding information is assigned and which is classified as faulty; determine a second image of a second recording area, wherein the first recording area and the second recording area at least partially overlap in an overlap area; determine a second pixel spatially corresponding to the at least one first pixel in the second image, wherein the first pixel and the second pixel represent at least part of the overlap area;determining replacement pixel coding information for the first pixel of the first image using second pixel coding information associated with the second pixel of the second image and not classified as defective; and assigning the determined replacement pixel coding information to the first pixel of the first image to obtain a first replacement image;

Description

Area

[0001] Aspects of the disclosure relate to a system and in particular to a system for background segmentation. background

[0002] In a conventional background segmentation system, faulty sensor data (or data generated based on faulty sensor data) can lead to errors in background segmentation. When these results are used to control robots or other systems, they can, for example, generate faulty movements. Description of the characters

[0003] Aspects of the disclosure are illustrated in the figures and are explained in more detail below. shows Fig. 1 a schematic representation of a control system for a recording area; shows Fig. 2 an example of a poor quality depth image; shows Fig. 3 illustrates an example of a control system including steps for background segmentation in accordance with aspects of the disclosure; show Fig. 4a, Fig. 4b and Fig. 4c shows an example of background segmentation according to aspects of the disclosure; shows Fig. 5 shows an example of projecting a distribution of measured values according to aspects of the disclosure; shows Fig. 6 illustrates an example of determining dynamic pixels according to aspects of the disclosure; shows Fig. 7 shows an example of a system according to aspects of the disclosure; shows Fig. 8 shows an example of a system according to aspects of the disclosure; shows Fig. 9 Steps for which a system is arranged to perform in accordance with aspects of the disclosure; shows Fig. 10a further steps for which a system is arranged to carry out in accordance with aspects of the disclosure; is Fig. 10b a continuation of Fig. 10a; is Fig. 10c a continuation of Fig. 10b; is Fig. 10d a continuation of Fig. 10c; shows Fig. 11 Steps for which a system is arranged to perform in accordance with aspects of the disclosure; shows Fig. 12a further steps for which a system is arranged according to aspects of the disclosure; is Fig. 12b a continuation of Fig. 12a; is Fig. 12c a continuation of Fig. 12b; is Fig. 12d a continuation of Fig. 12c; Fig. Figure 13 shows an example of the application of aspects of Revelation. Description

[0004] In the following detailed description, reference is made to the accompanying drawings, which form a part hereof, and in which, by way of illustration, aspects of the disclosure are shown. In this regard, directional terminology such as "top", "bottom", "front", "back", "fore", "rear", etc., will be used with reference to the orientation of the described figure(s). Since components of aspects of the disclosure may be positioned in a number of different orientations, the directional terminology is for the purpose of illustration and is in no way limiting. It is to be understood that other aspects of the disclosure may be utilized and structural or logical changes may be made without departing from the scope of the invention. It is to be understood that the features of the various aspects of the disclosure described herein may be combined with one another unless specifically stated otherwise.The following detailed description is therefore not to be taken in a limiting sense, and the scope of protection is defined by the appended claims.

[0005] Throughout this description, the terms "connected," "attached," and "coupled" are used to describe both a direct and indirect connection, a direct or indirect connection, and a direct or indirect coupling. In the figures, identical or similar elements are provided with identical reference numerals where appropriate.

[0006] According to aspects of the present disclosure, in an image acquired by a first imaging device or in an image generated based on an image acquired by a first imaging device, a piece of pixel coding information (which is classified as erroneous) is replaced by a spatially corresponding piece of pixel coding information (which is not classified as erroneous) from an image acquired by a second imaging device or from an image generated based on an image acquired by a second imaging device.Alternatively, based on the fact that an image acquired by a first imaging device or an image generated based on an image acquired by a first imaging device contains image element coding information classified as erroneous, and an image acquired by a second imaging device or an image generated based on an image acquired by a second imaging device contains spatially corresponding image element coding information classified as erroneous, the respective image element coding information can be discarded. The underlying technology according to aspects of the disclosure can also be referred to as an early fusion approach, since image element coding information classified as erroneous is already fused at the level of the respective imaging device (e.g.The image element coding information can be replaced or discarded by image element coding information from another imaging device that is not classified as erroneous. This can increase the reliability of background segmentation based on the images.

[0007] Fig. 1 shows a schematic representation of a control system 102 for a recording area 104. The control system 102 can be configured to receive sensor data from various imaging devices (e.g., cameras) 106. The control system 102 can be configured to generate and transmit control commands for actuators (e.g., robots) 108 based on the received sensor data.

[0008] Fig. 2 shows an example of a poor-quality depth image 202. The depth image 202 is an example of sensor data from an imaging device 106 (here, a depth camera). The depth image 202 has areas with valid image data 204 and areas with invalid image data 206. In addition, both valid and invalid image data can be provided with probability information relating to validity or invalidity (on a pixel, sub-image, or image basis). This probability information can also express the quality of the image information. Areas with invalid image data 206 can arise, for example, from strong glare effects or reflections in an image scene (e.g., recorded using the imaging device 106). In areas with invalid image data 206, the imaging device 106 is referred to as blind (i.e., the data can be classified as erroneous).Areas with invalid image data 206 can, for example, have NAN (from English "not a number"), infinity, or a predefined (e.g., sensor-specific) value. Invalid image data, unless previously corrected or replaced, can lead to the generation of faulty commands for actuators 108.

[0009] Fig. 3 shows an example of a control system 102 with steps for background segmentation according to aspects of the disclosure. As in Fig. 3, imaging devices 106 (e.g., a first imaging device and a second imaging device) can transmit sensor data to the control system 102. The control system 102 (e.g., a processor thereof) can be configured to (e.g., in each case) perform a probabilistic background segmentation on the (e.g., on the respective) sensor data of the imaging devices 106. For this purpose, the control system 102 can be configured to generate a sigma image (e.g., a first sigma image and / or a second sigma image) and a mean image (e.g., a first mean image and / or a second mean image, e.g., a first combined mean and sigma image and / or a second combined mean and sigma image) from the respective sensor data (e.g., from a (temporal) sequence of recorded sensor data). Furthermore, the control system 102 can be configured to generate a NAN image (e.g.,a first NAN image and / or a second NAN image) from the respective sensor data (e.g., from the (temporal) sequence of recorded sensor data). In this case, the control system 102 can be configured to assign image element coding information to image elements in the respective NAN image, which indicate that in the associated sigma image and / or mean value image (e.g., in the associated combined mean and sigma image), an image element is assigned image element coding information that is / will be classified as faulty (e.g., which has an invalid value, e.g., which is / was generated based on an invalid value). Furthermore, the control system 102 can be configured to assign, based on the respective NAN image, those image elements of the associated sigma image and / or mean value image (e.g.,of the associated combined mean-and-sigma image) that are classified as erroneous, to assign (e.g., replace) pixel coding information of the other mean image and / or the other sigma image (or the other combined mean-and-sigma image) at a spatially corresponding location that are not classified as erroneous, in order to obtain an updated sigma image and / or mean image (e.g., an updated combined mean-and-sigma image). That is, according to aspects of the present disclosure, pixel coding information in an image of one imaging device can be replaced (e.g., updated) based on spatially corresponding pixel coding information in an image of another imaging device, so that pixel coding information classified as erroneous can be replaced at a very early stage of background segmentation.Furthermore, the control system 102 can be configured to extract dynamic scene elements in another image of the respective imaging device based on the updated sigma image and / or the updated mean image (e.g., or the updated combined mean and sigma image), fuse the extracted dynamic scene elements, and generate control commands (e.g., robot safety commands) based thereon. The control system 102 can further be configured to transmit the generated control commands to an actuator (e.g., a robot) 108.

[0010] Fig. 4a, Fig. 4b and Fig. 4c show an example of background segmentation according to aspects of the disclosure. Fig. Figure 4a shows a scene captured by an imaging device 106, which has static elements 402 but no dynamic elements. Fig. 4a shows a histogram 404, which represents a number of pixels as a function of the distance of the pixels to the imaging device 106. In the histogram 404, a peak 405 is shown, which represents the static elements 402. In Fig. 4b, dynamic elements 406 have been added to the static elements 402. Here, Fig. 4b a histogram 408 in which, in addition to the peak 405, which represents the static elements 402, an outlier line 409 is shown, which represents the dynamic elements 406. As in Fig. 4c, based on a comparison of the histograms 404 and 408, the dynamic elements 406 can be segmented (i.e., the static elements 402 can be removed).

[0011] Fig. 5 shows an example of projecting a distribution of measured values according to aspects of the disclosure. Fig. 5 shows a first mean image 502 as a point cloud, which was recorded by a first imaging device 106a in a first recording area, and a second mean image 504 as a point cloud, which was recorded by a second imaging device 106b in a second recording area. The second mean image 504 has an area without valid measured values 505 (i.e., a hole). Since the first recording area and the second recording area overlap in the area without valid measured values 505, the first mean image 502 can be projected onto the second mean image 504, and the area without valid measured values 505 in the second mean image 504 can be filled with spatially corresponding information from the first mean image 502 to obtain a replacement mean image 508, which has no area without valid measured values. Formally, this can be represented, for example, as be displayed.

[0012] Fig. 6 is an example of determining dynamic pixels according to aspects of the disclosure. Fig. 6 illustrates a new first sensor image 602 acquired by a first imaging device 106a, as well as a first mean image 604 and a first sigma image 606 associated with the first imaging device 106a. In the mean image 604, two pixels are each assigned pixel coding information with the value NAN. An outlier analysis based on the new first sensor image 602, the mean image 604, and the sigma image 606 would not reveal any dynamic pixels at the corresponding locations for the pixels to which NAN is assigned as pixel coding information. Fig. 6 further illustrates a second mean image 608 and a second sigma image 610, which are associated with the second imaging device 106b and which are projected in an associated manner into the first mean image 604 and the first sigma image 606. An outlier analysis based on the new first sensor image 602, the projected second mean image 608, and the projected second sigma image 610 results in a dynamic pixel 614 for each of the image elements to which image element coding information with the value NAN is associated in the mean image 604.

[0013] Fig. 7 is an example of a system 700 according to aspects of the disclosure. As in Fig. 7, a system 700 (e.g., a background segmentation system 700) according to aspects of the disclosure may include: a processor 702, a memory 704, a user interface 704, an imaging device interface 704, a command output interface 706, and / or a communication interface 708. The processor 702, the memory 704, the user interface 704, the imaging device interface 704, the command output interface 706, and / or the communication interface 708 may be interconnected via an interconnect bus 710. Furthermore, the imaging device interface 704 (and thus, for example, the system 700) can be connected by means of an imaging connection 720 to a first imaging device 722 (external to the system 700), a second imaging device 724 (external to the system 700) and / or a third imaging device 726 (external to the system 700).For example, additional imaging devices (external to system 700) may be connected to system 700 via imaging connection 720.

[0014] The system 700 (e.g., the processor 702 thereof) may receive inputs (e.g., control input) from a user and / or send outputs to the user via the user interface 704.

[0015] The system 700 (e.g., the processor 702 thereof) may generate control commands and may transmit the generated control commands to devices (external to the system 700) via the command output interface 706.

[0016] The system 700 (e.g., the processor 702 thereof) may communicate with an external communication network via the communication interface 708.

[0017] Fig. 8 is an example of a system 800 according to aspects of the disclosure. As in Fig. 8, a system 800 (e.g., a background segmentation system 800) according to aspects of the disclosure may include a processor 802, a memory 804, a user interface 804, an imaging device interface 804, a command output interface 806, a communication interface 808, a first imaging device 822, a second imaging device 824, and / or a third imaging device 826. The processor 802, the memory 804, the user interface 804, the imaging device interface 804, the command output interface 806, and / or the communication interface 808 may be interconnected via an interconnect bus 810. Furthermore, the imaging device interface 804 (and thus the system 800) may be connected to the first imaging device 822, the second imaging device 824 and / or the third imaging device 826 via an imaging connection 820.For example, additional imaging devices may be connected to the system 800 via the imaging connection 820.

[0018] The system 800 (e.g., the processor 802 thereof) may receive inputs (e.g., control input) from a user and / or send outputs to the user via the user interface 804.

[0019] The system 800 (e.g., the processor 802 thereof) may generate control commands and may transmit the generated control commands to devices (external to the system 800) via the command output interface 806.

[0020] The system 800 (e.g., the processor 802 thereof) may communicate with an external communication network via the communication interface 808.

[0021] In relation to Fig. 7 and Fig. 8, the imaging devices 722, 724, 726, 822, 824, 826 (or the further imaging devices) may also be (for example, in a mixed manner) internal (ie, contained therein) and / or external to the system 700 or the system 800. The system 700 and the system 800 are not limited in any way.

[0022] In relation to Fig. 7 and Fig. 8, the imaging devices 722, 724, 726, 822, 824, 826 (or the further imaging devices) can be arranged in a fixed spatial location. Alternatively, all or some of the imaging devices 722, 724, 726, 822, 824, 826 (or the further imaging devices) can be arranged to be movable in space.

[0023] The first imaging device 722, 822 may be a first camera, a first LiDAR camera, or a first depth camera; and / or the second imaging device 724, 824 may be a second camera, a second LiDAR camera, or a second depth camera. Furthermore, the third imaging device 726, 826 (and also, for example, any further imaging device) may also be a first camera, a first LiDAR camera, or a first depth camera.

[0024] Fig. 9 shows steps that a system 700 and / or a system 800 is configured to perform according to aspects of the disclosure. Fig. 10a shows further steps that a system 700 and / or a system 800 is configured to perform according to aspects of the disclosure. Fig. 10b is a continuation of Fig. 10a. Fig. 10c is a continuation of Fig. 10b. Fig. 10d is a sequel to Fig. 10c.

[0025] As in Fig. 9, the processor 702 of the system 700 and / or the processor 802 of the system 800 can be configured to: determine (902) a first image of a first recording area, which has at least one first pixel to which first pixel coding information is assigned that is classified as erroneous; determine (904) a second image of a second recording area, wherein the first recording area and the second recording area at least partially overlap in an overlap area; determine (906) a second pixel spatially corresponding to the at least one first pixel in the second image, wherein the first pixel and the second pixel represent at least a part of the overlap area;determining (908) replacement pixel coding information for the first pixel of the first image using second pixel coding information associated with the second pixel of the second image and not classified as erroneous; and assigning (910) the determined replacement pixel coding information to the first pixel of the first image to obtain a first replacement image;

[0026] The first picture element may, for example, be a first pixel, and / or the second picture element may, for example, be a second pixel.

[0027] The pixel coding information (e.g., the first and / or second pixel coding information) may be any coding information associated with a pixel, such as a value, a numeric value, an integer, a word, a floating point value, or the like.

[0028] As in Fig. 10a, Fig. 10b, Fig. 10c and Fig. 10d, the processor 702 of the system 700 and / or the processor 802 of the system 800 may initially be configured as shown in Fig. 9, that is to say, to: determine (1002) a first image of a first recording area, which has at least one first picture element to which first picture element coding information is assigned, which is classified as erroneous; to determine (1004) a second image of a second recording area, wherein the first recording area and the second recording area at least partially overlap in an overlap area; to determine (1006) a second picture element spatially corresponding to the at least one first picture element in the second image, wherein the first picture element and the second picture element represent at least part of the overlap area; to determine (1008) replacement picture element coding information for the first picture element of the first image using second picture element coding information that is assigned to the second picture element of the second image and that is not classified as erroneous;and assigning (1010) the determined replacement pixel coding information to the first pixel of the first image to obtain a first replacement image;

[0029] The first image may be a first three-dimensional image (e.g., an image indicating respective distance information for each picture element / pixel), which may optionally include distance information; and / or the second image may be a second three-dimensional image (e.g., an image indicating respective distance information for each picture element / pixel), which may optionally include distance information.

[0030] The first image may be a 3D point cloud or a depth image; and / or the second image may be a 3D point cloud or a depth image.

[0031] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 may be configured to calculate the first image from one or more images / data acquired by a first imaging device 722 (for example, in chronological sequence); and / or to calculate the second image from one or more images / data acquired by a second imaging device 724 (for example, in chronological sequence).

[0032] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 can be configured to: determine (1012) the at least one first pixel of the first image as a distribution of measured values, optionally as a mean with variance or as an outlier (for example from one or more images / data acquired (for example in chronological sequence) by means of a first imaging device 722); and / or determine (1012) the at least one second pixel of the second image as a distribution of measured values, optionally as a mean with variance or as an outlier (for example from one or more images / data acquired (for example in chronological sequence) by means of a second imaging device 724).

[0033] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 may be configured to: determine (1014) the second image element spatially corresponding to the at least one first image element in the second image, wherein the first image element and the second image element represent at least the part of the overlap region, wherein the determining comprises performing a coordinate transformation of the first image element from a coordinate system of the first image into a coordinate system of the second image.

[0034] Further, the processor 702 of the system 700 and / or the processor 802 of the system 800 may be configured to: determine the coordinate transformation (1016).

[0035] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 may be configured to: classify (1018) the first pixel coding information associated with the first pixel as erroneous if it corresponds to a predefined value (e.g., NAN, e.g., infinity, e.g., a predefined number (e.g., specific to a respective imaging device).

[0036] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 can be configured to: determine the first image of the first recording area based on image data from a first imaging device 722, 822 (1020); and / or determine the second image of the second recording area based on image data from a second imaging device 724, 824 (1020).

[0037] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 can be configured to: determine (1022) a spatial relationship between the first imaging device 722, 822 and the second imaging device 724, 824; determine (1022) the second image element spatially corresponding to the at least one first image element in the second image, wherein the first image element and the second image element represent at least a portion of the overlap region, using the determined spatial relationship between the first imaging device 722, 822 and the second imaging device 724, 824.

[0038] Performing (1014) the coordinate transformation of the first image element from the coordinate system of the first image to the coordinate system of the second image may be performed (1024) using the determined spatial relationship between the first imaging device 722, 822 and the second imaging device 724, 824 (ie, the processor 702 of the system 700 and / or the processor 802 of the system 800 may be configured to do so).

[0039] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 may be configured to: assign (1026) a replacement mark to the first picture element of the first image to which the replacement picture element coding information is assigned, which mark indicates that the determined replacement picture element coding information is assigned to the first picture element of the first image.

[0040] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 may be configured to: determine (1028) a first result image of the first recording area based on the first replacement image; and determine (1028) a second result image of the second recording area based on the second image.

[0041] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 can be configured to: determine the first result image of the first recording area based on the first replacement image by performing an outlier analysis on a further first image (e.g., an image newly acquired by the first imaging device 722, which, for example, was not used to calculate the first image) of the first recording area (1030); and determine the second result image of the second recording area based on the second image by performing an outlier analysis on a further second image (e.g., an image newly acquired by the second imaging device 724, which, for example, was not used to calculate the second image) of the second recording area (1030).

[0042] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 can be configured to: determine the first result image of the first recording area based on the first replacement image by performing an outlier analysis on a further first image of the first recording area, omitting the first image element of the first image to which the replacement marking is assigned (1032); and determine the second result image of the second recording area based on the second image by performing an outlier analysis on a further second image of the second recording area (1032).

[0043] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 may be configured to: generate a first result image (e.g.an image representing a result of background segmentation) of the first recording area by (1034): determining (1034) replacement pixel coding information based on the determined replacement pixel coding information, which is assigned to the first pixel of the first replacement image; updating (1034) the first replacement image by assigning the determined replacement pixel coding information to the first pixel of the first replacement image; and determining (1034) the first result image based on the updated first replacement image, optionally by performing an outlier analysis on a further first image of the first recording area; and determining (1034) a second result image of the second recording area based on the second image, optionally by performing an outlier analysis on a further second image of the second recording area.

[0044] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 may be configured to: determine the further first image (1036); and / or determine the further second image (1036).

[0045] The further first image may be a further first three-dimensional image, which may optionally include distance information; and / or the further second image may be a further second three-dimensional image, which may optionally include distance information.

[0046] The further first image may be a 3D point cloud or a depth image; and / or the further second image may be a 3D point cloud or a depth image.

[0047] Further, the processor 702 of the system 700 and / or the processor 802 of the system 800 may be configured to: fuse the first result image and the second result image.

[0048] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 may be configured to: determine at least one further result image of at least one further acquisition region; and to fuse the first result image, the second result image, and the at least one further result image.

[0049] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 may be configured to: fuse the resultant images based on determining, in each of the resultant images, spatially corresponding pixels associated with pixel coding information indicating a predefined property.

[0050] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 may be configured to: fuse the resultant images based on determining spatially corresponding pixels associated with pixel coding information indicating a predefined property in each of the resultant images, with the exception of spatially corresponding pixels determined based on the first pixel of the first image associated with a replacement marker.

[0051] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 may be configured to: fuse the resultant images based on determining, in a predefined number of the resultant images, spatially corresponding pixels associated with pixel coding information indicating a predefined property.

[0052] As in Fig. 11, the processor 702 of the system 700 and / or the processor 802 of the system 800 can be configured to: determine (1102) a first image of a first recording area, which has at least one first pixel to which first pixel coding information is assigned that is classified as erroneous; determine (1104) a second image of a second recording area, wherein the first recording area and the second recording area at least partially overlap in an overlap area; determine (1106) a second pixel spatially corresponding to the at least one first pixel in the second image, wherein the first pixel and the second pixel represent at least a part of the overlap area;determining (1108) replacement pixel coding information for the first pixel of the first image using second pixel coding information associated with the second pixel of the second image and classified as erroneous; and, based on the determined replacement pixel coding information, discarding (1110) the pixel coding information associated with the first pixel to obtain a first replacement image;

[0053] The first picture element may, for example, be a first pixel, and / or the second picture element may, for example, be a second pixel.

[0054] The pixel coding information (e.g., the first and / or second pixel coding information) may be any coding information associated with a pixel, such as a value, a numeric value, an integer, a word, a floating point value, or the like.

[0055] As in Fig. 12a, Fig. 12b, Fig. 12c and Fig. 12d, the processor 702 of the system 700 and / or the processor 802 of the system 800 may initially be configured as shown in Fig. 11, that is to say, to: determine (1202) a first image (e.g. a first image representing a result of background segmentation) of a first recording area, which has at least one first picture element to which first picture element coding information is assigned that is classified as faulty; determine (1204) a second image (e.g. a second image representing a result of background segmentation) of a second recording area, wherein the first recording area and the second recording area at least partially overlap in an overlap area; determine (1206) a second picture element spatially corresponding to the at least one first picture element in the second image, wherein the first picture element and the second picture element represent at least part of the overlap area;determining (1208) replacement pixel coding information for the first pixel of the first image using second pixel coding information associated with the second pixel of the second image and classified as erroneous; and, based on the determined replacement pixel coding information, discarding (1210) the first pixel coding information associated with the first pixel to obtain a first replacement image.

[0056] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 can be configured to: determine (1212) the first image as a first result image (for example, as an image representing the result of background segmentation), optionally by performing an outlier analysis on a further first image of the first recording area; and determine (1212) the second image as a second result image (for example, as an image representing the result of background segmentation), optionally by performing an outlier analysis on a further second image of the second recording area.

[0057] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 may be configured to: determine the further first image (1214); and / or determine the further second image (1214).

[0058] The further first image may be a further first three-dimensional image, which may optionally include distance information; and / or wherein the further second image may be a further second three-dimensional image, which may optionally include distance information.

[0059] The further first image may be a 3D point cloud or a depth image; and / or the further second image may be a 3D point cloud or a depth image.

[0060] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 can be configured to: determine the further first image of the first recording area based on image data from a first imaging device 722, 822 (1216); and / or determine the further second image of the second recording area based on image data from a second imaging device 724, 824 (1216).

[0061] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 can be configured to: determine the first image of the first recording area based on image data from a / the first imaging device 722, 822 (1218); and / or determine the second image of the second recording area based on image data from a / the second imaging device 724, 824 (1218).

[0062] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 may be configured to: determine (1220) a spatial relationship between the first imaging device 722, 822 and the second imaging device 724, 824; and determine (1220) the second image element spatially corresponding to the at least one first image element in the second image, wherein the first image element and the second image element represent at least a portion of the overlap region, using the determined spatial relationship between the first imaging device 722, 822 and the second imaging device 724, 824.

[0063] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 can be configured to: determine (1206) the second image element spatially corresponding to the at least one first image element in the second image, wherein the first image element and the second image element represent at least the part of the overlap region, wherein the determining (1206) comprises performing (1222) a coordinate transformation of the first image element from a coordinate system of the first image into a coordinate system of the second image.

[0064] Further, the processor 702 of the system 700 and / or the processor 802 of the system 800 may be configured to: determine the coordinate transformation (1224).

[0065] Performing (1222) the coordinate transformation of the first image element from the coordinate system of the first image to the coordinate system of the second image may be performed using the determined spatial relationship between the first imaging device 722, 822 and the second imaging device 724, 824 (ie, the processor 702 of the system 700 and / or the processor 802 of the system 800 may be configured to do so).

[0066] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 may be configured to: classify the first pixel coding information associated with the first pixel as erroneous by determining (1228) that the first pixel coding information associated with the first pixel is erroneous based on a first output pixel of a first output image (iefor example, an image which was used to determine the first image), wherein the first output pixel corresponds spatially to the first pixel, and wherein the first output pixel is assigned a first output pixel coding information which is classified as erroneous; and / or to classify the second pixel coding information which is assigned to the second pixel as erroneous by determining (1228) that the second pixel coding information which is assigned to the second pixel is based on a second output pixel of a second output image (iefor example, an image which was used to determine the second image), wherein the second output pixel corresponds spatially to the second pixel, and wherein the second output pixel is assigned a second output pixel coding information which is classified as erroneous.

[0067] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 may be configured to: classify (1230) the first output pixel coding information associated with the first output pixel as erroneous if it corresponds to a first predefined value; and / or classify (1230) the second output pixel coding information associated with the second output pixel as erroneous if it corresponds to a second predefined value.

[0068] The first output image may be a first three-dimensional output image, which may optionally include distance information; and / or the second output image may be a second three-dimensional output image, which may optionally include distance information.

[0069] The first output image may be a 3D point cloud or a depth image; and / or the second output image may be a 3D point cloud or a depth image.

[0070] Further, the processor 702 of the system 700 and / or the processor 802 of the system 800 may be configured to: fuse the first replacement image and the second image (1232).

[0071] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 may be configured to: determine (1234) at least one further image of at least one further recording area; and fuse (1234) the first replacement image, the second image, and the at least one further image.

[0072] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 may be configured to: fuse (1236) the first replacement image, the second image, and the at least one further image based on determining, in a predefined number of the first replacement image, the second image, and the at least one further image, spatially corresponding image elements associated with image element coding information indicating a predefined property.

[0073] Furthermore, the processor 702 of the system 700 and / or the processor 802 of the system 800 may be configured to: fuse (1238) the first replacement image, the second image, and the at least one further image based on determining, in each of the first replacement image, the second image, and the at least one further image, spatially corresponding pixels associated with pixel coding information indicating a predefined property.

[0074] In the sense of aspects of the disclosure, the term “spatially corresponding” can refer to the fact that, for example, image elements of different images represent or depict the same point or area in space.

[0075] Furthermore, according to aspects of the disclosure, a non-transitory computer-readable storage medium may be provided that stores instructions that, when executed by a processor, cause the processor to perform the steps that, for example, processor 702 of system 700 and / or processor 802 of system 800 are (or may be) configured to perform as described above.

[0076] Furthermore, according to aspects of the disclosure, a system may be provided which has means for performing the steps which, for example, the processor 702 of the system 700 and / or the processor 802 of the system 800 are (or can be) configured to perform as described above.

[0077] According to aspects of the disclosure, system 700 and / or system 800 can be used, for example, to monitor a workspace of actuators (e.g., robots). System 700 and / or system 800 can, for example, generate control commands to stop the actuators (e.g., robots) when system 700 and / or system 800 detects dynamic image elements (e.g., caused by people moving into a workspace of the actuators / robots).

[0078] Fig. Figure 13 shows an example of the application of aspects of Revelation. Fig.13 schematically illustrates a train 1302 with two door sections 1304. Here, the system 700 and / or the system 800 can be used, for example, to monitor a surveillance area 1306 in the area of the door sections 1304 of the train 1306 for persons or objects, in order to generate, for example, control commands for actuating the door sections 1304 (e.g., to prevent the door sections 1304 from closing if persons are detected in the area thereof).

[0079] Examples of aspects of the disclosure may be as follows.

[0080] Example 1: System comprising: a processor configured to: determine a first image of a first recording area, which has at least one first pixel to which first pixel coding information is assigned that is classified as faulty; determine a second image of a second recording area, wherein the first recording area and the second recording area at least partially overlap in an overlap area; determine a second pixel spatially corresponding to the at least one first pixel in the second image, wherein the first pixel and the second pixel represent at least a part of the overlap area;determining replacement pixel coding information for the first pixel of the first image using second pixel coding information associated with the second pixel of the second image and not classified as defective; and assigning the determined replacement pixel coding information to the first pixel of the first image to obtain a first replacement image;

[0081] Example 2: The system of example 1, wherein the first image is a first three-dimensional image optionally including distance information; and / or wherein the second image is a second three-dimensional image optionally including distance information.

[0082] Example 3: The system of any of examples 1 to 2, wherein the first image is a 3D point cloud or a depth image; and / or wherein the second image is a 3D point cloud or a depth image.

[0083] Example 4: The system of any one of examples 1 to 3, wherein the processor is further configured to: determine the at least one first pixel of the first image as a distribution of measured values, optionally as a mean with variance or as an outlier; and / or determine the at least one second pixel of the second image as a distribution of measured values, optionally as a mean with variance or as an outlier.

[0084] Example 5: The system of any one of examples 1 to 4, wherein the processor is further configured to: determine the second pixel spatially corresponding to the at least one first pixel in the second image, wherein the first pixel and the second pixel represent at least the portion of the overlap region, wherein determining comprises performing a coordinate transformation of the first pixel from a coordinate system of the first image to a coordinate system of the second image.

[0085] Example 6: The system of example 5, wherein the processor is further configured to: determine the coordinate transformation.

[0086] Example 7: The system of any one of examples 1 to 6, wherein the processor is further configured to: classify the first pixel coding information associated with the first pixel as erroneous if it corresponds to a predefined value.

[0087] Example 8: The system of any one of examples 1 to 7, wherein the processor is further configured to: determine the first image of the first recording area based on image data from a first imaging device; and / or determine the second image of the second recording area based on image data from a second imaging device.

[0088] Example 9: The system of example 8, further comprising: the first imaging device and / or the second imaging device.

[0089] Example 10: The system of any of examples 8 to 9, wherein the processor is further configured to: determine a spatial relationship between the first imaging device and the second imaging device; determine the second image element spatially corresponding to the at least one first image element in the second image, wherein the first image element and the second image element represent at least a portion of the overlap region, using the determined spatial relationship between the first imaging device and the second imaging device.

[0090] Example 11: The system of example 10, wherein: performing the coordinate transformation of the first image element from the coordinate system of the first image to the coordinate system of the second image is performed using the determined spatial relationship between the first imaging device and the second imaging device.

[0091] Example 12: The system of any of examples 8 to 11, wherein the first imaging device is a first camera, a first LiDAR camera, or a first depth camera; and / or wherein the second imaging device is a second camera, a second LiDAR camera, or a second depth camera.

[0092] Example 13: The system of any one of examples 1 to 12, wherein the processor is further configured to: assign a replaced flag to the first pixel of the first image to which the replacement pixel coding information is assigned, the replaced flag indicating that the determined replacement pixel coding information is assigned to the first pixel of the first image.

[0093] Example 14: The system of any one of examples 1 to 13, wherein the processor is further configured to: determine a first result image of the first capture area based on the first replacement image; and determine a second result image of the second capture area based on the second image.

[0094] Example 15: The system of example 14, wherein the processor is further configured to: determine the first result image of the first capture area based on the first replacement image by performing an outlier analysis on a further first image of the first capture area; and determine the second result image of the second capture area based on the second image by performing an outlier analysis on a further second image of the second capture area.

[0095] Example 16: The system according to example 13 and example 14, wherein the processor is further configured to: determine the first result image of the first recording area based on the first replacement image by performing an outlier analysis on a further first image of the first recording area, omitting the first pixel of the first image to which the replacement marker is assigned; and determine the second result image of the second recording area based on the second image by performing an outlier analysis on a further second image of the second recording area.

[0096] Example 17: The system according to any one of examples 1 to 13, wherein the processor is further configured to: determine a first result image of the first recording area by: determining replacement pixel coding information based on the determined replacement pixel coding information, which is assigned to the first pixel of the first replacement image; updating the first replacement image by assigning the determined replacement pixel coding information to the first pixel of the first replacement image; and determining the first result image based on the updated first replacement image, optionally by performing an outlier analysis on a further first image of the first recording area; and determining a second result image of the second recording area based on the second image, optionally by performing an outlier analysis on a further second image of the second recording area.

[0097] Example 18: The system of any one of examples 15 to 17, wherein the processor is further configured to: determine the further first image; and / or determine the further second image.

[0098] Example 19: The system of any one of examples 15 to 18, wherein the further first image is a further first three-dimensional image optionally comprising distance information; and / or wherein the further second image is a further second three-dimensional image optionally comprising distance information.

[0099] Example 20: The system of any one of examples 15 to 19, wherein the further first image is a 3D point cloud or a depth image; and / or wherein the further second image is a 3D point cloud or a depth image.

[0100] Example 21: The system of any one of examples 14 to 20, wherein the processor is further configured to: fuse the first result image and the second result image.

[0101] Example 22: The system of any one of examples 14 to 21, wherein the processor is further configured to: determine at least one further result image of at least one further capture region; and fuse the first result image, the second result image, and the at least one further result image.

[0102] Example 23: The system of any one of examples 14 to 22, wherein the processor is further configured to: fuse the resultant images based on determining, in each of the resultant images, spatially corresponding pixels associated with pixel coding information indicating a predefined property.

[0103] Example 24: The system of any of Examples 14 to 23 when combined with Example 13, wherein the processor is further configured to: fuse the resultant images based on determining spatially corresponding pixels associated with pixel coding information indicating a predefined property in each of the resultant images, except for spatially corresponding pixels determined based on the first pixel of the first image associated with a replacement marker.

[0104] Example 25: The system of any one of examples 14 to 23, wherein the processor is further configured to: fuse the resultant images based on determining, in a predefined number of the resultant images, spatially corresponding pixels associated with pixel coding information indicating a predefined property.

[0105] Example 26: System comprising: a processor configured to: determine a first image of a first recording area having at least one first pixel to which first pixel coding information is assigned that is classified as faulty; determine a second image of a second recording area, wherein the first recording area and the second recording area at least partially overlap in an overlap area; determine a second pixel spatially corresponding to the at least one first pixel in the second image, wherein the first pixel and the second pixel represent at least a part of the overlap area;determining replacement pixel coding information for the first pixel of the first image using second pixel coding information associated with the second pixel of the second image and classified as defective; and, based on the determined replacement pixel coding information, discarding the first pixel coding information associated with the first pixel to obtain a first replacement image.

[0106] Example 27: The system of example 26, wherein the processor is further configured to: determine the first image as a first result image, optionally by performing an outlier analysis on a further first image of the first capture area; and determine the second image as a second result image, optionally by performing an outlier analysis on a further second image of the second capture area.

[0107] Example 28: The system of example 27, wherein the processor is further configured to: determine the further first image; and / or determine the further second image.

[0108] Example 29: The system of any one of examples 27 to 28, wherein the further first image is a further first three-dimensional image optionally comprising distance information; and / or wherein the further second image is a further second three-dimensional image optionally comprising distance information.

[0109] Example 30: The system of any one of examples 27 to 29, wherein the further first image is a 3D point cloud or a depth image; and / or wherein the further second image is a 3D point cloud or a depth image.

[0110] Example 31: The system of any one of examples 27 to 30, wherein the processor is further configured to: determine the further first image of the first recording area based on image data from a first imaging device; and / or determine the further second image of the second recording area based on image data from a second imaging device.

[0111] Example 32: The system of any one of examples 26 to 31, wherein the processor is further configured to: determine the first image of the first recording area based on image data from a / the first imaging device; and / or determine the second image of the second recording area based on image data from a / the second imaging device.

[0112] Example 33: The system of any of Examples 31 to 32, further comprising: the first imaging device and / or the second imaging device.

[0113] Example 34: The system of any of examples 31 to 33, wherein the first imaging device is a first camera, a first LiDAR camera, or a first depth camera; and / or wherein the second imaging device is a second camera, a second LiDAR camera, or a second depth camera.

[0114] Example 35: The system of any of examples 31 to 34, wherein the processor is further configured to: determine a spatial relationship between the first imaging device and the second imaging device; and determine the second image element spatially corresponding to the at least one first image element in the second image, wherein the first image element and the second image element represent at least a portion of the overlap region, using the determined spatial relationship between the first imaging device and the second imaging device.

[0115] Example 36: The system of any one of examples 26 to 35, wherein the processor is further configured to: determine the second pixel spatially corresponding to the at least one first pixel in the second image, wherein the first pixel and the second pixel represent at least the portion of the overlap region, wherein determining comprises performing a coordinate transformation of the first pixel from a coordinate system of the first image to a coordinate system of the second image.

[0116] Example 37: The system of example 36, wherein the processor is further configured to: determine the coordinate transformation.

[0117] Example 38: The system of any of Examples 36 to 37 when combined with Example 35, wherein: performing the coordinate transformation of the first pixel from the coordinate system of the first image to the coordinate system of the second image is performed using the determined spatial relationship between the first imaging device and the second imaging device.

[0118] Example 39: The system of any one of examples 26 to 38, wherein the processor is further configured to: classify the first pixel coding information associated with the first pixel as erroneous by determining that the first pixel coding information associated with the first pixel is determined based on a first output pixel of a first output image, wherein the first output pixel spatially corresponds to the first pixel, and wherein the first output pixel is associated with first output pixel coding information that is classified as erroneous;and / or classify the second pixel coding information associated with the second pixel as erroneous by determining that the second pixel coding information associated with the second pixel is determined based on a second output pixel of a second output image, wherein the second output pixel spatially corresponds to the second pixel, and wherein the second output pixel is associated with second output pixel coding information that is classified as erroneous;

[0119] Example 40: The system of example 39, wherein the processor is further configured to: classify the first output pixel coding information associated with the first output pixel as erroneous if it corresponds to a first predefined value; and / or classify the second output pixel coding information associated with the second output pixel as erroneous if it corresponds to a second predefined value.

[0120] Example 41: The system of any one of examples 39 to 40, wherein the first output image is a first three-dimensional output image optionally including range information; and / or wherein the second output image is a second three-dimensional output image optionally including range information.

[0121] Example 42: The system of any of examples 39 to 41, wherein the first output image is a 3D point cloud or a depth image; and / or wherein the second output image is a 3D point cloud or a depth image.

[0122] Example 43: The system of any of examples 26 to 42, wherein the processor is further configured to: fuse the first replacement image and the second image.

[0123] Example 44: The system of any of examples 26 to 43, wherein the processor is further configured to: determine at least one further image of at least one further capture area; and fuse the first replacement image, the second image, and the at least one further image.

[0124] Example 45: The system of example 44, wherein the processor is further configured to: fuse the first replacement image, the second image, and the at least one further image based on determining, in a predefined number of the first replacement image, the second image, and the at least one further image, spatially corresponding pixels associated with pixel coding information indicating a predefined property.

[0125] Example 46: The system of any one of examples 44 to 45, wherein the processor is further configured to: fuse the first replacement image, the second image, and the at least one further image based on determining, in each of the first replacement image, the second image, and the at least one further image, spatially corresponding pixels associated with pixel coding information indicating a predefined property.

[0126] Example 47: Non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to: determine a first image of a first recording area having at least one first pixel associated with first pixel coding information that is classified as defective; determine a second image of a second recording area, wherein the first recording area and the second recording area at least partially overlap in an overlap area; determine a second pixel spatially corresponding to the at least one first pixel in the second image, wherein the first pixel and the second pixel represent at least a portion of the overlap area;determining replacement pixel coding information for the first pixel of the first image using second pixel coding information associated with the second pixel of the second image and not classified as defective; and assigning the determined replacement pixel coding information to the first pixel of the first image to obtain a first replacement image;

[0127] Example 48: The non-transitory computer-readable storage medium of Example 47, wherein the first image is a first three-dimensional image optionally including distance information; and / or wherein the second image is a second three-dimensional image optionally including distance information.

[0128] Example 49: The non-transitory computer-readable storage medium of any of examples 47 to 48, wherein the first image is a 3D point cloud or a depth image; and / or wherein the second image is a 3D point cloud or a depth image.

[0129] Example 50: The non-transitory computer-readable storage medium of any of examples 47 to 49, wherein the instructions, when executed by the processor, further cause the processor to: determine the at least one first pixel of the first image as a distribution of measurements, optionally as a mean with variance or as an outlier; and / or determine the at least one second pixel of the second image as a distribution of measurements, optionally as a mean with variance or as an outlier.

[0130] Example 51: The non-transitory computer-readable storage medium of any of examples 47 to 50, wherein the instructions, when executed by the processor, further cause the processor to: determine the second pixel spatially corresponding to the at least one first pixel in the second image, the first pixel and the second pixel representing at least the portion of the overlap region, wherein determining comprises performing a coordinate transformation of the first pixel from a coordinate system of the first image to a coordinate system of the second image.

[0131] Example 52: The non-transitory computer-readable storage medium of Example 51, wherein the instructions, when executed by the processor, further cause the processor to determine the coordinate transformation.

[0132] Example 53: The non-transitory computer-readable storage medium of any of examples 47 to 52, wherein the instructions, when executed by the processor, further cause the processor to: classify the first pixel coding information associated with the first pixel as erroneous if it corresponds to a predefined value.

[0133] Example 54: The non-transitory computer-readable storage medium of any of examples 47 to 53, wherein the instructions, when executed by the processor, further cause the processor to: determine the first image of the first capture area based on image data from a first imaging device; and / or determine the second image of the second capture area based on image data from a second imaging device.

[0134] Example 55: The non-transitory computer-readable storage medium of example 54, wherein the instructions, when executed by the processor, further cause the processor to: determine a spatial relationship between the first imaging device and the second imaging device; determine the second image element spatially corresponding to the at least one first image element in the second image, wherein the first image element and the second image element represent at least a portion of the overlap region, using the determined spatial relationship between the first imaging device and the second imaging device.

[0135] Example 56: The non-transitory computer-readable storage medium of example 55, wherein: performing the coordinate transformation of the first image element from the coordinate system of the first image to the coordinate system of the second image is performed using the determined spatial relationship between the first imaging device and the second imaging device.

[0136] Example 57: The non-transitory computer-readable storage medium of any of examples 54 to 56, wherein the first imaging device is a first camera, a first LiDAR camera, or a first depth camera; and / or wherein the second imaging device is a second camera, a second LiDAR camera, or a second depth camera.

[0137] Example 58: The non-transitory computer-readable storage medium of any of examples 47 to 57, wherein the instructions, when executed by the processor, further cause the processor to: associate with the first pixel of the first image associated with the replacement pixel coding information a replacement flag indicating that the first pixel of the first image is associated with the determined replacement pixel coding information.

[0138] Example 59: The non-transitory computer-readable storage medium of any of examples 47 to 58, wherein the instructions, when executed by the processor, further cause the processor to: determine a first result image of the first capture area based on the first replacement image; and determine a second result image of the second capture area based on the second image.

[0139] Example 60: The non-transitory computer-readable storage medium of Example 59, wherein the instructions, when executed by the processor, further cause the processor to: determine the first result image of the first capture region based on the first replacement image by performing an outlier analysis on a further first image of the first capture region; and determine the second result image of the second capture region based on the second image by performing an outlier analysis on a further second image of the second capture region.

[0140] Example 61: The non-transitory computer-readable storage medium of example 58 and example 59, wherein the instructions, when executed by the processor, further cause the processor to: determine the first result image of the first capture area based on the first replacement image by performing an outlier analysis on a further first image of the first capture area, omitting the first pixel of the first image to which the replacement marker is assigned; and determine the second result image of the second capture area based on the second image by performing an outlier analysis on a further second image of the second capture area.

[0141] Example 62: The non-transitory computer-readable storage medium of any of examples 47 to 58, wherein the instructions, when executed by the processor, further cause the processor to: determine a first result image of the first capture area by: determining, based on the determined replacement pixel coding information, a replacement pixel coding information associated with the first pixel of the first replacement image; updating the first replacement image by associating the determined replacement pixel coding information with the first pixel of the first replacement image; and determining the first result image based on the updated first replacement image, optionally by performing an outlier analysis on another first image of the first capture area;and to determine a second result image of the second recording area based on the second image, optionally by performing an outlier analysis on a further second image of the second recording area;

[0142] Example 63: The non-transitory computer-readable storage medium of any of examples 60 to 62, wherein the instructions, when executed by the processor, further cause the processor to: determine the further first image; and / or determine the further second image.

[0143] Example 64: The non-transitory computer-readable storage medium of any of examples 60 to 63, wherein the further first image is a further first three-dimensional image optionally comprising distance information; and / or wherein the further second image is a further second three-dimensional image optionally comprising distance information.

[0144] Example 65: The non-transitory computer-readable storage medium of any of examples 60 to 64, wherein the further first image is a 3D point cloud or a depth image; and / or wherein the further second image is a 3D point cloud or a depth image.

[0145] Example 66: The non-transitory computer-readable storage medium of any of examples 59 to 65, wherein the instructions, when executed by the processor, further cause the processor to: fuse the first result image and the second result image.

[0146] Example 67: The non-transitory computer-readable storage medium of any of examples 59 to 66, wherein the instructions, when executed by the processor, further cause the processor to: determine at least one further result image of at least one further capture region; fuse the first result image, the second result image, and the at least one further result image.

[0147] Example 68: The non-transitory computer-readable storage medium of any of examples 59 to 67, wherein the instructions, when executed by the processor, further cause the processor to: fuse the resultant images based on determining, in each of the resultant images, spatially corresponding pixels associated with pixel coding information indicating a predefined property.

[0148] Example 69: The non-transitory computer-readable storage medium of any of Examples 59 to 68 when combined with Example 58, wherein the instructions, when executed by the processor, further cause the processor to: fuse the resultant images based on determining, in each of the resultant images, spatially corresponding pixels associated with pixel coding information indicating a predefined property, except for spatially corresponding pixels determined based on the first pixel of the first image associated with a replacement marker.

[0149] Example 70: The non-transitory computer-readable storage medium of any of examples 59 to 68, wherein the instructions, when executed by the processor, further cause the processor to: fuse the resultant images based on determining, in a predefined number of the resultant images, spatially corresponding pixels associated with pixel coding information indicating a predefined property.

[0150] Example 71: Non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to: determine a first image of a first recording area having at least one first pixel associated with first pixel coding information that is classified as defective; determine a second image of a second recording area, wherein the first recording area and the second recording area at least partially overlap in an overlap area; determine a second pixel spatially corresponding to the at least one first pixel in the second image, wherein the first pixel and the second pixel represent at least a portion of the overlap area;determining replacement pixel coding information for the first pixel of the first image using second pixel coding information associated with the second pixel of the second image and classified as defective; and, based on the determined replacement pixel coding information, discarding the first pixel coding information associated with the first pixel to obtain a first replacement image.

[0151] Example 72: The non-transitory computer-readable storage medium of example 71, wherein the instructions, when executed by the processor, further cause the processor to: determine the first image as a first result image, optionally by performing an outlier analysis on a further first image of the first capture area; and determine the second image as a second result image, optionally by performing an outlier analysis on a further second image of the second capture area.

[0152] Example 73: The non-transitory computer-readable storage medium of example 72, wherein the instructions, when executed by the processor, further cause the processor to: determine the further first image; and / or determine the further second image.

[0153] Example 74: The non-transitory computer-readable storage medium of any of examples 72 to 73, wherein the further first image is a further first three-dimensional image optionally comprising distance information; and / or wherein the further second image is a further second three-dimensional image optionally comprising distance information.

[0154] Example 75: The non-transitory computer-readable storage medium of any one of examples 72 to 74, wherein the further first image is a 3D point cloud or a depth image; and / or wherein the further second image is a 3D point cloud or a depth image.

[0155] Example 76: The non-transitory computer-readable storage medium of any of examples 72 to 75, wherein the instructions, when executed by the processor, further cause the processor to: determine the further first image of the first capture area based on image data from a first imaging device; and / or determine the further second image of the second capture area based on image data from a second imaging device.

[0156] Example 77: The non-transitory computer-readable storage medium of any of examples 71 to 76, wherein the instructions, when executed by the processor, further cause the processor to: determine the first image of the first capture area based on image data from a / the first imaging device; and / or determine the second image of the second capture area based on image data from a / the second imaging device.

[0157] Example 78: The non-transitory computer-readable storage medium of any of examples 76 to 77, wherein the first imaging device is a first camera, a first LiDAR camera, or a first depth camera; and / or wherein the second imaging device is a second camera, a second LiDAR camera, or a second depth camera.

[0158] Example 79: The non-transitory computer-readable storage medium of any of examples 76 to 78, wherein the instructions, when executed by the processor, further cause the processor to: determine a spatial relationship between the first imaging device and the second imaging device; and determine the second image element spatially corresponding to the at least one first image element in the second image, the first image element and the second image element representing at least a portion of the overlap region, using the determined spatial relationship between the first imaging device and the second imaging device.

[0159] Example 80: The non-transitory computer-readable storage medium of any of Examples 71 to 79, wherein the instructions, when executed by the processor, further cause the processor to: determine the second pixel spatially corresponding to the at least one first pixel in the second image, the first pixel and the second pixel representing at least the portion of the overlap region, wherein determining comprises performing a coordinate transformation of the first pixel from a coordinate system of the first image to a coordinate system of the second image.

[0160] Example 81: The non-transitory computer-readable storage medium of Example 80, wherein the instructions, when executed by the processor, further cause the processor to determine the coordinate transformation.

[0161] Example 82: The non-transitory computer-readable storage medium of any of Examples 80 to 81 when combined with Example 79, wherein: performing the coordinate transformation of the first image element from the coordinate system of the first image to the coordinate system of the second image is performed using the determined spatial relationship between the first imaging device and the second imaging device.

[0162] Example 83: The non-transitory computer-readable storage medium of any of examples 71 to 82, wherein the instructions, when executed by the processor, further cause the processor to: classify the first pixel coding information associated with the first pixel as erroneous by determining that the first pixel coding information associated with the first pixel is determined based on a first output pixel of a first output image, wherein the first output pixel spatially corresponds to the first pixel, and wherein the first output pixel is associated with first output pixel coding information that is classified as erroneous;and / or classify the second pixel coding information associated with the second pixel as erroneous by determining that the second pixel coding information associated with the second pixel is determined based on a second output pixel of a second output image, wherein the second output pixel spatially corresponds to the second pixel, and wherein the second output pixel is associated with second output pixel coding information that is classified as erroneous;

[0163] Example 84: The non-transitory computer-readable storage medium of example 83, wherein the instructions, when executed by the processor, further cause the processor to: classify the first output pixel coding information associated with the first output pixel as erroneous if it corresponds to a first predefined value; and / or classify the second output pixel coding information associated with the second output pixel as erroneous if it corresponds to a second predefined value.

[0164] Example 85: The non-transitory computer-readable storage medium of any of examples 83 to 84, wherein the first output image is a first three-dimensional output image optionally including distance information; and / or wherein the second output image is a second three-dimensional output image optionally including distance information.

[0165] Example 86: The non-transitory computer-readable storage medium of any of examples 83 to 85, wherein the first output image is a 3D point cloud or a depth image; and / or wherein the second output image is a 3D point cloud or a depth image.

[0166] Example 87: The non-transitory computer-readable storage medium of any of examples 71 to 86, wherein the instructions, when executed by the processor, further cause the processor to: merge the first replacement image and the second image.

[0167] Example 88: The non-transitory computer-readable storage medium of any of examples 71 to 87, wherein the instructions, when executed by the processor, further cause the processor to: determine at least one further image of at least one further capture area; and fuse the first replacement image, the second image, and the at least one further image.

[0168] Example 89: The non-transitory computer-readable storage medium of example 88, wherein the instructions, when executed by the processor, further cause the processor to: fuse the first replacement image, the second image, and the at least one further image based on determining, in a predefined number of the first replacement image, the second image, and the at least one further image, spatially corresponding pixels associated with pixel coding information indicating a predefined property.

[0169] Example 90: The non-transitory computer-readable storage medium of any of examples 88 to 89, wherein the instructions, when executed by the processor, further cause the processor to: merge the first replacement image, the second image, and the at least one further image based on determining, in each of the first replacement image, the second image, and the at least one further image, spatially corresponding pixels associated with pixel coding information indicating a predefined property.

[0170] Example 91: A system comprising: means for determining a first image of a first recording area, which has at least one first picture element to which first picture element coding information is assigned, which is classified as erroneous; means for determining a second image of a second recording area, wherein the first recording area and the second recording area at least partially overlap in an overlap area; means for determining a second picture element spatially corresponding to the at least one first picture element in the second image, wherein the first picture element and the second picture element represent at least a part of the overlap area;means for determining replacement pixel coding information for the first pixel of the first image using second pixel coding information associated with the second pixel of the second image and not classified as erroneous; and means for assigning the determined replacement pixel coding information to the first pixel of the first image to obtain a first replacement image.

[0171] Example 92: The system of example 91, wherein the first image is a first three-dimensional image optionally including range information; and / or wherein the second image is a second three-dimensional image optionally including range information.

[0172] Example 93: The system of any of examples 91 to 92, wherein the first image is a 3D point cloud or a depth image; and / or wherein the second image is a 3D point cloud or a depth image.

[0173] Example 94: The system of any one of examples 91 to 93, further comprising: means for determining the at least one first pixel of the first image as a distribution of measured values, optionally as a mean with variance or as an outlier; and / or means for determining the at least one second pixel of the second image as a distribution of measured values, optionally as a mean with variance or as an outlier.

[0174] Example 95: The system of any one of examples 91 to 94, further comprising: the means for determining the second pixel spatially corresponding to the at least one first pixel in the second image, wherein the first pixel and the second pixel represent at least the portion of the overlap region, wherein determining comprises performing a coordinate transformation of the first pixel from a coordinate system of the first image to a coordinate system of the second image.

[0175] Example 96: The system of example 5, further comprising: means for determining the coordinate transformation.

[0176] Example 97: The system of any one of examples 91 to 96, further comprising: means for classifying the first pixel coding information associated with the first pixel as erroneous if it corresponds to a predefined value.

[0177] Example 98: The system of any one of examples 91 to 97, further comprising: means for determining the first image of the first recording area based on image data from a first imaging device; and / or means for determining the second image of the second recording area based on image data from a second imaging device.

[0178] Example 99: The system of example 98, further comprising: the first imaging device and / or the second imaging device.

[0179] Example 100: The system of any one of examples 98 to 99, further comprising: means for determining a spatial relationship between the first imaging device and the second imaging device; means for determining the second image element spatially corresponding to the at least one first image element in the second image, wherein the first image element and the second image element represent at least a portion of the overlap region, using the determined spatial relationship between the first imaging device and the second imaging device.

[0180] Example 101: The system of example 100, wherein: performing the coordinate transformation of the first pixel from the coordinate system of the first image to the coordinate system of the second image is performed using the determined spatial relationship between the first imaging device and the second imaging device.

[0181] Example 102: The system of any of examples 98 to 101, wherein the first imaging device is a first camera, a first LiDAR camera, or a first depth camera; and / or wherein the second imaging device is a second camera, a second LiDAR camera, or a second depth camera.

[0182] Example 103: The system of any one of examples 91 to 102, further comprising: means for associating a replacement flag indicating that the determined replacement pixel coding information is associated with the first pixel of the first image to the first pixel of the first image to which the replacement pixel coding information is associated.

[0183] Example 104: The system of any of examples 91 to 103, further comprising: means for determining a first result image of the first capture area based on the first replacement image; and means for determining a second result image of the second capture area based on the second image.

[0184] Example 105: The system of example 104, further comprising: the means for determining the first result image of the first recording area based on the first replacement image by performing an outlier analysis on a further first image of the first recording area; and the means for determining the second result image of the second recording area based on the second image by performing an outlier analysis on a further second image of the second recording area.

[0185] Example 106: The system according to example 103 and example 104, further comprising: the means for determining the first result image of the first recording area based on the first replacement image by performing an outlier analysis on a further first image of the first recording area, omitting the first image element of the first image to which the replacement marker is assigned; and the means for determining the second result image of the second recording area based on the second image by performing an outlier analysis on a further second image of the second recording area.

[0186] Example 107: The system according to any one of examples 91 to 103, further comprising: means for determining a first result image of the first recording area by: determining replacement pixel coding information based on the determined replacement pixel coding information associated with the first pixel of the first replacement image; updating the first replacement image by assigning the determined replacement pixel coding information to the first pixel of the first replacement image; and determining the first result image based on the updated first replacement image, optionally by performing an outlier analysis on a further first image of the first recording area; and means for determining a second result image of the second recording area based on the second image, optionally by performing an outlier analysis on a further second image of the second recording area.

[0187] Example 108: The system of any one of examples 105 to 107, further comprising: means for determining the further first image and / or means for determining the further second image.

[0188] Example 109: The system of any one of examples 105 to 108, wherein the further first image is a further first three-dimensional image optionally comprising distance information; and / or wherein the further second image is a further second three-dimensional image optionally comprising distance information.

[0189] Example 110: The system of any of examples 105 to 109, wherein the further first image is a 3D point cloud or a depth image; and / or wherein the further second image is a 3D point cloud or a depth image.

[0190] Example 111: The system of any one of examples 104 to 110, further comprising: means for fusing the first result image and the second result image.

[0191] Example 112: The system of any one of examples 104 to 111, further comprising: means for determining at least one further result image of at least one further recording area and means for fusing the first result image, the second result image, and the at least one further result image.

[0192] Example 113: The system of any one of examples 104 to 112, further comprising: means for fusing the resultant images based on determining, in each of the resultant images, spatially corresponding pixels associated with pixel coding information indicating a predefined property.

[0193] Example 114: The system of any of examples 104 to 113 when combined with example 103, further comprising: means for fusing the resultant images based on determining spatially corresponding pixels associated with pixel coding information indicating a predefined property in each of the resultant images, except for spatially corresponding pixels determined based on the first pixel of the first image associated with a replacement marker.

[0194] Example 115: The system of any one of examples 104 to 113, further comprising: means for fusing the resultant images based on determining, in a predefined number of the resultant images, spatially corresponding pixels associated with pixel coding information indicating a predefined property.

[0195] Example 116: A system comprising: means for determining a first image of a first recording area, which has at least one first picture element to which first picture element coding information is assigned, which is classified as erroneous; means for determining a second image of a second recording area, wherein the first recording area and the second recording area at least partially overlap in an overlap area; means for determining a second picture element spatially corresponding to the at least one first picture element in the second image, wherein the first picture element and the second picture element represent at least a part of the overlap area;means for determining replacement pixel coding information for the first pixel of the first image using second pixel coding information associated with the second pixel of the second image and classified as defective; and means for discarding the first pixel coding information associated with the first pixel based on the determined replacement pixel coding information to obtain a first replacement image.

[0196] Example 117: The system of example 116, further comprising: means for determining the first image as a first result image, optionally by performing an outlier analysis on a further first image of the first capture area; and means for determining the second image as a second result image, optionally by performing an outlier analysis on a further second image of the second capture area.

[0197] Example 118: The system of example 117, further comprising: means for determining the further first image; and / or means for determining the further second image.

[0198] Example 119: The system of any one of examples 117 to 118, wherein the further first image is a further first three-dimensional image optionally comprising distance information; and / or wherein the further second image is a further second three-dimensional image optionally comprising distance information.

[0199] Example 120: The system of any of examples 117 to 119, wherein the further first image is a 3D point cloud or a depth image; and / or wherein the further second image is a 3D point cloud or a depth image.

[0200] Example 121: The system according to any one of examples 117 to 120, further comprising: means for determining the further first image of the first recording area based on image data from a first imaging device; and / or means for determining the further second image of the second recording area based on image data from a second imaging device.

[0201] Example 122: The system of any one of examples 116 to 121, further comprising: means for determining the first image of the first recording area based on image data from a / the first imaging device; and / or means for determining the second image of the second recording area based on image data from a / the second imaging device.

[0202] Example 123: The system of any of examples 121 to 122, further comprising: the first imaging device and / or the second imaging device.

[0203] Example 124: The system of any of examples 121 to 123, wherein the first imaging device is a first camera, a first LiDAR camera, or a first depth camera; and / or wherein the second imaging device is a second camera, a second LiDAR camera, or a second depth camera.

[0204] Example 125: The system of any of Examples 121 to 124, further comprising: means for determining a spatial relationship between the first imaging device and the second imaging device; and means for determining the second image element spatially corresponding to the at least one first image element in the second image, wherein the first image element and the second image element represent at least a portion of the overlap region, using the determined spatial relationship between the first imaging device and the second imaging device.

[0205] Example 126: The system of any one of examples 116 to 125, further comprising: means for determining the second image element spatially corresponding to the at least one first image element in the second image, wherein the first image element and the second image element represent at least the portion of the overlap region, wherein determining comprises performing a coordinate transformation of the first image element from a coordinate system of the first image to a coordinate system of the second image.

[0206] Example 127: The system of example 126, further comprising: means for determining the coordinate transformation.

[0207] Example 128: The system of any of Examples 126 to 127 when combined with Example 125, wherein: performing the coordinate transformation of the first pixel from the coordinate system of the first image to the coordinate system of the second image is performed using the determined spatial relationship between the first imaging device and the second imaging device.

[0208] Example 129: The system of any one of Examples 116 to 128, further comprising: means for classifying the first pixel coding information associated with the first pixel as erroneous by determining that the first pixel coding information associated with the first pixel is determined based on a first output pixel of a first output image, wherein the first output pixel spatially corresponds to the first pixel, and wherein the first output pixel is associated with first output pixel coding information that is classified as erroneous;and / or means for classifying the second pixel coding information associated with the second pixel as erroneous by determining that the second pixel coding information associated with the second pixel is determined based on a second output pixel of a second output image, wherein the second output pixel spatially corresponds to the second pixel, and wherein the second output pixel is associated with second output pixel coding information that is classified as erroneous.;

[0209] Example 130: The system of example 129, further comprising: means for classifying the first output pixel coding information associated with the first output pixel as erroneous if it corresponds to a first predefined value; and / or means for classifying the second output pixel coding information associated with the second output pixel as erroneous if it corresponds to a second predefined value.

[0210] Example 131: The system of any one of examples 129 to 130, wherein the first output image is a first three-dimensional output image optionally including range information; and / or wherein the second output image is a second three-dimensional output image optionally including range information.

[0211] Example 132: The system of any of examples 129 to 131, wherein the first output image is a 3D point cloud or a depth image; and / or wherein the second output image is a 3D point cloud or a depth image.

[0212] Example 133: The system of any of examples 116 to 132, further comprising: means for fusing the first replacement image and the second image.

[0213] Example 134: The system of any of examples 116 to 133, further comprising: means for determining at least one further image of at least one further capture area; and means for fusing the first replacement image, the second image, and the at least one further image.

[0214] Example 135: The system of example 134, further comprising: means for fusing the first replacement image, the second image, and the at least one further image based on determining spatially corresponding pixels associated with pixel coding information indicating a predefined property in a predefined number of the first replacement image, the second image, and the at least one further image.

[0215] Example 136: The system of any one of examples 134 to 135, further comprising: means for fusing the first replacement image, the second image, and the at least one further image based on determining, in each of the first replacement image, the second image, and the at least one further image, spatially corresponding pixels associated with pixel coding information indicating a predefined property.

Claims

[1] System (700, 800), which has: a processor (702, 802) configured to: to determine a first image of a first recording area having at least one first pixel to which a first pixel coding information is assigned that is classified as faulty; to determine a second image of a second recording area, wherein the first recording area and the second recording area at least partially overlap in an overlap area; to determine a second image element spatially corresponding to the at least one first image element in the second image, wherein the first image element and the second image element represent at least a part of the overlap area; to determine replacement pixel coding information for the first pixel of the first image using second pixel coding information associated with the second pixel of the second image and not classified as erroneous; and to assign the determined replacement pixel coding information to the first pixel of the first image in order to obtain a first replacement image. [2] The system (700, 800) of claim 1, wherein the processor (702, 802) is further configured to: to assign a replacement mark to the first picture element of the first image to which the replacement picture element coding information is assigned, which mark indicates that the determined replacement picture element coding information is assigned to the first picture element of the first image. [3] System (700, 800) according to any one of claims 1 to 2, wherein the processor (702, 802) is further configured to: to determine a first result image of the first recording area based on the first replacement image; and to determine a second result image of the second recording area based on the second image. [4] System (700, 800) according to claim 2 and claim 3, wherein the processor (702, 802) is further configured to: to determine the first result image of the first recording area based on the first replacement image by performing an outlier analysis on a further first image of the first recording area, omitting the first image element of the first image to which the replacement marking is assigned; and to determine the second result image of the second recording area based on the second image by performing an outlier analysis on a further second image of the second recording area. [5] System (700, 800) according to any one of claims 1 to 4, wherein the processor (702, 802) is further configured to: to determine a first result image of the first recording area by: Determining replacement pixel coding information based on the determined replacement pixel coding information associated with the first pixel of the first replacement image; Updating the first replacement image by assigning the determined replacement pixel coding information to the first pixel of the first replacement image; and Determining the first result image based on the updated first replacement image, optionally by performing an outlier analysis on another first image of the first acquisition area; and to determine a second result image of the second recording area based on the second image, optionally by performing an outlier analysis on a further second image of the second recording area. [6] System (700, 800) according to any one of claims 3 to 5 when combined with claim 2, wherein the processor (702, 802) is further configured to: to fuse the resultant images based on determining spatially corresponding picture elements associated with picture element coding information indicating a predefined property in each of the resultant images, with the exception of spatially corresponding picture elements determined based on the first picture element of the first image associated with a replacement marker. [7] Non-transitory computer-readable storage medium that stores instructions that, when executed by a processor, cause the processor to: to determine a first image of a first recording area having at least one first pixel to which a first pixel coding information is assigned that is classified as faulty; to determine a second image of a second recording area, wherein the first recording area and the second recording area at least partially overlap in an overlap area; to determine a second image element spatially corresponding to the at least one first image element in the second image, wherein the first image element and the second image element represent at least a part of the overlap area; to determine replacement pixel coding information for the first pixel of the first image using second pixel coding information associated with the second pixel of the second image and not classified as erroneous; and to assign the determined replacement pixel coding information to the first pixel of the first image in order to obtain a first replacement image. [8] System (700, 800), which has: a processor (702, 802) configured to: to determine a first image of a first recording area having at least one first pixel to which a first pixel coding information is assigned that is classified as faulty; to determine a second image of a second recording area, wherein the first recording area and the second recording area at least partially overlap in an overlap area; to determine a second image element spatially corresponding to the at least one first image element in the second image, wherein the first image element and the second image element represent at least a part of the overlap area; to determine replacement pixel coding information for the first pixel of the first image using second pixel coding information associated with the second pixel of the second image and classified as erroneous; and based on the determined replacement pixel coding information, to discard the first pixel coding information associated with the first pixel to obtain a first replacement image. [9] The system (700, 800) of claim 8, wherein the processor (702, 802) is further configured to: to classify the first pixel coding information associated with the first pixel as erroneous by determining that the first pixel coding information associated with the first pixel is determined based on a first output pixel of a first output image, wherein the first output pixel spatially corresponds to the first pixel, and wherein the first output pixel is associated with first output pixel coding information that is classified as erroneous; and / or to classify the second pixel coding information associated with the second pixel as erroneous by determining that the second pixel coding information associated with the second pixel is determined based on a second output pixel of a second output image, wherein the second output pixel spatially corresponds to the second pixel, and wherein the second output pixel is associated with second output pixel coding information that is classified as erroneous. [10] Non-transitory computer-readable storage medium that stores instructions that, when executed by a processor, cause the processor to: to determine a first image of a first recording area having at least one first pixel to which a first pixel coding information is assigned that is classified as faulty; to determine a second image of a second recording area, wherein the first recording area and the second recording area at least partially overlap in an overlap area; to determine a second image element spatially corresponding to the at least one first image element in the second image, wherein the first image element and the second image element represent at least a part of the overlap area; to determine replacement pixel coding information for the first pixel of the first image using second pixel coding information associated with the second pixel of the second image and classified as erroneous; and based on the determined replacement pixel coding information, discard the pixel coding information associated with the first pixel to obtain a first replacement image.

Images