Method and control unit for identifying individual object outlines in a detection area of ​​a sensor

Abstract

The invention relates to a computer-implemented method 800 for identifying individual object outlines 155, 110a, 110b, 110c in a detection range 132 of a sensor 130, wherein the method 800 comprises a step of reading 810 an image 125 representing an object outline 140 detected by the sensor 130 in the detection range 132. Furthermore, the method 800 comprises a step of segmenting 820 the object outline 140 into a plurality of individual object outlines 155, 110a, 110b, 110c. Finally, the method 800 comprises a step of comparing 830 the individual object outlines 155, 110a, 110b, 110c with individual object information 165, which represents an outer shape of the individual object outlines 155, 110a, 110b, 110c and is retrieved from a memory 170, in order to identify the individual object outlines 155, 110a, 110b, 110c if the individual object outlines 155, 110a, 110b, 110c segmented from the object outline 140 meet a predefined criterion.

Description

State of the art

[0001] The present invention relates to a computer-implemented method for identifying individual object outlines in a detection range of a sensor, to a corresponding control unit, to a corresponding computer program product, and to a machine-readable storage medium.

[0002] In warehouse logistics, typically several small items (also called an item or single object) are packed into a large carton (also called a package or container) for shipping. Packing one or more items into the package at once is called an insert or object. Here, the number of items contained in an insert should be automatically detected, allowing only cube-shaped or rectangular items of the same type and size. If the insert consists of more than one element, it can be in the shape of a simple rectangle, which is called a block. It could also form an L- or T-shape, which is called a cross-shaped insert. Disclosure of the invention

[0003] Against this background, the approach presented here presents a computer-implemented method for identifying individual object outlines in a detection range of a sensor, furthermore a control unit that uses this method and finally a corresponding computer program product according to the main claims.

[0004] Advantageous embodiments emerge from the respective subclaims and the following description.

[0005] The approach presented here creates a computer-implemented method for identifying individual object outlines in a detection area of ​​a sensor, the method having the following features: - Reading in an object outline representing an object outline detected by a sensor in the detection area; - Segmenting the object outline into a plurality of individual object outlines; and - Comparing the single object outlines with single object information representing an external shape of the single object outlines and retrieved from a memory to identify the single object outlines if the single object outlines segmented from the object outline meet a predefined criterion.

[0006] Preferably, the steps of the proposed method are carried out by means of a control unit or a computing unit or a computer.

[0007] In this case, an image can be understood as, for example, an image taken by a camera of a detection area on which, for example, a container or package is lying, into which objects or inserts are to be placed. An object outline can be understood as, for example, a shape or a polygon that is recognized as coherent in the image, which is assumed to be the image of several individual objects that have been placed on top of one another and can therefore be captured or interpreted in the image as a coherent object. An individual object outline can be understood as, for example, a shape or an image that is assigned to a single object that can, for example, be placed separately in a container and can lie next to other individual objects, so that these objects can be interpreted together as a (complete) object by an image processing algorithm.Individual object information can, for example, be understood as information regarding the size, width, height, or general shape of the individual objects that are to be placed in a container and can then be recognized from the image as a coherent (overall) object by an image processing algorithm. For example, the individual object information can indicate how many individual objects of which size and shape are to be expected, so that the segmentation of the object outline into the plurality of individual object outlines can be checked in the comparison step in such a way that the individual object outlines are identified if, for example, a size and / or number of individual object outlines segmented from the object outline corresponds, as a predefined criterion, to a size and / or number of individual object outlines expected from the individual object information.

[0008] The approach presented here offers the advantage that, through efficient image processing of an object outline captured and identified in an image by a technically simple optical sensor such as a camera, the number and / or size of individual objects that may be contained in this object outline can be determined using very simple technical means. If, for example, it is detected that an expected number and / or size of individual objects was detected in the container after segmenting the object outline, this can be used to verify a packing process for a logistics service provider. In addition to a camera, which is often already available, this requires only minor reconfiguration of existing image processing software, which can be implemented technically simply and flexibly.This allows for significant efficiency gains in the verification of a delivery to be completed in the logistics area.

[0009] A particularly advantageous embodiment of the approach proposed here is one in which the segmentation and comparison steps are repeated at least once, whereby in the repeated segmentation step, the object outline is segmented in a different way than in the previous segmentation step.In a first step of segmenting the object outline into a plurality of individual object outlines, the object outline can be segmented along at least one edge of the object outline, which edge is extrapolated through at least one projection of the object outline in order to obtain the individual object outlines through the parts of the object outline obtained from the segmentation, and in a second step of segmentation, the edge can be extended along at least one outer boundary of the object outline through a partial region of the object outline in order to obtain the individual object outlines through parts of the object outline obtained from the segmentation, or vice versa. Such an embodiment offers the advantage of being able to carry out several different segmentations of the object outline in order to obtain a different number and / or size of the individual object outlines from the object outline.It is also conceivable that, in a first segmentation step, a line-by-line evaluation of the image is evaluated for brightness differences in order to detect at least one edge, and, in a second segmentation step, a column-by-column evaluation of the image is evaluated for brightness differences in order to detect at least one edge, or vice versa. These different sizes can thus compensate for possible errors in the segmentation of the object outline, which, for example, are permissible but do not correspond to the expected individual objects in terms of number and / or size and / or shape.

[0010] Another advantageous embodiment of the approach proposed here is one in which, in the reading step, an object outline is read in that represents an L-shaped or a T-shaped object outline. Another advantageous embodiment of the approach proposed here is one in which

[0011] According to a further embodiment of the approach proposed here, in the segmentation step, the object outline can be segmented along at least one edge of the object outline, which is extrapolated through at least one projection of the object outline to obtain the individual object outlines using the parts of the object outline obtained from the segmentation. Alternatively or additionally, the edge can be extended along at least one outer boundary of the object outline through a partial region of the object outline to obtain the individual object outlines using parts of the object outline obtained from the segmentation.Such an embodiment of the approach proposed here offers the advantage of a technically very simple and reliable possibility of segmenting the object outline into corresponding individual object outlines, which often also corresponds to an actual orientation and / or size of individual objects that are recognized as the object outline when lying together.

[0012] Another possible embodiment of the approach proposed here is one in which, in the segmentation step, a row-by-row and / or column-by-column analysis of the image is evaluated for brightness differences in order to detect at least one edge. Such an embodiment offers the advantage of creating a segmentation with high precision in the detection of individual outlines that actually represent adjacent individual objects through a technically very simple variant of image processing or edge detection.

[0013] According to another embodiment of the approach proposed here, the object outline can be read by an optical sensor in the reading step, in particular, wherein the object outline represents an object in the interior of a container arranged in the detection area and / or an object arranged in the detection area. Such an embodiment offers the advantage of being able to assign the detected object outline to an actual object, which appears in the image as a whole due to the creation of several individual objects, so that the individual objects can be reliably recognized based on their corresponding object outlines through segmentation.

[0014] Another conceivable embodiment of the approach presented here is one in which the individual object outlines are recognized in the comparison step if the individual object outlines segmented from the object outline correspond, within a tolerance range, to the external dimensions predefined by the individual object information. For example, individual object outlines can be identified if they deviate by no more than ten percent, or at least 20 percent, from the external dimensions predefined by the individual object information. Such an embodiment offers the advantage of still being able to reliably recognize the corresponding individual objects, for example in the event of individual damage or changes in shape of the respective individual objects compared to the expected external dimensions of the individual objects defined by the individual object information.

[0015] Particularly advantageous, as it is technically simple to implement, is an embodiment of the approach proposed here in which, in the comparison step, information about rectangular external dimensions of packages is read in as individual object outlines, in particular where the individual object outlines correspond to packages that are to be placed in a container. Such an embodiment offers the advantage of being able to recognize such shapes of external dimensions in an image in a technically simple and precise manner, so that for most applications in which the individual objects are actually rectangular or cuboid-shaped, these can be extracted from the object outline and clearly identified as individual objects.

[0016] A particularly advantageous embodiment of the approach proposed here is one in which the individual object outlines are recognized in the comparison step if a piece of individual object information can be assigned to each individual object outline segmented from the object outline. Such an embodiment offers the advantage of being able to quickly and unambiguously determine the actual composition of the object from individual objects based on the individual object outlines contained in the object outline through the predetermined assignment of the individual object outlines expected from the individual object information and the individual object outlines obtained through segmentation.

[0017] According to a further embodiment of the approach proposed here, a segmentation of the object outline into individual object outlines can be discarded in the comparison step if the individual object outlines segmented from the object outline do not meet the predefined criterion. Such an embodiment offers the advantage of being able to abandon a segmentation that does not correspond to actual conditions and subsequently focus on other segmentation solutions that are likely to be closer to reality. This reduces numerical or circuit complexity.

[0018] According to a further embodiment, the approach presented here can be used particularly in a logistics facility in which many identical or similar packaging processes are to be monitored. For this purpose, according to such an embodiment, the steps of reading, segmenting, and comparing can be carried out repeatedly, wherein a further object outline is read in, which represents a further object detected by a sensor in the detection area, wherein the further object outline is segmented into further individual object outlines, and the further individual object outlines are compared with further individual object information, which represents an external shape of the further individual object outlines and is retrieved from the memory in order to identify the further individual object outlines if the further individual object outlines segmented from the further object outline meet a predefined criterion.

[0019] The additional object outline can be read in from an image and depict a new packing process in which, for example, additional individual objects with the corresponding additional individual object outlines are packed together in a container such as a carton, and which then together form the additional object outline. However, it is not necessary for the object outline or the individual object outlines or the individual object information to correspond to the additional object outline, the additional individual object outlines, or the additional individual object information. Rather, different packing processes can also be monitored if, for example, the additional individual object information specifies different numbers and / or sizes of additional individual objects than those defined by the individual object information in a previous run of the method presented here.Such an embodiment thus offers the advantage of being able to efficiently monitor different packing processes with the approach presented here, whereby only the corresponding individual object information needs to be provided, for example, which number and / or size of individual objects are to be placed in a container.

[0020] The approach presented here further provides a control unit configured to perform or implement the steps of a variant of a method presented here in corresponding devices. This embodiment of the invention in the form of a control unit also allows the problem underlying the invention to be solved quickly and efficiently.

[0021] In this case, a control unit can be understood as an electrical device that processes sensor signals and outputs control and / or data signals depending on them. The control unit can have an interface that can be implemented in hardware and / or software. In a hardware implementation, the interfaces can, for example, be part of a so-called system ASIC, which contains a wide variety of functions of the control unit. However, it is also possible for the interfaces to be separate integrated circuits or to consist at least partially of discrete components. In a software implementation, the interfaces can be software modules that are present, for example, on a microcontroller alongside other software modules.

[0022] Also advantageous is a computer program product with program code that can be stored on a machine-readable storage medium or a machine-readable carrier, such as a semiconductor memory, a hard disk memory, or an optical memory, and is used to carry out the steps of the method according to one of the embodiments described above when the program product is executed on a computer or a control unit. The machine-readable storage medium with the computer program product stored thereon is also advantageous.

[0023] The approach presented here thus enables the division of cross-shaped insets into blocks for the detection of the elements within the inset. The approach presented here enables reproducible and precise detection of objects in the cross-shaped insets, with robust results that do not depend on any parameters (except for noise removal).

[0024] The approach presented here is explained in more detail below using the attached drawings. They show: Fig. 1 is a schematic representation of an application scenario in which an embodiment of a control unit is used to analyze the contents of a container; Fig. 2 representations of a T-shaped insert, which as an object is composed of four parts or individual objects and from which an object outline was obtained; Fig. 3 representations of two ways to divide the object outline into blocks or individual object outlines for the individual objects; Fig. 4a-e Representations of the procedure for segmenting an object outline using different decision thresholds; Fig. 5a-l Representations of individual object outlines obtained by different procedures, from which an object outline can be composed; Fig. 6a-d representations of individual object outlines obtained using different procedures, from which an object outline can be composed; Fig. 7a-c Representations of different segmentation solutions for an object outline; and Fig. 8 a flow diagram of an embodiment of the approach presented here as a method for identifying individual object outlines in a detection area of ​​a sensor.

[0025] In the following description of advantageous embodiments of the present invention, the same or similar reference numerals are used for the elements shown in the various figures and having a similar effect, whereby a repeated description of these elements is omitted.

[0026] Fig. 1 shows a schematic representation of an application scenario in which an embodiment of a control unit 100 is used to analyze the contents of a container 105. The container 105 can, for example, be a package and a box into which individual objects 110a, 110b, and 110c have been placed through an opening 115, which, for example, come to rest on a base 120 of the container 105. The individual objects 110 can be individual packages or containers, such as small, rectangular packages or the like, which are to be packed together in a loading unit in the container 105 and subsequently shipped.The individual objects 110 are often arranged so close to one another or so touching that they appear as one object 135 having an object outline 140 in an image 125, which was recorded, for example, by a camera 130 as an optical sensor for detecting objects in the detection area 132, which is arranged above the containers 105 with a view through the opening 115. However, from this object outline 140, it is no longer possible, for example using conventional image processing algorithms, to draw conclusions about the position and / or number or size of the individual objects 110, since the specific boundaries of the individual objects 110 are sometimes not clearly identifiable. As a result, it is not possible to efficiently check using conventional means whether a sufficient number of packages orIndividual objects or the packages or individual objects specifically requested by a customer were packed together in the container 105, so that a complaint or claim may possibly be expected after shipping an incorrectly filled container 105.

[0027] To avoid such a situation, according to the approach presented here, the image 124 or an object outline 140 determined therefrom is read into the control unit 100 via a read-in interface 145 and fed to a segmentation unit 150. In the segmentation unit 150, the object outline 140 is then segmented into a plurality of individual object outlines 155, which can, for example, correspond to borders or shapes of the outer boundaries of the individual objects 110a, 110b, and 110c and can thus also be synonymous with individual objects 110a, 110b, and 110c.In order to check whether the individual object outlines 155 specifically obtained from the object outline 140 also correspond to the individual object outlines that correspond to the individual objects 110a, 110b, and 110c to be sorted into the container 105, the individual object outlines 155 obtained in the segmentation unit 150 are compared in a comparison unit 160 with individual object information 165 that is loaded from a memory 170. This individual object information 165 represents the expected external dimensions or shapes of individual objects 110 that are to be packed into the container 105 during the current packing process.If the individual object outlines 155 supplied by the segmentation unit 150 correspond (for example, within a tolerance range of ten percent) as a predefined criterion in number and / or size to the outlines, dimensions, and / or shapes of the individual objects that are to be packed into the container 105 during the current packing process and that are provided by the individual object information of the comparison unit 160, it can be assumed that the correct number and / or type of individual objects 110 was packed into the container 105. In this case, a corresponding identification signal 172 can be output, which represents a correct identification of the individual object outlines.

[0028] However, if the comparison unit 160 detects that the individual object outlines 155 supplied by the segmentation unit 150 do not correspond to the predefined criterion, i.e., the number and / or size of the outlines, dimensions and / or shapes of the individual objects 110 to be parked in the container 105 during the current packing process, it can be assumed that either the segmentation of the object outline 140 was incorrect or that the correct number or the correct individual objects have not yet been placed in the container 105.To ensure that an unfavorable segmentation of the object outline 140 does not lead to an error message, the comparison unit 160 can control the segmentation unit 150 in such a way that a new segmentation of the object outline 140 is carried out, but this time a different segmentation is carried out, so that other individual object outlines 155 are obtained, which are then subsequently compared analogously in the comparison unit 160 with the corresponding individual object information 165. In this way, with technically simple means of adapting an image processing algorithm for carrying out the segmentation in the segmentation unit 150, a technically simple check can be made possible as to whether a correct number and / or type of individual objects were placed as packages in the container 105 as a parcel or box.If this is not the case, a warning can be issued, for example, to instruct a logistics employee to check the contents of container 105.

[0029] The approach presented here can thus be used, for example, in conjunction with a (for example, 3D) camera 130 for monitoring the packaging process. The (3D) camera, which provides both color and depth information, is mounted above a packing table, for example, and looks down onto the package or container 105. Based on the depth information from the image 125 of the camera 130, an insert, for example, is identified as a flat area or as an object 135 with the object outline 140, since its depth differs from the rest of the package contents. The detection of elements or individual objects 110 in an insert or the object 135 can, for example, comprise three steps: (1) Determine whether the insertion or object 135 is already a block, or two touching blocks that share a common corner, or a cross-shaped block. (2) If the insertion does not consist of a single block, the blocks are split. (3) For each block, determine the number of elements and their arrangement in the block using an exhaustive search. This approach, known as the legacy method, fails for insertions or objects 135 of a wide T-shape, as is the case, for example, in the Fig. 1: At the two shorter edges of its rotated rectangle, the number of white pixels is usually less than 80% of the scan line length. Therefore, the legacy method immediately reverts and assumes that there is no new block. The problem here is that the hard parameter of 80% doesn't always work, and the white noise that could affect the scan result is also not taken into account.

[0030] As mentioned above, the present approach introduces a new method for separating cross-shaped (or T- or L-shaped) inlays, objects, or object outlines into blocks or single object outlines. It consists of the following steps: a) Work out the possible solutions. b) Evaluate the possible solutions. c) Choose the solution.

[0031] The procedure presented here is explained in more detail below using an example.

[0032] Fig. Figure 2 shows, in the left-hand illustration, a T-shaped inset, which is composed of four parts or individual objects 110 as object 135. The foreground image of the inset or object 135, which is obtained by thresholding the depth image, is marked by the white area. The black background is the parcel content. In the right-hand illustration of the Fig. 3 thus depicts an object outline 140 obtained from an image processing algorithm by processing the image taken with the camera, as shown in the left illustration.

[0033] As in the two representations of the Fig. 3, there are two ways to divide the foreground image or the object outline 140 into blocks or individual object outlines 155 for the individual objects 110: either by cutting the projection 300 along the longer edge 305, creating two blocks, or along the shorter edge 310, which results in the correct solution with three blocks or individual objects 110. Here, each block or individual object 110 is highlighted with a different hatching. The next steps describe in detail how such a segmentation of the object outline 140 is carried out.

[0034] It can be assumed that a "cutter" is used to divide the foreground image or object outline 140 into blocks or individual object outlines 155. Since, for example, morphological image processing is used here, not only the orientation of the cutter is required, but also its size in order to separate the blocks correctly and efficiently.

[0035] The rotated rectangle of the overlay is defined as the minimum rotated rectangle that encloses the foreground image of the overlay. The angle of the rotated rectangle is the degree between the horizontal axis and the first edge of the rectangle in a clockwise direction. The lowest point of the rotated rectangle is its first vertex. The second, third, and fourth vertices follow in a clockwise direction. The height alignment connects the first and second vertices, while the width alignment connects the second and third points. The following procedure can then be applied: a-1) Determine the size of the cutting tool

[0036] Fig. Figure 4a shows how the foreground image of the overlay can first be cropped using its rotated rectangle as an object outline 140. It is then rotated upright so that the next image processing can be performed directly and efficiently on the image matrix. In the upright rectangle of this example, the height alignment is the vertical direction, and the width alignment is the horizontal direction. a-1-1) Determine the size of the cutting tool in height orientation

[0037] To determine the size of the cutting tool in height orientation, one can assume that, for example, in height orientation, a "cut" is desired that divides the image into two regions, one with many white pixels and the other with few. This is achieved by scanning the foreground image from left to right (i.e., row by row) and counting the number of non-zero pixels per column.

[0038] Fig. Figure 4b shows a representation in which points are plotted with such a number of non-zero pixels in the row direction. Note that the left part of the image consists of 400 points with large values, since in this region white pixels extend from top to bottom. On the other hand, the points in the right part have a lower value, since here the white pixels are only present in the center. To divide the points into only two regions, they are first sorted by their value. Here, on the vertical axis, 400 points correspond to the number of non-zero pixels per column in the foreground image. The horizontal axis shows the width of the foreground image, with line 410 representing the achieved threshold.

[0039] Fig. Figure 4c shows a representation of such a subdivision, represented by the white dots 400 in Figure a-2. The threshold 410 that separates the white dots 400 into two regions can be determined, for example, using the Otsu method. On the vertical axis, the white dots represent the ascending number of non-zero pixels per column in the foreground image, with the horizontal axis representing the width of the foreground image and the line 410 representing the threshold resulting from the separation of the white dots.

[0040] The threshold 410 is used in both Fig. 4b and Fig. 4c by the horizontal line, where the threshold value 410 corresponds to the distance from the image floor to the line 410. When "cutting" the overlay in the vertical alignment with a cutter of this size, the overlay is divided into a left part with white pixels extending from top to bottom and a right part with white pixels present only in the center. a-1-2) Determine the size of the cutting tool in width orientation

[0041] Similarly, when scanning the foreground image from top to bottom, the number of non-zero pixels is first recorded.

[0042] Fig. Figure 4d shows a diagram illustrating this number of non-zero pixels by the points 420. Here, the similarity between the shape of the points 420 and that of the original foreground image or the object outline 140 from Fig. 4a can be recognized.

[0043] For example, the division of the points 420 into two regions according to their horizontal coordinates is estimated by first sorting them in ascending order. These are indicated by the white points in Fig. 4d. The separation threshold 410 is also optimally determined using the Otsu method and is shown in both figures Fig. 4d and Fig. 4e represented by line 410.

[0044] In the diagram from the Fig. 4d, the points are represented on the vertical axis as the number of non-zero pixels per line in the foreground image and on the horizontal axis the width of the image in the foreground or the object outline 140. The line 410 represents the achieved threshold.

[0045] In the diagram from the Fig. 4e, on the vertical axis, the points represent an increasingly sorted number of non-zero pixels per row in the foreground image or object outline 140, and on the horizontal axis, a height of the foreground image or object outline 140. The line 410 represents the threshold resulting from the separation of the white points.

[0046] After determining the cutter size for an orientation, the structuring element (SE) of the next morphological image processing step is created as an image containing a single line whose angle is the orientation angle. The length of the line corresponds (for example) exactly to the size of the cutout. The image size is calculated by projecting the line onto the two axes, plus a padding of two pixels at each image edge. a-2) Separating the foreground image or object outline 140 into the potential blocks or individual object outlines 110

[0047] The block separation or segmentation is tested in both the height and width alignment. a-2-1) Block separation in height orientation

[0048] First, the holes or black noise in the foreground source image are processed by closing them with a line structuring element (LSE). This is useful when there are unwanted spaces or gaps between the elements.

[0049] For example, a line length can be understood as 10% of the size of the cutter in the height orientation and a line angle as an angle of the rotated rectangle + 90. In this way, a final line structuring element LSE can be obtained, as shown in the Fig. 5a (scaled for better view), wherein this line structure element 500 has a direction of the longitudinal edge of the object outline 140 according to the illustration from the Fig. 5b.

[0050] The first block is obtained by opening with a line structure element 500 or LSE in the height orientation, where a line length corresponds to a size of the cutter in the height orientation and a line angle corresponds to an angle of the rotated rectangle +90. The resulting line structure element 500 or LSE is then Fig. 5c, wherein a section along this aligned LSE to a single object 110 according to the illustration from Fig. 5d leads.

[0051] Furthermore, a slight noise can be seen here, which is caused by the depth image, which is removed by opening with an LSE: A line length is defined as the size of the cutter in the width direction and a line angle as the angle of the rotated rectangle. Noise reduction for a Fig. 5e shown line structure element 500 LSE (scaled for better view), whose application in the segmentation to a single object 110 according to the representation from the Fig. 5f leads.

[0052] To compensate for the unwanted removal, the opening or segmentation is extended by a reconstruction which is Fig. 5g. With such an approach, the segmentation is extended three times with a square structural element of size 5x5, whereby the extension is limited by the original foreground image or the object outline 140.

[0053] The remaining part of the crop results from excluding the first part or opening from the foreground. Note the presence of noise on the right side. The noise of the remaining part is removed by the same LSE that was used to clean the opening part. The single object 110' obtained after such an operation, or the corresponding single object outline, is shown in the Fig. 5h shown.

[0054] The remaining part is also enlarged by reconstruction to compensate for the unwanted distance, although not all noise is removed. The single object 110' obtained after such an operation, or the corresponding single object outline, is shown in the Fig. 5j shown.

[0055] The overlap between the remaining block and the opening is processed by giving preference to the opening, since its entire body should match the cut orientation. Finally, all separated blocks are checked to see if their area is sufficiently large, i.e., at least as large as the minimum area of ​​the block. If this is the case, they are selected as part of the potential solution. The single object 110' obtained after such an operation, or the corresponding single object outline, together with the previously detected single object 110 or single object outline, is shown in the Fig. 5l shown.

[0056] Here, the original foreground image or object outline 140 was split into two blocks or individual objects 110 or corresponding individual object outlines 155. Note that the noise has been removed here and this potential solution contains only two blocks as individual objects 110 or 110' with different hatching. a-2-2) Separation in the width orientation

[0057] According to this procedure, the same steps described in step a-2-1 are applied, except that they are performed with the cutter size in the width orientation, and the angle of the opening line structure elements 500 or LSE is the same as that of the rotated rectangle. To avoid duplication of information, only the result of the main steps for this orientation is mentioned.

[0058] After segmenting the insertion or object outline 140 in the width orientation and compensating for unwanted losses, the following procedure can be used to obtain the first individual object 110: The obtained single object 110 in the direction of the Fig. 6a shown line element 500 for segmentation in the width orientation is in the Fig. 6b shown scaled for better clarity.

[0059] The remaining part of the object outline 1401 consists of two blocks or individual objects 110, as shown in the Fig. 6c: After checking the validity of such a segmentation, three blocks are selected for the potential solution of the segmented individual objects 110, as shown in the Fig. 6d. These are the result of the separation of the insertion in the width orientation.

[0060] In summary, at the end of the segmentation, two fundamentally possible separation steps, two possible solutions of the object outline 140 into individual objects 110, appear conceivable: one with two blocks or individual objects 110 and the other with three blocks or individual objects 110.

[0061] In the Fig. 7a, Fig. 7b and Fig. 7c of the Fig. 7 are initially the object outline 140 with the individual objects 110 contained therein or closely adjacent thereto ( Fig. 7a) and the two possible segmentation results obtained with the two ( Fig. 7b) or three ( Fig. 7c) Individual objects 110 are shown.

[0062] Following this, the individual objects can be identified in the following steps, which are associated with a

[0063] Step b) of evaluating the possible solution(s) continues. Once we have identified possible segmentation solutions, they can be evaluated to select the segmentation solution with the best confidence factor. The evaluation is based, for example, on the legacy method and can be summarized as follows. For each potential (segmentation) solution (containing one or more blocks), the following steps are performed: • Determine the 3D size (width, height, depth) of each contained block, for example, in millimeters. This is done using the depth information and the camera's internal information. • Adjust all six possible orientations of the object and assign the highest confidence to these orientations. This can be done by remembering that the elements in the block can be of the same type and size. This approach can also utilize individual object information. • The confidence in the potential (segmentation) solution is the product of the confidence factors of the individual blocks.

[0064] Following this, a step c) of selecting the segmentation solution can take place.

[0065] The potential (segmentation) solution with the best confidence value is selected as the segmentation solution, so that the arrangement or number of individual objects presented here is identified as the one actually present in the object outline 140. Since the number of elements contained in the blocks and their orientation have been determined, this is also the solution for detecting the elements in a cross-shaped insertion or a T-shaped object. However, the identification of L-shaped objects is also conceivable, which can be segmented into corresponding separate blocks or individual elements and the identified elements.

[0066] The approach presented here thus enables the detection of articles or individual objects in cross-shaped (or L- and T-shaped) bays or structures extracted as object outlines from images in warehouse logistics. The approach presented here can enable reliable and precise detection of articles or individual objects in cross-shaped bays or objects in warehouse logistics. The separation of the cross-shaped or T-shaped bays or objects into blocks or individual objects is based, for example, on morphological image processing and / or optimal image thresholding. The approach presented here for identifying individual objects is based on the best possible solution among the potential solutions; it is not a parameterized heuristic.

[0067] However, it is also conceivable that not all possible segmentations of the object outline into individual object outlines need to be performed in advance. Instead, a segmentation of the object outline can be performed, which is then evaluated using the individual object information before further segmentation is performed. This approach avoids unnecessary numerical or circuitry effort for evaluating the segmentation solutions.

[0068] Fig.8 shows a flowchart of an embodiment of the approach presented here as a method 800 for identifying individual object outlines in a detection range of a sensor. The method 800 comprises a step 810 of reading in an image representing an object outline detected by a sensor in the detection range. Furthermore, the method 800 comprises a step 820 of segmenting the object outline into a plurality of individual object outlines and a step 830 of comparing the individual object outlines with individual object information representing an external shape of the individual object outlines and retrieved from a memory in order to identify the individual object outlines if the individual object outlines segmented from the object outline meet a predefined criterion.

[0069] The embodiments described and shown in the figures are selected only as examples. Different embodiments can be combined with each other completely or with regard to individual features. Furthermore, one embodiment can be supplemented by features of another embodiment.

[0070] Furthermore, the process steps presented here can be repeated and carried out in a different order than that described.

[0071] If an embodiment comprises an “and / or” link between a first feature and a second feature, this is to be read as meaning that the embodiment according to one embodiment has both the first feature and the second feature and according to another embodiment has either only the first feature or only the second feature.

Claims

[1] A computer-implemented method (800) for identifying individual object outlines (155, 110a, 110b, 110c) in a detection area (132) of a sensor (130), the method (800) comprising the following steps: - reading (810) an image (125) representing an object outline (140) detected by the sensor (130) in the detection area (132); - segmenting (820) the object outline (140) into a plurality of individual object outlines (155, 110a, 110b, 110c); and - comparing (830) the individual object outlines (155, 110a, 110b, 110c) with individual object information (165) representing an outer shape of the individual object outlines (155, 110a, 110b, 110c) and retrieved from a memory (170) in order to identify the individual object outlines (155, 110a, 110b, 110c) if the individual object outlines (155, 110a, 110b, 110c) segmented from the object outline (140) meet a predefined criterion. [2] Method (800) according to claim 1, wherein the steps (820) of segmenting and comparing (830) are carried out repeatedly at least once, wherein in the repeatedly carried out step (820) of segmenting the object outline (140) is segmented in a different way than in the preceding step (820) of segmenting. [3] Method (800) according to one of the preceding claims, wherein in the step (810) of reading in an object outline (140) is read in which represents an L-shaped or a T-shaped object outline (140). [4] Method (800) according to one of the preceding claims, wherein in the segmenting step (820) the object outline (140) is segmented along at least one edge (305, 310) of the object outline (140), which is extrapolated through at least one projection (300) of the object outline (140) in order to obtain the individual object outlines (155, 110a, 110b, 110c) through the parts (155) of the object outline (140) obtained from the segmentation and / or wherein the / an edge (305, 310) is extended along at least one outer boundary of the object outline (140) through a partial region of the object outline (140) in order to extrapolate the individual object outlines (155, 110a, 110b, 110c) through the parts (155) of the object outline (140) obtained from the segmentation of the object outline (140). [5] Method (800) according to claim 4, wherein in the segmenting step (820) a row-by-row and / or column-by-column evaluation of the image (125) is evaluated with regard to brightness differences in order to detect the at least one edge (305, 310). [6] Method (800) according to one of the preceding claims, wherein in the step (810) of reading in, the image (125) is read in by an optical sensor (130), in particular wherein the image (125) represents an interior of a container (105) arranged in the detection area (132) and / or an object outline (140) arranged in the detection area (132). [7] Method (800) according to one of the preceding claims, wherein in the step (830) of comparing the individual object outlines (155, 110a, 110b, 110c) are recognized if the individual object outlines (155, 110a, 110b, 110c) segmented from the object outline (140) correspond within a tolerance range to the external dimensions predefined by the individual object information (165). [8] Method (800) according to claim 7, wherein in the step (830) of comparing as individual object information (165) information about rectangular external dimensions of packages is read in as individual object outlines (155, 110a, 110b, 110c), in particular wherein the individual object outlines (155, 110a, 110b, 110c) correspond to packages that are to be placed in a container (105). [9] Method (800) according to claim 7 or 8, wherein in step (830) of the comparison the individual object outlines (155, 110a, 110b, 110c) are recognized if for each individual object outline (155, 110a, 110b, 110c) segmented from the object outline (140) a single object information item (165) can be assigned. [10] Method (800) according to one of the preceding claims, wherein in the step (830) of comparing, a segmentation of the object outline (140) into individual object outlines (155, 110a, 110b, 110c) is rejected if the individual object outlines (155, 110a, 110b, 110c) segmented from the object outline (140) do not meet the predefined criterion. [11] Method (800) according to one of the preceding claims, in which the steps of reading (810), segmenting (820) and comparing (830) are carried out repeatedly, wherein a further image (125) is read in which represents a further object outline (140) detected by the sensor (130) in the detection area (132), wherein the further object outline (140) is segmented into further individual object outlines (155, 110a, 110b, 110c), and the further individual object outlines (155, 110a, 110b, 110c) are compared with further individual object information (165) which represents an outer shape of the further individual object outlines (155, 110a, 110b, 110c) and which is retrieved from the memory (170) in order to determine the further individual object outlines (155, 110a, 110b, 110c) if the further individual object outlines (155, 110a, 110b, 110c) segmented from the further object outline (140) meet a predefined criterion. [12] Control unit (100) which is designed to carry out the steps (810, 820, 830) of a method (800) according to one of claims 1 to 11 in corresponding devices (145, 150, 160). [13] Computer program product with program code for carrying out and / or controlling the steps (810, 820, 830) of the method (800) according to one of claims 1 to 11, when the program product is executed on a control unit (100). [14] Machine-readable storage medium with a computer program product for carrying out and / or controlling the steps of the method (800) according to one of claims 1 to 11, when the program product is executed on a computer or a control unit (100).

Images