IMAGE PROCESSING DEVICE

Abstract

An image processing device according to one aspect of the present disclosure comprises: a three-dimensional information acquisition unit that acquires three-dimensional information, including distance information to two-dimensional positions, and a two-dimensional image from an image sensor; an image recognition unit that recognizes candidate areas of objects from the two-dimensional image; an overlap determination unit that determines whether the candidate areas overlap or not; and a distance determination unit that uses the distance information as a basis to assess whether two candidate areas, which have been determined by the overlap determination unit to be overlapping, lie in the same plane or not.If the two candidate areas are found to be in the same plane, the distance determination unit determines that the candidate area with the smaller area is part of an object shown by the candidate area with the larger area, whereas if the two candidate areas are found not to be in the same plane, the distance determination unit determines that the candidate area with the smaller area is an independent object located in front of an object shown by the candidate area with the larger area.

Description

TECHNICAL AREA

[0001] The present invention relates to an image processing device. STATE OF THE ART

[0002] To improve logistical efficiency, it is common practice to transport multiple packages stacked in several layers on a pallet. To further increase efficiency, it is desirable for the stacked packages to be automatically removed sequentially by a robot or similar device. To achieve this, a method was investigated: a distance image is captured by picking up the stacked packages from above; the planar positions and vertical relationships of the packages are analyzed using a pattern matching technique; and the packages are removed sequentially, starting with the topmost package.

[0003] Specifically, it has been proposed that features considered to be corners or edges of the packaging in an image obtained by extracting a distance region corresponding to the top layer from the distance image are recognized, and that the positions of the packages are estimated by comparing them with templates of contour information of the packages that were previously registered (see, for example, patent document 1). In a distance image, an edge may not be recognized if adjacent packages are in close contact with each other. Furthermore, information may be lost in a distance image, for example, due to irregular reflections caused by adhesive tape used to seal the packages.Accordingly, it was also suggested to perform a comparison with a shape generated by combining contour information, or to perform the comparison taking into account information losses. Bibliography of patent documents

[0004] Patent document 1: Japanese patent application, not yet examined, publication number JP H10-31742 A DISCLOSURE OF THE INVENTION Problems to be solved by the invention

[0005] The project also explores the possibility of loading different types of packaging with varying shapes and sizes onto the same pallet. In such cases, it is difficult to capture contour information for all types of packaging in advance. Therefore, an image processing technique is desirable that enables accurate packaging recognition, even when different types of packaging with varying shapes and sizes are stacked in multiple layers. Measures to solve the problems

[0006] An image processing device according to one aspect of the present disclosure comprises: a three-dimensional information acquisition unit configured to acquire three-dimensional information from a visual sensor, comprising a two-dimensional image and distance information from two-dimensional positions; an image recognition unit configured to recognize candidate areas of an object from the two-dimensional image;an overlap determination unit configured to determine whether the candidate areas overlap, and a distance determination unit configured to determine, based on the distance information, for two candidate areas where the overlap determination unit has detected an overlap, whether the two candidate areas are on the same plane, determines that if the two candidate areas are determined to be on the same plane, a candidate area with a smaller area is part of an object indicated by a candidate area with a larger area, and determines that if the two candidate areas are determined not to be on the same plane, the candidate area with the smaller area is an independent object located in front of the object indicated by the candidate area with the larger area. BRIEF DESCRIPTION OF THE DRAWINGS Fig. Figure 1 is a block diagram illustrating a configuration of a depalletizing system including an image processing device according to a first embodiment of the present disclosure; Fig. Figure 2 is a flowchart that illustrates the process of image recognition processing in the image processing device. Fig. 1 illustrated; and Fig. Figure 3 is a block diagram illustrating a configuration of a depalletizing system including an image processing device according to a second embodiment of the present disclosure. PREFERRED METHOD FOR IMPLEMENTING THE INVENTION

[0007] In the following, embodiments of the present disclosure are described with reference to the drawings. In embodiments described later, identical components are assigned the same reference numerals as those described previously, and repeated descriptions may be omitted. (First embodiment)

[0008] Fig. Figure 1 is a block diagram illustrating a configuration of a depalletizing system 1. The depalletizing system 1 is a device configured to sequentially remove a variety of objects (packages) L arranged on a pallet P. The variety of objects L removed by the depalletizing system 1 can include a mixture of objects with different shapes (length, width, and height ratios), sizes, and patterns (including adhesive tape for sealing and labels for displaying information).

[0009] The depalletizing system 1 includes: a three-dimensional sensor 10 configured to detect the objects L on the pallet P from above and generate three-dimensional information, including a two-dimensional image and distance information for two-dimensional positions; a robot 20 configured to hold and move the objects L; and an image processing device 30 configured to instruct the robot 20 to pick up the objects L sequentially based on the three-dimensional information generated by the three-dimensional sensor 10.

[0010] The three-dimensional sensor 10 can be any device configured to generate three-dimensional information, including a two-dimensional image and distance information for two-dimensional positions (typically for each pixel of the two-dimensional image). Examples include a stereo camera configured to capture multiple two-dimensional images and generate distance information from the parallax between the images, and a combination of a two-dimensional camera configured to capture a two-dimensional image and a laser scanner configured to capture a distance image.

[0011] The robot 20 can be a vertically articulated robot, as illustrated, but is not limited to this and can alternatively be, for example, an orthogonal coordinate robot, a SCARA robot, or a parallel-articulated robot. The robot 20 includes a head 21 at its end, which is configured to hold various types of objects L. The configuration of the head 21 is not particularly restricted and can include a configuration with a suction cup, as shown, or a configuration with gripping fingers that open and close.

[0012] The image processing device 30 includes a memory, a processor, an input / output interface, and the like, and can be implemented by one or more computer devices configured to execute a suitable control program. In addition to the functions of the image processing device according to the present disclosure, the image processing device 30 of the present embodiment includes functions of a robot control device designed to control the robot 20, wherein the functions of the robot control device can alternatively be implemented independently by another computer device.

[0013] The image processing device 30 comprises a three-dimensional information acquisition unit 31, a machine learning unit 32, an image recognition unit 33, an overlap determination unit 34, an area threshold setting unit 35, a distance determination unit 36, a distance threshold setting unit 37, an information integration unit 38, an identifier assignment unit 39, a candidate exclusion unit 40, and a robot control unit 41. Of these components, all components except the robot control unit 41 are components of the image processing device according to the present disclosure. These components classify functions of the image processing device 30 and need not be clearly separated from one another with respect to physical configuration, hardware configuration, or program configuration.

[0014] The three-dimensional information acquisition unit 31 acquires three-dimensional information from the three-dimensional sensor 10, including a two-dimensional image and distance information for two-dimensional positions. The acquisition of three-dimensional information can be performed each time an object L is removed, or alternatively, only once for the removal of a large number of objects L belonging to the same plane (an area where the height positions are essentially the same).

[0015] The machine learning unit 32 uses AI to create a model configured to detect objects L in the two-dimensional image. Specifically, the machine learning unit 32 uses supervised learning, in which a two-dimensional image specifying candidate areas is fed into a neural network, to create a training model for the image recognition unit 33 to detect candidate areas that are considered to be areas where the objects L are present in the two-dimensional image. In the two-dimensional image, each object L can be assumed to have a rectangular outline. It is advantageous for the machine learning unit 32 to perform the learning using suitable input data to minimize the possibility of a pattern or similar feature attached to the packaging material of the object L being incorrectly identified as a candidate area.

[0016] Based on the two-dimensional image acquired by the three-dimensional information acquisition unit 31, the image recognition unit 33 identifies candidate areas that are considered image areas of the objects L. In the present embodiment, the image recognition unit 33 is intended to perform image recognition using the learning model provided by the machine learning unit 32; however, the image recognition unit 33 is not limited to this and can alternatively be configured to identify candidate areas by template matching or the like. The candidate areas identified by the image recognition unit 33 may include areas that are incorrectly identified as independent objects L, such as labels attached to the objects L or patterns on the objects L.

[0017] The overlap detection unit 34 determines whether candidate areas overlap when multiple candidate areas are detected in the same two-dimensional image. In particular, the overlap detection unit 34 can be configured to determine that two candidate areas overlap if the ratio of the overlap area between the two candidate areas to an area of ​​the smaller of the two candidate areas is equal to or greater than an area threshold.

[0018] The area threshold setting device 35 receives user input of an area threshold, which is a threshold for the overlap area ratio used by the overlap determination device 34 to determine whether an overlap exists. If no input is received from a user, the area threshold setting device 35 can alternatively be configured to apply a predefined default value. The area threshold setting device 35 can accept any numerical input, or alternatively, it can allow the user to select a numerical value to be used as the area threshold from a variety of predefined numerical values.

[0019] With respect to two candidate areas identified as overlapping by the overlap determination unit 34, the distance determination unit 36 ​​uses the distance information to determine whether the two candidate areas lie on the same plane. Specifically, the distance determination unit 36 ​​can be configured to determine that the two candidate areas lie on the same plane if the difference between representative distances, which represent representative values ​​for distances from the three-dimensional sensor 10 for the respective candidate areas, is equal to or less than a distance threshold value.If the distance determination unit 36 ​​determines that the two candidate areas lie on the same plane, the distance determination unit 36 ​​determines that a candidate area with a smaller area is part of the object L to which a candidate area with a larger area points, for example, an area of ​​a label attached to an object L behind it, and is not an area of ​​an independent object L. If the distance determination unit 36 ​​determines that the two candidate areas do not lie on the same plane, the distance determination unit 36 ​​determines that the candidate area with the smaller area is an area of ​​an independent object L located in front of the object L indicated by the candidate area with the larger area.

[0020] A representative distance checked by the distance determination unit 36 ​​can be a distance from a pixel representative of the candidate region, an average of the distances of a large number of pixels, a mode value of the distances of a large number of pixels, or the like. This allows a relatively accurate determination of whether the candidate region is a region of an independent object L, while simultaneously reducing the computational load. If the representative distance is derived as the distance of a pixel or a small number of pixels representative of the candidate region, it is advantageous for the pixel representative of the candidate region to be selected from a central section spaced from an outer boundary of the candidate region.

[0021] The distance determination unit 36 ​​can alternatively be configured to estimate, based on the distance information, the planes on which the two candidate areas exist, and, based on distances between the planes or angles of the normals of the planes, to determine whether the two candidate areas exist on the same plane.

[0022] The distance threshold setting unit 37 receives input of a distance threshold value, which is used by unit 36 ​​for distance determination to ascertain whether a candidate area with a smaller area is a separate object. If no input is received from the user, the distance threshold setting unit 37 can alternatively be configured to apply a predefined default value. The distance threshold setting unit 37 can accept any numeric input, but can alternatively allow the user to select a numeric value to be used as the area threshold from a variety of predefined numeric values.

[0023] When the distance determination unit 36 ​​determines that a smaller candidate area is part of the object L indicated by a rearward candidate area, the information integration unit 38 integrates information from the candidate area into information from the larger candidate area. Accordingly, it is possible to map information indicating the location of a label or similar item, identified as the smaller candidate area, on the rearward object L indicated by the larger candidate area. Therefore, an operator can subsequently easily verify the picked object L by placing it at a transfer position, taking into account the position of a target label identified as the smaller candidate area.By integrating the information from the candidate area with the smaller area into the information from the candidate area with the larger area, the candidate area with the smaller area is excluded from candidate areas.

[0024] If the distance determination unit detects that a candidate area with a larger area contains a candidate area with a smaller area as part of it, the identifier assignment unit 39 assigns an identifier to the information of the candidate area with the larger area, indicating that such a detection has been made. By assigning such an identifier, for example, if the candidate area with the smaller area, identified as part of a rear object L, is a marker of the object L with the larger area, an object L with a marker and an object L without a marker can be moved to different positions. This allows for appropriate sorting of the objects L taken from the palette P.

[0025] If the distance determination unit 36 ​​determines that the candidate area with the smaller area is an independent object L, the candidate exclusion unit 40 excludes the candidate area with the larger area from the candidate areas, at least temporarily. Accordingly, it is possible to avoid a problem in which an attempt is made to remove a lower object L on which another object L is placed, thereby damaging or dropping an upper object L.

[0026] The robot control unit 41 commands operations of the robot 20 such that the robot 20 picks up an object L corresponding to a candidate area, which ultimately remains as a "candidate area" indicating an independent object L. The configuration of the robot control unit 41 can resemble a configuration of a conventional robot control device.

[0027] Next, illustrate Fig. 2. Object Recognition Processing Procedures in the Image Processing Device 30. The object recognition processing in the image processing device 30 includes: a three-dimensional information acquisition step (step S01), a two-dimensional image recognition step (step S02), a candidate area number check step (step S03), a candidate area combination determination step (step S04), an overlap ratios check step (step S05), a distance verification step (step S06), an information integration step (step S07), a distance determination identifier assignment step (step S08), a candidate area exclusion step (step S09), and a termination check step (step S10).

[0028] In the three-dimensional information acquisition step S01, the three-dimensional information acquisition unit 31 acquires three-dimensional information from the three-dimensional sensor 10. In the two-dimensional image recognition step S02, the image recognition unit 33 recognizes candidate areas of the objects L from the two-dimensional image contained in the three-dimensional information.

[0029] In the candidate area number check step S03, it is verified whether multiple candidate areas were detected in the dimensional image recognition step. If the number of candidate areas is one or less, the object recognition process is terminated. If the number of candidate areas is more than one, the process proceeds to step S04, which determines the combination of candidate areas, and a combination of candidate areas is selected to verify independence; that is, two candidate areas are chosen.

[0030] When a combination of candidate areas is specified, the overlap determination unit 34 calculates the ratio of the overlap area of ​​the two candidate areas to the area of ​​the smaller of the two candidate areas in the overlap ratio check step S05 and checks whether the ratio of the overlap area is equal to or greater than the area threshold. If the ratio of the overlap area in the smaller candidate area is equal to or greater than the area threshold, the two candidate areas are found to overlap, and processing continues to step S06. If the ratio of the overlap area in the smaller candidate area is less than the area threshold, the two candidate areas are found not to overlap, and processing continues to step S10.

[0031] In the distance check step of step S06, the distance determination unit determines 36 representative distances of the respective candidate areas and compares any difference between these representative distances with the distance threshold. If the difference between the representative distances is equal to or less than the distance threshold, the candidate area with the smaller area is identified as part of the object L indicated by the candidate area with the larger area. If the difference between the representative distances exceeds the distance threshold, the candidate area with the smaller area is identified as indicating a separate object L placed on top of the object L indicated by the candidate area with the larger area.

[0032] If in step S06 it is determined that the candidate space with the smaller area is part of the candidate space with the larger area, then in step S07 the information integration unit 38 integrates information from the candidate space with the smaller area into information from the candidate space with the larger area, and in step S08 the identifier mapping unit 39 assigns an identifier indicating that the candidate space with the larger area has been determined to include the candidate space with the smaller area as part of it; the processing then proceeds to step S10.

[0033] If in step S06 it is determined that the candidate area with the smaller area represents an independent object L, the candidate exclusion unit 40 excludes the candidate area with the larger area from the candidate areas in step S09; the processing then proceeds to step S10.

[0034] In step S10, it is checked whether the processing of step S04 and the subsequent steps has been carried out for all combinations of candidate regions, with the exception of the candidate regions integrated in step S07 and the candidate regions excluded in step S09. If there is any combination of candidate regions for which processing has not yet been carried out, the processing of step S04 and the subsequent steps is carried out for the new combination. When the processing for all combinations of candidate regions is complete, the object detection processing is terminated.

[0035] As described above, the image processing device 30 of the present embodiment first defines candidate areas by image recognition of a two-dimensional image that has a relatively low error rate, and can therefore extract candidate areas more accurately than when using distance images or point cloud data, which tend to have a higher error rate. In particular, when box-shaped objects are arranged in close proximity to one another, the boundaries between the objects cannot be recognized in a distance image or the like, but by image recognition of the two-dimensional image, the closely spaced objects can be separated and detected with a high degree of probability.Since the image processing device 30 determines, based on a difference between representative distances of candidate areas, whether an image recognized as a candidate area is based on a contour of object L or on a marking or the like of object L, misinterpretations caused by the two-dimensional image can be corrected relatively easily. Accordingly, the image processing device 30 can accurately recognize object L compared to pattern matching of a two-dimensional image and pattern matching of a distance image or point cloud data. (Second embodiment)

[0036] Fig. Figure 3 is a block diagram illustrating a configuration of a depalletizing system 1A. Depalletizing system 1A is a device configured to sequentially remove several objects L arranged on a pallet P, similar to depalletizing system 1. Fig. 1.

[0037] The depalletizing system 1 includes: a three-dimensional sensor 10 configured to pick up the objects L on the pallet P from above and generate three-dimensional information, including a two-dimensional image and distance information for two-dimensional positions; a robot 20 configured to hold and move the objects L; and an image processing device 30A configured to instruct the robot 20 to pick up the objects L sequentially based on the three-dimensional information generated by the three-dimensional sensor 10.

[0038] The image processing device 30A includes a memory, a processor, an input / output interface, and the like, and can be implemented by one or more computer devices configured to execute a suitable control program. In addition to the functions of the image processing device according to the present disclosure, the image processing device 30A of the present embodiment includes functions of a robot control device configured to control the robot 20, wherein the functions of the robot control device can alternatively be implemented independently by another computer device.

[0039] The image processing device 30A includes: a three-dimensional information acquisition unit 31; a machine learning unit 32A; a comprehensive estimation unit 42; and a robot control unit 41. The machine learning unit 32A and the comprehensive estimation unit 42 are components of the image processing device according to the present disclosure.

[0040] The machine learning unit 32A uses AI to create a model based on three-dimensional information, configured to recognize object L. Specifically, the machine learning unit 32A performs supervised learning, where three-dimensional information is fed into a neural network. This three-dimensional information specifies candidate regions and areas such as patterns to be excluded from a two-dimensional image. Accordingly, the machine learning unit 32A generates a training model for the comprehensive estimation unit 42 to identify candidate regions that are considered to be areas where object L is present in the two-dimensional image.

[0041] The comprehensive estimation unit 42 identifies candidate areas, which are considered image regions of object L in the two-dimensional image, based on the three-dimensional information acquired by the three-dimensional information acquisition unit 31. Using the learning model generated by the machine learning unit 32A, the comprehensive estimation unit 42 excludes patterns of object L or labels attached to object L and identifies as candidate areas only those areas that correspond to the contours of the objects L to be picked by the robot 20. Preferably, the comprehensive estimation unit 42 is further configured to identify, taking into account the distance information, a candidate area indicating the object L that is closest to the front (i.e., actually at the very top) in the two-dimensional image. This enables the identification of the object L to be picked next by the robot 20.

[0042] As described above, the image processing device 30 of the present embodiment uses AI to determine only candidate areas that correspond to object L, while excluding patterns of object L and labels attached to object L, and can therefore identify the object L to be picked next by the robot 20 in a relatively small number of steps.

[0043] Further remarks on the above embodiments and modified examples are disclosed below. (Additional Note 1)

[0044] The image processing device (30) includes a three-dimensional information acquisition unit (31) configured to acquire three-dimensional information from a visual sensor, comprising a two-dimensional image and distance information for two-dimensional positions; an image recognition unit (33) configured to recognize candidate areas of an object from the two-dimensional image; an overlap determination unit (34) configured to determine whether candidate areas overlap; and a distance determination unit (36) configured to determine, based on the distance information, whether two candidate areas identified as overlapping by the overlap determination unit (34) lie on the same plane. If it is determined that the two candidate areas lie on the same plane,a candidate region with a smaller area is part of an object indicated by the candidate region with a larger area, and determines that if the two candidate regions are found not to be on the same plane, the candidate region with the smaller area is an independent object located in front of the object indicated by the candidate region with the larger area. (Additional Note 2)

[0045] The image processing device (30) described in Additional Note 1 may further include an information integration unit (38) configured to integrate information from a smaller candidate area, identified by the distance determination unit as part of an object specified by a larger candidate area, into the information of the larger candidate area. (Additional Note 3)

[0046] The image processing device (30) described in Supplementary Note 1 or 2 may further comprise an identifier assignment unit (39) configured to assign an identifier to a candidate area with a larger area, which has been determined by the distance determination unit (36) to contain a candidate area with a smaller area as part thereof, indicating that such a determination has been made. (Additional note 4)

[0047] The image processing device (30) described in one of the additional notes 1 to 3 may further include a candidate exclusion unit (40) configured to exclude a candidate area with a larger area from the candidate areas if the distance determination unit (36) determines that a candidate area with a smaller area is a separate object. (Additional note 5)

[0048] The image processing device (30), as described in one of Additional Notes 1 to 4, may further include an area threshold setting device (35) configured to receive an input of a threshold value for an overlap area ratio, which is used by the overlap determination device (34) to determine whether an overlap exists. (Additional Note 6)

[0049] The image processing device (30), as described in any of the additional notes 1 to 5, may further include a distance threshold setting unit (37) configured to receive an input of a distance threshold value which is used by the distance determination device (36) to determine whether a candidate area is a separate object. (Additional note 7)

[0050] The image processing device (30A) includes a three-dimensional information acquisition unit (31) configured to acquire three-dimensional information, including a two-dimensional image and distance information for two-dimensional positions; a machine learning unit (32A) configured to perform machine learning of a model to identify an object in the two-dimensional image by using the three-dimensional information as input for a neural network; and a comprehensive estimation unit (42) configured to estimate candidate areas of the object in the three-dimensional information using the model learned by the machine learning unit (32A). (Additional Note 8)

[0051] In the image processing device (30A) described in Additional Note 7, the comprehensive estimation unit (42) can identify a candidate area of ​​an object that is closest to the front.

[0052] Although the present disclosure has been described in detail above, it is not limited to the individual embodiments described above. These embodiments may be subject to various additions, substitutions, modifications, or partial deletions without departing from the core of the present disclosure, to an extent that does not deviate from the spirit of the present disclosure as derived from the content described in the claims and their equivalents. For example, the sequence of individual operations or the sequence of individual processing steps is shown as an example in the embodiments described above, and the present disclosure is not limited thereto. EXPLANATION OF THE REFERENCE SYMBOLS 1, 1A Depalletizing System 10 three-dimensional sensors 20 robots 21 heads 30, 30A Image processing device 31 Three-dimensional information acquisition unit 32, 32A Machine learning unit 33 Image recognition unit 34 Overlap determination unit 35 Area threshold setting unit 36 Distance determination unit 37 Distance threshold setting unit 38 Information Integration Unit 39 Identifier assignment unit 40 Unit on the exclusion of candidates 41 Robot control unit 42 units for comprehensive estimation L Object P Palette QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] JP H10-31742 A

[0004]

Claims

[1] Image processing device comprising: a three-dimensional information acquisition unit configured to acquire three-dimensional information from a visual sensor, including a two-dimensional image and distance information from two-dimensional positions; an image recognition unit configured to identify candidate areas of an object from a two-dimensional image; an overlap determination unit configured to determine whether the candidate areas overlap; and A distance determination unit configured to determine, based on distance information, whether two candidate areas identified as overlapping by the overlap determination unit are on the same plane, to determine that if the two candidate areas are on the same plane, a candidate area with a smaller area is part of an object indicated by a candidate area with a larger area, and to determine that if the two candidate areas are not on the same plane, the candidate area with the smaller area is a separate object located in front of the object indicated by the candidate area with the larger area. [2] Image processing device according to claim 1, further comprising an information integration unit configured to integrate information from a candidate area with a smaller area, which has been determined by the distance determination unit as part of an object specified by a candidate area with a larger area, into the information of the candidate area with a larger area, wherein this integration is based on distances. [3] Image processing device according to claim 1 or 2, further comprising an identifier assignment unit configured to assign an identifier to a candidate area with a larger area, which has been determined by the distance determination unit to contain a candidate area with a smaller area as part thereof, indicating that such a determination has been made. [4] Image processing device according to any one of claims 1 to 3, further comprising a candidate exclusion unit configured to exclude a candidate area with a larger area from the candidate areas when the distance determination unit determines that a candidate area with a smaller area is a separate object. [5] Image processing device according to any one of claims 1 to 4, further comprising an area threshold setting device configured to receive an input of a threshold value for an overlap area ratio, which is used by the overlap determination device to determine whether an overlap exists. [6] Image processing device according to any one of claims 1 to 5, further comprising a distance threshold setting device configured to receive an input of a distance threshold value which is used by the distance determination device to determine whether a candidate area is an independent object. [7] Image processing device comprising: a three-dimensional information acquisition unit configured to acquire three-dimensional information, including a two-dimensional image and distance information from two-dimensional positions; a machine learning unit configured to perform machine learning of a model configured to identify an object in the two-dimensional image by using the three-dimensional information as input for a neural network; and a comprehensive estimation unit configured to estimate candidate areas of the object in the three-dimensional information using the model learned by the machine learning unit. [8] Image processing device according to claim 7, wherein the comprehensive estimation unit identifies a candidate area of ​​an object that is closest to the front.

Images