Information processing device, information processing method and information processing program

Abstract

To provide a technology capable of improving detection accuracy for an object.SOLUTION: An information processing device (1) includes: a contour information generation unit (12) acquires an image including depth information to generate contour information for an object extracted from the image; and a determination unit (13) that determines whether or not the object is a detection target on the basis of information obtained from a vertical width and a lateral width of a rectangle surrounding the contour of the object, of the contour information.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an information processing device that detects an object in a depth image. [Background technology]

[0002] A technology for detecting an object in an image captured by a camera has been known for some time. For example, the following Patent Document 1 discloses a system for detecting and recognizing an object in a scene and determining the 3D pose (three-dimensional position and orientation) of the object. In this system, a novel edge connection is performed to complete missing depth edges in a scene of an unoccluded part in a bin captured by a single camera, and the 3D pose of the part in the bin is determined. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2009-115783 Summary of the Invention [Problem to be solved by the invention]

[0004] The above system detects only objects of the same shape in a certain shooting environment, such as picking parts from a bin with a robot arm attached to an industrial robot that performs manufacturing and assembly, etc. However, when attempting to detect objects of different shapes and sizes, the technology of Patent Document 1 has problems such as the depth edge of a missing part not matching the shape of the object, and the detection accuracy of the object in the image decreases.

[0005] An object of one embodiment of the present invention is to provide an information processing device or the like that is capable of improving the detection accuracy of objects of various shapes. [Means for solving the problem]

[0006] In order to solve the above problems, an information processing device according to one embodiment of the present invention includes a contour information generation unit that acquires an image including depth information and generates contour information of an object extracted from the image, and a determination unit that determines whether the object is a detection target or not based on information obtained from the contour information, including the vertical and horizontal widths of a rectangle surrounding the contour of the object.

[0007] In order to solve the above problems, an information processing device according to one embodiment of the present invention includes a contour information generation unit that acquires an image including depth information and generates contour information of an object extracted from the image, a rectangular image conversion unit that converts the contour information generated by the contour information generation unit into a rectangle group image composed of a plurality of rectangles, a rectangular image selection unit that selects a rectangular image to be processed from the rectangle group image generated by the rectangular image conversion unit, a ratio calculation unit that calculates the aspect ratio of the rectangular image selected by the rectangular image selection unit, and an object determination unit that determines that the rectangular image selected by the rectangular image selection unit is the object to be detected when the ratio calculated by the ratio calculation unit satisfies a predetermined threshold condition.

[0008] In order to solve the above problems, an information processing device according to one embodiment of the present invention includes a contour information generation unit that acquires an image including depth information and generates contour information of an object extracted from the image, and a determination unit that determines that the object is a detection target when the number of sides that constitute the contour of the object in the contour information is within a predetermined range.

[0009] In addition, in order to solve the above-mentioned problems, an information processing method according to one embodiment of the present invention includes a step of acquiring an image including depth information, generating contour information of an object extracted from the image, and a step of determining whether the object is a detection target based on information obtained from the vertical and horizontal widths of a rectangle surrounding the contour of the object, among the contour information.

[0010] In addition, in order to solve the above-mentioned problems, an information processing program according to one embodiment of the present invention causes a computer to execute a process of acquiring an image including depth information and generating contour information of an object extracted from the image, and a process of determining whether or not the object is a detection target based on information obtained from the contour information, which is obtained from the vertical and horizontal widths of a rectangle surrounding the contour of the object. Effect of the Invention

[0011] According to one aspect of the present invention, it is possible to improve the accuracy of object detection. [Brief description of the drawings]

[0012] [Figure 1] 1 is a block diagram showing an example of the configuration of an information processing system according to a first embodiment of the present invention. [Diagram 2] 1 is a diagram showing a garbage collection truck dropping garbage into a garbage pit at a garbage incineration facility. [Diagram 3] A diagram showing the inside of a garbage pit. [Figure 4] 2 is a diagram showing details of a control unit of the information processing device. [Diagram 5] 5 is a flowchart showing an example of a processing procedure of the information processing device according to the first embodiment of the present invention. [Figure 6] 6 is a flowchart showing details of the object detection processing procedure shown in FIG. 5. [Figure 7] FIG. 13 is a diagram for explaining a case where a contour includes a plurality of objects. [Figure 8] 10 is a diagram showing details of a control unit of an information processing device according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] In recent years, the dumping of materials unsuitable for incineration (hereinafter simply referred to as unsuitable materials) into waste incineration facilities has become a problem. When unsuitable materials are dumped into an incinerator, it can cause problems such as poor combustion in the incinerator and blockages in the incinerator's ash discharge facility, and in some cases, it can even cause the incinerator to be shut down in an emergency.

[0014] Furthermore, it is extremely difficult to accurately detect various unsuitable items from the large amount of garbage being transported.

[0015] (Embodiment 1) An information processing device 1 according to an embodiment of the present invention can solve the above problems. The information processing device 1 has a function of detecting unsuitable objects from the garbage brought into a garbage incineration facility. Specifically, the information processing device 1 detects unsuitable objects using images of the garbage being thrown into a garbage pit. The garbage pit will be described later with reference to FIG. 3. In addition, unsuitable objects may be detected after the garbage is dropped in. In addition, unsuitable objects are objects that should not be incinerated in an incinerator installed in a garbage incineration facility. In the following description, the unsuitable objects may also be called objects to be detected. Specific examples of unsuitable objects will be described later.

[0016] [Example of information processing system configuration] Fig. 1 is a block diagram showing a configuration example of an information processing system according to an embodiment of the present invention. As shown in Fig. 1, the information processing system 100 includes an information processing device 1 and a 3D (three-dimensional) camera 2. Note that, in this embodiment, a case will be described in which the information processing system 100 is installed in a waste incineration facility, but the present invention is not limited to this and can be used in any system that detects an object.

[0017] The 3D camera 2 is a camera capable of capturing not only vertical and horizontal information but also depth information. The 3D camera 2 will be described as a stereo camera using a stereo system, but is not limited thereto, and may be, for example, a ToF camera using a ToF (Time of Flight) system, a camera including structured illumination using a projector system or a laser dot system, or the like.

[0018] The information processing device 1 includes a control unit 10, a storage unit 20, an image IF (Interface) unit 30, a communication unit 40, an input unit 50, and an output unit 60. The information processing device 1 may be, for example, a personal computer, a server, a workstation, etc. The control unit 10 has a function of performing overall control of the information processing device 1, which will be described in detail later.

[0019] The storage unit 20 is a storage device that stores various data used by the information processing device 1. As the storage unit 20, for example, a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous DRAM (SDRAM), a hard disk drive (HDD), a solid-state drive (SSD), a secure digital (SD) card, an embedded multi-media controller (eMMC), or the like, or a combination of these can be applied.

[0020] The image IF unit 30 acquires an image including depth information captured by the 3D camera 2, and outputs it to the control unit 10. The 3D camera 2 outputs depth information for each pixel included in the image together with the image, and outputs the image as a moving image at a predetermined frame period.

[0021] The image IF unit 30 is an interface for communicating between the 3D camera 2 and the information processing device 1. The image IF unit 30 may be for wired communication or for wireless communication. For example, a USB (Universal Serial Bus), a LAN (Local-Area Network), a wireless LAN, or the like may be applied as the IF unit.

[0022] The communication unit 40 is a part that communicates with other devices, and can use, for example, a wireless LAN (Local Area Network), a wired LAN, a WAN (Wide Area Network), a public line network, a mobile data communication network, or a combination of these networks.

[0023] The input unit 50 is a part that accepts input of various data to the information processing device 1, and may be, for example, a keyboard, a pointing device such as a mouse, a touch panel, or a combination of these.

[0024] The output unit 60 is a part for the information processing device 1 to output various data, and may be, for example, a display device such as a liquid crystal panel, an audio output device, a printer, or a combination of these. Note that the input unit 50 and the output unit 60 may be, for example, a Universal Serial Bus (USB), a wireless LAN, or the like.

[0025] [Taking pictures of garbage] FIG. 2 is a diagram showing a garbage truck 200 dropping garbage into a garbage pit at a garbage incineration facility. FIG. 3 is a diagram showing the inside of the garbage pit. The garbage pit is a place for temporarily storing garbage collected at the garbage incineration facility, and the garbage in the garbage pit is sent to an incinerator one by one to be incinerated. As shown in FIG. 2, the garbage incineration facility is provided with a plurality of doors such as doors 300A and 300B (hereinafter, when there is no need to distinguish between them, they will be collectively referred to as door 300). Also, as shown in FIG. 3, a garbage pit is provided beyond door 300. That is, when door 300 is opened, a drop opening for dropping garbage into the garbage pit appears. As shown in FIG. 2, garbage truck 200 drops garbage into the garbage pit from the drop opening.

[0026] The 3D camera 2 is attached at a position where it can capture an image of the garbage flowing down the slope 600 in Fig. 3. For example, the 3D camera 2 may be attached to the attachment point 400 shown in Figs. 2 and 3. The attachment point 400 is located on the surface of each door 300, so if the 3D camera 2 is attached to the attachment point 400, the 3D camera 2 will be located above the slope 600 when the door 300 is opened, and this position is suitable for capturing images of the garbage. Of course, the attachment point of the 3D camera 2 can be any position where it can capture an image of the garbage flowing down the slope 600.

[0027] Fig. 4 is a diagram for explaining details of the control unit 10 of the information processing device 1 shown in Fig. 1. As shown in Fig. 4, the control unit 10 includes an image preprocessing unit 11, a contour information generating unit 12, a determination unit 13, and a contour dividing unit 14. The determination unit 13 includes a side number comparing unit 131, a rectangle calculating unit 132, a rectangle comparing unit 133, a length comparing unit 134, an area comparing unit 135, an actual width comparing unit 136, and a flow comparing unit 137.

[0028] When the 3D camera 2 is a stereo camera, the stereo camera includes two normal RGB or monochrome camera sensors, and compares information from both camera sensors to output depth information (distance information) for each pixel. Here, a normal camera image may also be output. In this embodiment, the target captured by the stereo camera is dust flowing on a slope, so the image of each frame of a video (moving image) may be used as the detection target. In the following object contour detection process, the contour is output as information, but the contour may also be output as an image.

[0029] The image pre-processing unit 11 binarizes the depth information included in the image captured by the 3D camera 2. For example, the image pre-processing unit 11 performs binarization by comparing the depth information of each pixel with a predetermined threshold value, and setting the depth information to "1" (or "255" if the image is in bytes) if the depth information is equal to or greater than the threshold value, and setting the depth information to "0" if the depth information is smaller than the threshold value. This allows the contour information generating unit 12 described below to obtain contour information including only the contour of the object. The contour information includes various information obtained from pixel values, such as contour information, i.e., the number of sides of the contour and the length of the sides, and is not limited to specific information. The contour information may also be an image.

[0030] The depth information is, for example, 8- or 16-bit information, and can be handled as a grayscale image, etc. The unit of the depth information is, for example, millimeters. Also, the image of the depth information may be, for example, only the portion where the contour is detected, and the pixels other than the contour may be, for example, an invalid value of "0".

[0031] The image preprocessing unit 11 performs preprocessing such as contraction, separation, and noise removal on a binarized image (hereinafter referred to as a binary image). By performing contraction on a binary image, adjacent objects can be easily separated. When separation is not possible, multiple objects may be detected as a single outline, so the image preprocessing unit 11 performs contraction as preprocessing.

[0032] Furthermore, the image preprocessing unit 11 compares the depth information of each pixel with that of adjacent pixels, and if there is a depth difference of a first predetermined value (e.g., 20 cm) or more, performs separation processing by invalidating the value of the deeper pixel, for example by setting the binarized depth information to "0." This is because if the binarized depth information is not set to "0," overlapping objects may be detected as a single, integrated contour.

[0033] The image preprocessing unit 11 also performs noise removal processing on the binarized depth image by removing small points contained in the binarized depth image.

[0034] The contour information generating unit 12 extracts the contours of multiple objects included in the binarized image to generate contour information. For example, the method described in Non-Patent Document 1 below can be used to extract the contours of an object. The contour of an object may be a set of point positions (x value, y value) in a binarized depth image. Here, one straight line side of the contour of the object becomes two consecutive points. The contour information generated by the contour information generating unit 12 also includes the number of sides.

[0035] [Non-Patent Document 1] Satoshi Suzuki and Keiichi Abe, "Topological Structural Analysis of Digitized Binary Images by Border Following", CVGIP 30, pp 32-46, 1985 The determination unit 13 determines (detects) whether each contour is a detection target object (unsuitable object) based on the result of the contour information generation unit 12. Specifically, the determination is made by the side number comparison unit 131 to the flow comparison unit 137 as follows.

[0036] The number of sides comparison unit 131 determines whether the number of sides is within a predetermined range, and if the number of sides of the object's contour is within the predetermined range, the object is considered to be a candidate for the object to be detected, and the detection process continues. If the number of sides of the object's contour is not within the predetermined range, for example, if the number of sides is small, the number of sides comparison unit 131 determines that the object is too small, and if the number of sides is large, the shape of the object is too complex, and the object is not considered to be a detection target. In this way, if the number of sides of the object's contour is not within the predetermined range, the amount of calculation can be reduced by determining that the object is not a detection target.

[0037] When an object is a candidate for the object to be detected, the rectangle calculation unit 132 calculates a rectangle (including a rectangle and a square) in which the contour of the object exactly fits. For example, the method described in the following Non-Patent Document 2 can be used. Note that a rectangle in which the contour of the object exactly fits refers to a rectangle in which the contour of the object touches all four sides.

[0038] [Non-Patent Document 2] Godfried Toussaint, "Solving Geometric Problems with the Rotating Calipers", Proceedings of IEEE MELECON'83, Athens, Greece, May 1983 If the ratio of the long side to the short side of the rectangle calculated by the rectangle calculation section 132 is within a predetermined range, the rectangle comparison section 133 detects an object having a corresponding contour as the object to be detected.

[0039] Length comparison section 134 may determine the length of an object from the contour shape of the object, and if the length of the object is within a predetermined range, detects the object as the object to be detected.

[0040] When the ratio between the area of ​​the rectangle and the area of ​​the corresponding contour falls within a predetermined range, area comparison section 135 may detect the object with the corresponding contour as the object to be detected.

[0041] Actual width comparison section 136 may determine the width of an object from the shape of the object's contour, and if the width of the object is within a predetermined range, detect the object as the object to be detected.

[0042] When the rectangle comparison unit 133 determines that the object is not a detection target object, the contour division unit 14 determines whether the object can be divided. When the contour division unit 14 determines that the object can be divided, it divides the object and makes the rectangle calculation unit 132 and the rectangle comparison unit 133 perform the above-mentioned processing again. The method of dividing the object will be described later.

[0043] As described above, the 3D camera 2 is attached at a position where it can capture the image of garbage flowing on the slope. When liquid objects such as sewage are attached to the slope, the liquid objects adhere to the slope of the slope, and the 3D camera 2 may capture the image of the liquid objects adhered to the slope of the slope. The 3D camera 2, for example, a stereo camera, may give depth to parts with different colors, and may erroneously determine that the liquid objects adhered to the slope of the slope are the objects to be detected.

[0044] Therefore, the flow comparison unit 137 may be configured to determine (detect) an object with a contour that does not match the direction of the long side of the contour as an object to be detected (unsuitable object). In other words, when the direction of the long side of the contour of an object matches the direction of the slope of the slope, the flow comparison unit 137 determines that the object is a liquid object adhering in the direction of the slope of the slope. The determination unit 13 determines whether or not the object is an object to be detected by the above-described process.

[0045] [Processing procedure of information processing device 1] 5 is a flowchart showing an example of a processing procedure of the information processing device 1 according to an embodiment of the present invention. First, the information processing device 1 starts inputting video from the 3D camera 2 (S1). As described above, this video is input as a moving image at a predetermined frame period.

[0046] The image pre-processing unit 11 acquires the next image (an image containing one frame's worth of depth information) from the 3D camera 2, compares the depth information of each pixel in the image with a predetermined threshold, and performs binarization by setting the depth information to "1" (or "255" if the image is in bytes) if the depth information is greater than or equal to the threshold, and setting the depth information to "0" if the depth information is less than the threshold (S2).

[0047] Next, the image preprocessing unit 11 performs preprocessing such as contraction, separation, and noise removal on the binarized image (S3). Then, the contour information generating unit 12 extracts contours of multiple objects included in the binarized image to generate contour information (S4).

[0048] Next, the determination unit 13 detects the object to be detected based on the contour shapes of the multiple objects included in the binarized image (S5). The details of this object detection process will be described later.

[0049] Next, the control unit 10 records information about the detected object to be detected (unsuitable object) in the storage unit 20, and also causes the output unit 60, for example a display device, to display the information (S6).

[0050] Finally, the control unit 10 judges whether or not to end the video input (S7). For example, if the user issues an instruction to end the process via the input unit 50 (S7, Yes), the process ends. If the user does not issue an instruction to end the process via the input unit 50 (S7, No), the process returns to step S2 and repeats the subsequent processes.

[0051] <Detailed procedure of object detection process (S5)> Fig. 6 is a flowchart for explaining details of the object detection process (S5) shown in Fig. 5. In Fig. 6, a long object will be described as an object to be detected (unsuitable object).

[0052] Long objects are, for example, umbrellas, logs, etc. Umbrellas contain metal parts that can cause deterioration of combustion in incinerators and blockages in the incinerator's ash discharge equipment, and are therefore considered to be objects to be detected (unsuitable objects). Logs, on the other hand, can cause blockages in the incinerator entrance before incineration, and are therefore also considered to be objects to be detected (unsuitable objects).

[0053] First, the determination unit 13 acquires a first contour from among the multiple contours extracted by the contour information generation unit 12 (S51). Then, the side number comparison unit 131 determines whether the number of sides is within a predetermined range (S52). When the process of step S51 is executed again, the determination unit 13 acquires the next contour from among the multiple contours.

[0054] If the number of sides of the object contour is not within the predetermined range (S52, No), the object is not considered to be a candidate for the object to be detected, and the process proceeds to step S68. If the number of sides of the object contour is within the predetermined range (S52, Yes), the rectangle calculation unit 132 calculates a rectangle that exactly fits the contour of the object (S53). Then, the rectangle comparison unit 133 determines whether the value obtained by dividing the short side of the rectangle by the long side (short side / long side) is smaller than 0.35 (a second predetermined value) (S54). Here, 0.35 is a coefficient for determining whether the object is elongated, and this coefficient may be changed depending on the shape of the object to be detected. Note that a rectangle that exactly fits the contour of the object refers to a rectangle in which the contour of the object touches all four sides of the rectangle.

[0055] If the value obtained by dividing the short side by the long side of the rectangle is smaller than 0.35 (S54, Yes), the object is determined to be a candidate for the object to be detected, and the process proceeds to step S59. If the value obtained by dividing the short side by the long side of the rectangle is 0.35 or more (S54, No), the contour division unit 14 determines whether the contour of the object can be divided (S55).

[0056] FIG. 7 is a diagram for explaining a case where a contour includes multiple objects. As shown in the left diagram of FIG. 7, two objects are extracted as one contour. The contour division unit 14 extracts a portion of this contour that is represented by straight lines, as shown in the center diagram of FIG. 7. In the center diagram of FIG. 7, multiple straight lines are displayed with dotted lines. When there are multiple pairs of parallel straight lines, the contour division unit 14 divides the contour into multiple contours, as shown in the right diagram of FIG. 7.

[0057] The contour division unit 14 also performs polygonal approximation on the contour, reducing the number of points on the contour and approximating the contour to a contour with fewer points. Then, only the long sides of the contour are left. At this point, it becomes equivalent to the one shown in the center diagram of FIG. 7. Then, when there are multiple pairs of parallel straight lines, the contour division unit 14 divides the contour into multiple contours as shown in the right diagram of FIG. 7. Note that these two methods may be used simultaneously to improve the detection effect.

[0058] When the contour is divided into a plurality of contours by the contour dividing unit 14, the processes of steps S56 to S67 are performed a number of times equal to the number of divided contours. In step S56, the rectangle calculation unit 132 acquires the next contour from among the plurality of contours divided by the contour dividing unit 14. Then, the rectangle calculation unit 132 calculates a rectangle that exactly fits the contour of the object (S57). Then, the rectangle comparison unit 133 determines whether or not the value obtained by dividing the short side of the rectangle by the long side is smaller than 0.35 (S58).

[0059] If the value obtained by dividing the short side of the rectangle by the long side is smaller than 0.35 (S58, Yes), the object is determined to be a candidate for the object to be detected, and the process proceeds to step S59. If the value obtained by dividing the short side of the rectangle by the long side is 0.35 or more (S58, No), the object is determined to be not a candidate for the object to be detected, and the process proceeds to step S67.

[0060] In step S59, the length comparison unit 134 calculates the length of the contour (the length of the long dimension). For example, the length comparison unit 134 searches for the closest point in the contour to each of the four points of the rectangle calculated in step S53 or S57. Next, the length comparison unit 134 searches for a point with valid depth information, for example, a point that is set to "1" (or "255" if the image is in byte units) by binarizing the depth information (a point that is not invalid), among the four points of the contour.

[0061] Note that since the contour information is obtained from a binarized image, the depth information is not necessarily present at the same pixel position as the contour point. If the x-axis and y-axis are the normal image axes and the depth is the z-axis, then by using the internal parameters of the stereo camera, the x-value and y-value of the actual dimension can be obtained from the x-value and y-value of the pixel and the z-value of the actual dimension (e.g., millimeters). In other words, all four points of the depth image obtained here can be converted into coordinates of the actual dimension, and the longest distance between any two of these four points (excluding the diagonal) is taken as the length of the long contour.

[0062] Next, the length comparison unit 134 determines whether the length of the elongated contour is within a predetermined range (S60). If the length of the elongated contour is not within the predetermined range (S60, No), the object is not considered to be a candidate for the object to be detected, and the process proceeds to step S67. If the length of the elongated contour is within the predetermined range (S60, Yes), the area comparison unit 135 calculates the ratio between the area of ​​the rectangle and the area of ​​the corresponding contour (S61).

[0063] Next, area comparison unit 135 determines whether the ratio between the area of ​​the rectangle and the area of ​​the corresponding contour is within a predetermined range (S62). If the ratio between the area of ​​the rectangle and the area of ​​the corresponding contour is not within the predetermined range (S62, No), the object is not considered to be a candidate for the object to be detected, and the process proceeds to step S67.

[0064] If the ratio between the area of ​​the rectangle and the area of ​​the corresponding contour falls within a predetermined range (S62, Yes), the actual width comparison unit 136 calculates the actual width of the elongated contour in the same manner as described in step S59 (S63). In this case, the shortest distance between any two of the four points is determined to be the actual width of the elongated contour.

[0065] Next, the actual width comparison unit 136 determines whether the actual width of the elongated contour is within a predetermined range (S64). If the actual width of the elongated contour is not within the predetermined range (S64, No), the object is not considered to be a candidate for the object to be detected, and the process proceeds to step S67.

[0066] If the actual width of the long contour falls within a predetermined range (S64, Yes), the flow comparator 137 determines whether the direction of the long side of the object contour coincides with (is parallel to) the inclination direction of the slope (S65). If the direction of the long side of the object contour coincides with the inclination direction of the slope (S65, Yes), the contour is not considered to be a candidate for the object to be detected, and the process proceeds to step S67.

[0067] Furthermore, if the direction of the long side of the object's contour does not match the inclination direction of the slope (S65, No), the determination unit 13 determines the object of the contour to be a detection target (unsuitable object) (S66).

[0068] In step S67, the determination unit 13 determines whether or not the processing of the multiple contours divided by the contour division unit 14 is completed. If it is determined in step S54 that the contours include only one contour, the process proceeds to step S68.

[0069] If the processing of the multiple contours divided by the contour dividing unit 14 has not been completed (S67, No), the process returns to step S56 and the subsequent processes are repeated. If the processing of the multiple contours divided by the contour dividing unit 14 has been completed (S67, Yes), it is determined whether the processing of all contours in the binarized image has been completed (S68).

[0070] If there are any unprocessed contours (S68, No), the process returns to step S51 and the subsequent processes are repeated. If there are no unprocessed contours (S68, Yes), the object detection process (S5) ends.

[0071] Note that the above explanation is an example of detecting an inappropriate object in a single image of a video, but in order to improve accuracy, it is also possible to track the same object in successive single images and determine that the object is an inappropriate object only if the same object is detected in multiple images.

[0072] In addition, by appropriately changing the values ​​used in the flowchart shown in Figure 6, it is also possible to detect other objects, for example, to detect other cars using images from a camera attached to a car, or to detect ships from satellite photos of the ocean. For example, it is also possible to detect a specific car by setting the threshold value obtained by dividing the short side of a rectangle by the long side to a value specific to the car to be detected.

[0073] [Effects of information processing device 1] As described above, according to the information processing device 1 of this embodiment, the determination unit 13 determines whether or not an object is a detection target based on information obtained from the vertical and horizontal widths of a rectangle surrounding the object's contour in the contour information. Therefore, even when the lighting conditions change, multiple objects overlap or touch each other, the detection target object can be appropriately detected.

[0074] In addition, the determination unit 13 determines whether or not an object is a detection target based on the number of sides constituting the object's contour in the contour information and the information obtained from the vertical and horizontal widths of a rectangle surrounding the object's contour, thereby making it possible to more appropriately detect the detection target object.

[0075] In addition, the determination unit 13 determines whether or not an object is a detection target by using at least one or more of the following information obtained from the height and width of a rectangle surrounding the contour of the object: the ratio of the height and width, the length of the height and width, the area of ​​the rectangle, and the actual width of the object. Therefore, the detection target object can be detected more appropriately.

[0076] Furthermore, if the direction of the long side of the contour does not match the inclination direction of the slope, the determination unit 13 detects the object of that contour as the object to be detected. Therefore, even if liquid such as dirty water is attached to the slope, it is possible to prevent the slope from being erroneously determined to be the object to be detected.

[0077] Furthermore, the contour division unit 14 approximates the contour with a plurality of straight lines, and divides the contour into a plurality of contours when there are a plurality of pairs of parallel straight lines. Therefore, even when one contour includes the contours of a plurality of objects, it can be appropriately divided into a plurality of contours.

[0078] Furthermore, the contour division unit 14 performs polygonal approximation on the contour, and divides the contour into multiple contours if there are multiple pairs of parallel straight lines among the contour sides that are longer than a predetermined value. Therefore, even if one contour contains the contours of multiple objects, it can be appropriately divided into multiple contours.

[0079] In addition, the image preprocessing unit 11 binarizes the depth information to generate a binary image, and the contour information generating unit 12 performs a contraction process on the binary image to generate contour information. Therefore, even if multiple objects overlap or touch each other, they can be extracted as the contours of separate objects.

[0080] When the difference between the depth information of a pixel in an image and the depth information of an adjacent pixel is equal to or greater than a predetermined value, the image preprocessing unit 11 determines that the pixel does not constitute a contour. Therefore, even if multiple objects overlap or touch each other, they can be extracted as the contours of separate objects.

[0081] The determination unit 13 determines whether or not an object is a detection target in each of the multiple consecutive images, and comprehensively determines whether or not an object is a detection target according to the determination results in each of the consecutive images. Therefore, the detection target object can be detected more appropriately.

[0082] Furthermore, when the number of sides constituting the contour of an object in the contour information is within a predetermined range, the determination unit 13 determines that the object is a detection target. Therefore, the detection target object can be appropriately detected.

[0083] (Embodiment 2) Fig. 8 is a diagram for explaining details of the control unit 10A of the information processing device according to the second embodiment. As shown in Fig. 8, the control unit 10A includes a contour information generating unit 12 and a determining unit 13A. The determining unit 13A also includes a rectangular image converting unit 123, a rectangular image selecting unit 124, a ratio calculating unit 125, and an object determining unit 126. Note that the same reference numerals are used to denote parts having the same configurations and functions as the control unit 10 of the information processing device 1 according to the first embodiment shown in Fig. 4, and detailed description thereof will not be repeated.

[0084] The rectangular image conversion unit 123 converts the contour information generated by the contour information generation unit 12 into a rectangle group image composed of a plurality of rectangles. The rectangular image conversion unit 123 calculates a rectangle (including a rectangle and a square) that exactly fits the contours of each of the plurality of objects included in the contour information, and converts it into a rectangle group image. For example, the method described in the above-mentioned non-patent document 2 can be used.

[0085] The rectangular image selection unit 124 selects rectangular images to be processed from the rectangle group image generated by the rectangular image conversion unit 123. For example, the rectangular image selection unit 124 sequentially selects rectangular images of objects closer to the 3D camera 2 from the rectangle group image, and sets the selected rectangular images as the processing targets.

[0086] The ratio calculation unit 125 calculates the aspect ratio of the rectangular image selected by the rectangular image selection unit 124. The method for calculating the aspect ratio of the rectangular image is the same as the method described in the first embodiment.

[0087] When the ratio calculated by the ratio calculation section 125 satisfies a predetermined threshold condition, the object determination section 126 determines that the rectangular image selected by the rectangular image selection section 124 is the detection object.

[0088] [Effects of the information processing device according to the first embodiment] As described above, in the information processing device according to the present embodiment, the determination unit 13A determines that a rectangular image is a detection target when the aspect ratio of the rectangular image satisfies a predetermined threshold condition. Therefore, even when the lighting conditions change, multiple objects overlap or touch each other, the detection target object can be appropriately detected.

[0089] [Software implementation example] The functions of the information processing device 1 (hereinafter referred to as the "device") are realized by a program for causing a computer to function as the device, and can be realized by a program for causing a computer to function as each control block of the device (in particular, the control unit 10, the control unit 10A).

[0090] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The control device and storage device execute the program to realize each function described in each of the above embodiments.

[0091] The program may be non-transitory and may be recorded in one or more computer-readable recording media. The recording media may or may not be included in the device. In the latter case, the program may be provided to the device via any wired or wireless transmission medium.

[0092] In addition, some or all of the functions of each of the control blocks can be realized by a logic circuit. For example, an integrated circuit in which a logic circuit that functions as each of the control blocks is formed is also included in the scope of the present invention. In addition, the functions of each of the control blocks can be realized by, for example, a quantum computer.

[0093] 〔summary〕 An information processing device according to a first aspect of the present invention includes: a contour information generating unit that acquires an image including depth information and generates contour information of an object extracted from the image; The apparatus further includes a determination unit that determines whether or not the object is a detection target based on information obtained from the vertical width and horizontal width of a rectangle surrounding the object's contour, among the contour information.

[0094] An information processing device according to a second aspect of the present invention is the information processing device according to the first aspect, The determination unit determines whether or not the object is a detection target based on information obtained from the number of sides that make up the object's contour in the contour information and the vertical width and horizontal width of a rectangle surrounding the object's contour.

[0095] An information processing device according to a third aspect of the present invention is the information processing device according to the first or second aspect, The determination unit determines whether or not the object is to be detected using at least one or more of the information obtained from the vertical width and horizontal width of a rectangle surrounding the contour of the object, the ratio between the vertical width and the horizontal width, the length of the vertical width and the horizontal width, the area of ​​the rectangle, and the actual width of the object.

[0096] An information processing device according to a fourth aspect of the present invention is the information processing device according to any one of the first to third aspects, the image is an image of the object flowing on an inclined slope, When a direction of a long side of the contour of the object in the contour information does not match an inclination direction of the slope, the determination unit determines that the object with that contour is the object to be detected.

[0097] An information processing device according to a fifth aspect of the present invention is the information processing device according to any one of the first to fourth aspects, The image processing device further includes a contour dividing section that approximates the contour with a plurality of straight lines, and divides the contour into a plurality of contours when there are a plurality of pairs of parallel straight lines.

[0098] An information processing device according to a sixth aspect of the present invention is the information processing device according to any one of the first to fourth aspects, The image processing device further includes a contour dividing section that performs polygonal approximation on the contour, and divides the contour into a plurality of contours if there are a plurality of pairs of parallel straight lines among the sides of the contour that are longer than a predetermined value.

[0099] An information processing device according to a seventh aspect of the present invention is the information processing device according to any one of the first to sixth aspects, The contour information generating unit binarizes the depth information to generate a binary image, and performs a contraction process on the binary image to generate the contour information.

[0100] An information processing device according to an eighth aspect of the present invention is the information processing device according to any one of the first to seventh aspects, The contour information generating unit determines that a pixel does not constitute the contour when a difference between depth information of the pixel in the image and depth information of an adjacent pixel is equal to or greater than a predetermined value.

[0101] An information processing device according to a ninth aspect of the present invention is the information processing device according to any one of the first to eighth aspects, the images including the depth information are a plurality of consecutive images; The determination unit is determining whether the object is the detection target in each of the plurality of consecutive images; Depending on the determination results for each of the consecutive images, a comprehensive determination is made as to whether or not the object is the detection target.

[0102] An information processing device according to a tenth aspect of the present invention comprises: a contour information generating unit that acquires an image including depth information and generates contour information of an object extracted from the image; a rectangle image conversion unit that converts the contour information generated by the contour information generation unit into a rectangle group image composed of a plurality of rectangles; a rectangular image selection unit that selects a rectangular image to be processed from the rectangle group image generated by the rectangular image conversion unit; a ratio calculation unit that calculates an aspect ratio of the rectangular image selected by the rectangular image selection unit; The image processing device further includes an object determining unit that determines that the rectangular image selected by the rectangular image selecting unit is the object to be detected when the ratio calculated by the ratio calculating unit satisfies a predetermined threshold condition.

[0103] An information processing device according to an eleventh aspect of the present invention comprises: a contour information generating unit that acquires an image including depth information and generates contour information of an object extracted from the image; and a determination unit that determines that the object is a detection target when a number of sides constituting the object's contour, among the contour information, is within a predetermined range.

[0104] An information processing method according to a twelfth aspect of the present invention includes the steps of: acquiring an image including depth information and generating contour information of an object extracted from the image; determining whether the object is a detection target based on information obtained from the vertical width and horizontal width of a rectangle surrounding the object's contour, among the contour information; Includes.

[0105] A program according to a thirteenth aspect of the present invention includes: On the computer, acquiring an image including depth information and generating contour information of an object extracted from the image; a process of determining whether the object is a detection target based on information obtained from the vertical width and horizontal width of a rectangle surrounding the contour of the object, among the contour information; Execute the command.

[0106] [Additional Notes] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0107] 1. Information processing device 2. 3D Camera 10,10A Control section 11 Image preprocessing section 12 Contour information generation unit 13,13A Judgment part 14 Contour division section 20 Memory section 30 Image IF section 40 Communications Department 50 Input section 60 Output section 100 Information Processing Systems 131 Side number comparison section 132 Rectangle calculation part 133 Rectangle comparison part 134 Length Comparison Section 135 Area comparison section 136 Actual width comparison section 137 Flow Comparison Section 123 Rectangular Image Conversion Unit 124 Rectangular Image Selection 125 Ratio Calculation Section 126 Object Determination Unit

Claims

1. a contour information generating unit that acquires an image including depth information and generates contour information of an object extracted from the image; and a determination unit that determines whether the object is a detection target based on information obtained from the vertical width and horizontal width of a rectangle surrounding the contour of the object, among the contour information.

2. the determination unit determines whether or not the object is a detection target based on information obtained from the number of sides constituting the object's contour in the contour information and the vertical width and the horizontal width of a rectangle surrounding the contour of the object. The information processing device according to claim 1 .

3. the determination unit determines whether or not the object is a detection target by using at least one or more of the following information obtained from the height and width of a rectangle surrounding the contour of the object: a ratio between the height and width, a length between the height and width, an area of ​​the rectangle, and an actual width of the object; 3. The information processing device according to claim 1 or 2.

4. the image is an image of the object flowing on an inclined slope, when a direction of a long side of the contour of the object in the contour information does not match a gradient direction of the slope, the determination unit determines that the object having the contour is the object to be detected.

3. The information processing device according to claim 1 or 2.

5. a contour dividing unit that approximates the contour with a plurality of straight lines and divides the contour into a plurality of contours when there are a plurality of pairs of parallel straight lines; 3. The information processing device according to claim 1 or 2.

6. a contour dividing unit that performs polygonal approximation on the contour, and divides the contour into a plurality of contours when there are a plurality of pairs of parallel straight lines among the sides of the contour that are longer than a predetermined value; 3. The information processing device according to claim 1 or 2.

7. The contour information generating unit binarizes the depth information to generate a binary image, and performs a contraction process on the binary image to generate the contour information.

3. The information processing device according to claim 1 or 2.

8. the contour information generating unit determines that a pixel does not constitute the contour when a difference between depth information of the pixel in the image and depth information of an adjacent pixel is equal to or greater than a predetermined value; 3. The information processing device according to claim 1 or 2.

9. the images including the depth information are a plurality of consecutive images; The determination unit is determining whether the object is the detection target in each of the plurality of consecutive images; a comprehensive determination as to whether the object is the detection target or not depending on the determination results for each of the consecutive images; 3. The information processing device according to claim 1 or 2.

10. a contour information generating unit that acquires an image including depth information and generates contour information of an object extracted from the image; a rectangle image conversion unit that converts the contour information generated by the contour information generation unit into a rectangle group image composed of a plurality of rectangles; a rectangular image selection unit that selects a rectangular image to be processed from the rectangle group image generated by the rectangular image conversion unit; a ratio calculation unit that calculates an aspect ratio of the rectangular image selected by the rectangular image selection unit; An information processing device comprising: an object determination unit that determines that the rectangular image selected by the rectangular image selection unit is a detection object when the ratio of the ratio calculation unit satisfies a predetermined threshold condition.

11. a contour information generating unit that acquires an image including depth information and generates contour information of an object extracted from the image; a determination unit that determines that the object is a detection target when a number of sides constituting the object's contour, from the contour information, is within a predetermined range.

12. acquiring an image including depth information and generating contour information of an object extracted from the image; determining whether the object is a detection target based on information obtained from the vertical width and horizontal width of a rectangle surrounding the object's contour, among the contour information; An information processing method comprising:

13. On the computer, acquiring an image including depth information and generating contour information of an object extracted from the image; a process of determining whether the object is a detection target based on information obtained from the vertical width and horizontal width of a rectangle surrounding the contour of the object, among the contour information; An information processing program that executes the above.

Images