Information processing apparatus, information processing method, and computer readable recording medium

Abstract

The invention provides a technology capable of improving detection accuracy of an object. An information processing device (1) is provided with: a contour information generation unit (12) that acquires an image including depth information and generates contour information of an object extracted from the image; and a determination unit (13) that determines whether or not the object is the object to be detected on the basis of information from the longitudinal length and the lateral length of a rectangle surrounding the object contour in the contour information.

Description

Technical Field

[0001] The present invention relates to an information processing device for detecting an object reflected in a depth image. Background Art

[0002] There is a known technology for detecting objects in an image captured by a camera. For example, Patent Document 1 below discloses a system that detects and recognizes objects in a scene and obtains the 3D posture (three-dimensional position and orientation) of the object. In this system, in order to complete the missing depth edges of unobstructed parts in a warehouse captured by a single camera, new edges are connected to obtain the 3D posture of the parts in the warehouse.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Laid-Open No. 2009-115783 Summary of the invention

[0006] 1. Technical issues to be resolved

[0007] The above system performs detection of only the same shape under a certain shooting environment, so that a component can be picked up from a warehouse by a robot arm installed in an industrial robot for manufacturing and assembly, etc. However, even if a plurality of objects of different shapes and sizes are to be detected, even if the depth edge of the defective part is obtained using the technology of Patent Document 1, problems such as inconsistency with the object shape will occur, and the detection accuracy of the object in the image will be reduced.

[0008] An object of one embodiment of the present invention is to provide an information processing device or the like capable of improving detection accuracy for objects of different shapes.

[0009] (II) Technical solution

[0010] In order to solve the above-mentioned problem, an information processing device of one embodiment of the present invention comprises: a contour information generating unit, which acquires an image containing depth information and generates contour information of an object extracted from the image; and a judging unit, which judges whether the object is a detection target based on information in the contour information obtained from the vertical length and horizontal length of a rectangle surrounding the contour of the object.

[0011] In order to solve the above-mentioned problem, an information processing device of one embodiment of the present invention comprises: a contour information generating unit, which acquires an image including depth information and generates contour information of an object extracted from the image; a rectangular image transforming unit, which transforms the contour information generated by the contour information generating unit into a rectangular group image composed of a plurality of rectangles; a rectangular image selecting unit, which selects a rectangular image as a processing object from the rectangular group image generated by the rectangular image transforming unit; a ratio calculating unit, which calculates the aspect ratio of the rectangular image selected by the rectangular image selecting unit; and an object determining unit, which determines that the rectangular image selected by the rectangular image selecting unit is a detection object when the ratio of the ratio calculating unit satisfies a predetermined threshold condition.

[0012] In order to solve the above-mentioned problem, an information processing device of one embodiment of the present invention comprises: a contour information generating unit, which acquires an image containing depth information and generates contour information of an object extracted from the image; and a judging unit, which judges that the object is a detection object when the number of edges constituting the contour of the object in the contour information is within a specified range.

[0013] In addition, in order to solve the above-mentioned problem, an information processing method of one embodiment of the present invention includes the following steps: acquiring an image containing depth information, generating contour information of an object extracted from the image; and determining whether the object is a detection object based on information in the contour information obtained from the vertical and horizontal lengths of a rectangle surrounding the contour of the object.

[0014] In addition, in order to solve the above-mentioned problem, a recording medium of one embodiment of the present invention is a computer-readable recording medium. When the program is executed by a processor, the following steps are performed: an image containing depth information is acquired, and contour information of an object extracted from the image is generated; based on the information in the contour information obtained from the vertical and horizontal lengths of the rectangle surrounding the contour of the object, it is determined whether the object is a detection object.

[0015] (III) Beneficial effects

[0016] According to one aspect of the present invention, it is possible to improve the accuracy of detecting an object. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a block diagram showing a configuration example of an information processing system according to Embodiment 1 of the present invention.

[0018] Figure 2 This figure shows a situation in which a garbage collection truck drops garbage into a garbage pit in a garbage incineration facility.

[0019] Figure 3 This is a picture showing the inside of a garbage pit.

[0020] Figure 4 This is a diagram showing the details of the control unit of the information processing device.

[0021] Figure 5 This is a flowchart showing an example of the processing flow of the information processing device according to Embodiment 1 of the present invention.

[0022] Figure 6 Yes means Figure 5 Detailed flowchart of the object detection processing flow shown.

[0023] Figure 7 This is a diagram for explaining a case where a contour includes a plurality of objects.

[0024] Figure 8 This is a diagram showing the details of the control unit of the information processing device according to the second embodiment of the present invention.

[0025] Description of Reference Numerals

[0026] 1-information processing device; 2-3D camera; 10, 10A-control unit; 11-image pre-processing unit; 12-contour information generating unit; 13, 13A-determination unit; 14-contour segmentation unit; 20-storage unit; 30-image IF unit; 40-communication unit; 50-input unit; 60-output unit; 100-information processing system; 131-side number comparison unit; 132-rectangle calculation unit; 133-rectangle comparison unit; 134-length comparison unit; 135-area comparison unit; 136-actual width comparison unit; 137-flow comparison unit; 123-rectangular image transformation unit; 124-rectangular image selection unit; 125-ratio calculation unit; 126-object determination unit. DETAILED DESCRIPTION

[0027] In recent years, there has been a problem of adding incompatible materials (hereinafter referred to as incompatible materials) to waste incineration facilities. Adding incompatible materials to the incinerator can cause deterioration of combustion in the incinerator, blockage of the incinerator's ash discharge equipment, etc., and sometimes cause the incinerator to be shut down urgently, depending on the situation.

[0028] Furthermore, it is extremely difficult to accurately detect various inappropriate objects from a large amount of garbage being transported during transportation.

[0029] (Implementation Method 1)

[0030] An information processing device 1 according to an embodiment of the present invention can solve the above-mentioned problem. The information processing device 1 has the function of detecting inappropriate substances from the garbage transported to the garbage incineration facility. Specifically, the information processing device 1 detects inappropriate substances using images obtained by photographing the garbage being thrown into the garbage pit. In addition, the garbage pit will be described later based on Figure 3In the following description, inappropriate objects are referred to as detection objects. Specific examples of inappropriate objects will be described later.

[0031] (Configuration example of information processing system)

[0032] Figure 1 FIG. 1 is a block diagram showing an example of the configuration of an information processing system according to an embodiment of the present invention. Figure 1 As shown, the information processing system 100 includes an information processing device 1 and a 3D (stereo) camera 2. In the present embodiment, the information processing system 100 is described as being disposed in a waste incineration facility, but the present invention is not limited thereto and can be used in any system that detects an object.

[0033] The 3D camera 2 is a camera that can capture not only vertical and horizontal information but also depth. The 3D camera 2 is described as a stereo camera using a stereo method, but is not limited thereto. For example, it may be a ToF (Time of Flight) camera, a camera including structured lighting using a projector method or a laser spot method, etc.

[0034] The information processing device 1 includes a control unit 10, a storage unit 20, an image IF (Interface) unit 30, a communication unit 40, an input unit 50, and an output unit 60. As an example, the information processing device 1 may be a personal computer, a server, a workstation, etc. The control unit 10 has a function of controlling the information processing device 1 as a whole, which will be described in detail later.

[0035] The storage unit 20 is a storage device for storing various data used by the information processing device 1. As the storage unit 20, for example, a single unit such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), SDRAM (Synchronous DRAM), HDD (Hard Disk Drive), SSD (Solid-State Drive), SD (Secure Digital) card, eMMC (embedded Multi-Media Controller), or a combination of the above can be used.

[0036] The image IF unit 30 acquires an image including depth information captured by the 3D camera 2 and outputs it to the control unit 10. The 3D camera 2 outputs an image and outputs depth information of each pixel included in the image as a dynamic image at a predetermined frame cycle.

[0037] The image IF unit 30 is an interface for enabling the 3D camera 2 to communicate with the information processing device 1. The image IF unit 30 may be an interface for wired communication or an interface for wireless communication. For example, a USB (Universal Serial Bus), a LAN (Local-Area Network), a wireless LAN, etc. may be used as the IF unit.

[0038] The communication unit 40 is a part that communicates with other devices, and can use, for example, a wireless LAN (Local Area Network), a wired LAN, a WAN (Wide Area Network), a public line network, a mobile data communication network, or a combination of these networks.

[0039] The input unit 50 is a part that receives input of various data to the information processing device 1 , and for example, a keyboard, a pointing device such as a mouse, a touch panel, or a combination of these can be used.

[0040] The output unit 60 is a part of the information processing device 1 for outputting various data, and can use, for example, a display device such as a liquid crystal panel, a sound output device, a printer, or a combination of the above. In addition, the input unit 50 and the output unit 60 can also use, for example, a USB (Universal Serial Bus), a wireless LAN, etc.

[0041] (Photo of garbage image)

[0042] Figure 2 This is a diagram showing a situation in which a garbage collection truck 200 throws garbage into a garbage pit in a garbage incineration facility. Figure 3 This is a diagram showing the inside of a garbage pit. A garbage pit is a place where garbage collected at a garbage incineration facility is temporarily stored. The garbage in the garbage pit is sequentially sent to the incinerator for incineration. Figure 2 As shown, a plurality of doors such as doors 300A and 300B (hereinafter collectively referred to as doors 300 when no distinction is necessary) are provided in the waste incineration facility. Figure 3 As shown in FIG. 3 , a garbage pit is provided in front of the door 300. That is, when the door 300 is opened, a drop opening for dropping garbage into the garbage pit appears. Figure 2 As shown, the garbage collection truck 200 throws garbage into the garbage pit from the dropping port.

[0043] The 3D camera 2 is installed to be able to capture Figure 3 The slope 600 is used to flow the garbage. For example, Figure 2 and Figure 3 The 3D camera 2 is installed at the installation position 400 shown. The installation position 400 is located on the surface of each door 300. Therefore, when the 3D camera 2 is installed at the installation position 400, when the door 300 is opened, the 3D camera 2 is located above the slope 600, which is suitable for photographing garbage. Of course, the installation position of the 3D camera 2 can be set to any position that can photograph garbage flowing along the slope 600.

[0044] Figure 4 It is used to illustrate Figure 1 Detailed diagram of the control unit 10 of the information processing device 1 shown in FIG. Figure 4 As shown, the control unit 10 includes: an image pre-processing unit 11, a contour information generating unit 12, a determining unit 13, and a contour segmenting unit 14. In addition, the determining unit 13 includes: an edge number comparing unit 131, a rectangle calculating unit 132, a rectangle comparing unit 133, a length comparing unit 134, an area comparing unit 135, an actual width comparing unit 136, and a flow comparing unit 137.

[0045] When the 3D camera 2 is a stereo camera, two common RGB or monochrome camera sensors are included in the stereo camera, and the information from the two camera sensors is compared, and the depth information (distance information) of each pixel is output. Here, a common camera image can also be output. In addition, in this embodiment, since the shooting object of the stereo camera is garbage flowing on the slope, each frame image of the image (dynamic image) can be used as the detection object. In addition, although the contour is output as information in the following object contour detection process, the contour can also be output as an image.

[0046] The image pre-processing unit 11 binarizes the depth information contained in the image taken by the 3D camera 2. For example, the image pre-processing unit 11 compares the depth information of each pixel with a predetermined threshold value. If the depth information is above the threshold value, it is set to "1" (or set to "255" when the image is in bytes). If the depth information is less than the threshold value, it is set to "0", and the binarization is performed. As a result, the contour information generating unit 12 described later can obtain contour information containing only the contour of the object. The contour information includes various information that can be obtained from pixel values, such as the number of edges of the contour, the length of the edges, etc., and is not limited to specific information. In addition, the contour information can also be an image.

[0047] The depth information is, for example, 8 or 16 bits of information, and can also be processed as a grayscale image. The unit of the depth information is, for example, millimeters. In addition, the image of the depth information can be set to only the part where the contour is detected, and the pixels outside the contour can be set to an invalid value of "0", for example.

[0048] The image pre-processing unit 11 performs pre-processing such as shrinkage processing, separation processing, and noise removal processing on the binarized image (hereinafter referred to as the binarized image). By shrinking the binarized image, it is easy to separate adjacent objects. If the objects cannot be separated, multiple objects may be detected as the outline of a single object. Therefore, the image pre-processing unit 11 performs shrinkage processing as pre-processing.

[0049] In addition, the image pre-processing unit 11 compares the depth information of each pixel with the depth information of the adjacent pixels, and when there is a depth difference greater than a first predetermined value (e.g., 20 cm), the value of the deeper pixel is invalidated, for example, the binarized depth information is set to "0" and separation processing is performed. This is because if the binarized depth information is not set to "0", overlapping objects may be detected as the outline of a single object.

[0050] Furthermore, the image pre-processing unit 11 removes small points included in the binarized depth image, thereby performing noise removal processing on the binarized depth image.

[0051] The contour information generating unit 12 extracts the contours of the multiple objects contained in the binary image and generates contour information. For example, in extracting the contour of the object, the method described in the following non-patent document 1 can be used. The contour of an object can be a collection of positions (x values, y values) of points in the binary depth image. Here, the edge of a straight line of the object contour is two consecutive points. In addition, the contour information generated by the contour information generating unit 12 also includes the number of edges.

[0052] Non-patent document 1: Satoshi Suzuki and Keiichi Abe, "Topological Structural Analysis of Digitized Binary Images by Border Following", CVGIP 30, pp 32-46, 1985.

[0053] The determination unit 13 determines (detects) whether each contour is a detection target object (unsuitable object) based on the result of the contour information generation unit 12. Specifically, the edge number comparison unit 131 to the flow comparison unit 137 perform the determination as follows.

[0054] The edge number comparison unit 131 determines whether the edge number is within a specified range. If the edge number of the object contour is within the specified range, the object is a candidate for the detection target object, and the detection process continues. In addition, when the edge number of the object contour is not within the specified range, for example, when the number of edges is small, the edge number comparison unit 131 determines that the object is too small, and when the number of edges is large, the shape of the object is too complex, and the object is determined not to be the detection target object. In this way, when the number of edges of the object contour is not within the specified range, the object is determined to be an object outside the detection target, thereby reducing the amount of calculation.

[0055] When the object is a candidate for the detection target object, the rectangle calculation unit 132 calculates a rectangle (including a rectangle and a square) into which the outline of the object just enters. For example, the method described in the following non-patent document 2 can be used. In addition, the rectangle into which the outline of the object just enters refers to a rectangle whose four sides are all in contact with the outline of the object.

[0056] Non-patent literature 2: Godfried Toussaint, "Solving Geometric Problems with the Rotating Calipers", Proceedings of IEEE MELECON'83, Athens, Greece, May 1983.

[0057] When the ratio of the long side to the short side of the rectangle calculated by the rectangle calculation unit 132 is within a predetermined value range, the rectangle comparison unit 133 detects an object having a corresponding outline as a detection target object.

[0058] The length comparison unit 134 may calculate the length of the object based on the shape of the outline of the object, and detect the object as a detection target object when the length of the object is within a predetermined range.

[0059] When the ratio of the area of ​​the rectangle to the area of ​​the corresponding contour is within a predetermined range, the area comparing unit 135 may detect the object of the corresponding contour as the detection target object.

[0060] The actual width comparison unit 136 may determine the width of the object based on the shape of the contour of the object, and detect the object as a detection target object when the width of the object is within a predetermined range.

[0061] When the rectangle comparison unit 133 determines that the object is an object outside the detection target, the contour segmentation unit 14 determines whether the object can be segmented. When the contour segmentation unit 14 determines that the object can be segmented, it segments the object and causes the rectangle calculation unit 132 and the rectangle comparison unit 133 to perform the above-mentioned processing again. In addition, the method of segmenting the object will be described later.

[0062] As described above, the 3D camera 2 is installed at a position where it can capture garbage flowing on the slope. When liquid objects such as sewage are attached to the slope, the liquid objects attached to the slope are sometimes captured by the 3D camera 2 because the liquid objects are attached to the slope. The 3D camera 2, such as a stereo camera, gives depth to parts with different colors, so it is possible that liquid objects attached to the slope are mistakenly judged as detection targets.

[0063] Therefore, when the long side direction of the object outline is inconsistent with the inclination direction of the slope, the flow comparison unit 137 determines (detects) the object of the outline as a detection target object (inappropriate object). That is, when the long side direction of the object outline is consistent with the inclination direction of the slope, the flow comparison unit 137 determines that the object is a liquid object attached to the inclination direction of the slope. The determination unit 13 determines whether it is a detection target object through the processing described above.

[0064] (Processing Flow of Information Processing Device 1)

[0065] Figure 5 This is a flowchart showing an example of the processing flow of the information processing device 1 according to one embodiment of the present invention. First, the information processing device 1 starts to input images from the 3D camera 2 (S1). As described above, the images are input as moving images at a predetermined frame cycle.

[0066] The image pre-processing unit 11 obtains the next image (an image containing depth information of one frame) from the 3D camera 2, and compares the depth information of each pixel in the image with a predetermined threshold value. If the depth information is above the threshold value, it is set to "1" (or set to "255" when the image is in bytes); if the depth information is less than the threshold value, it is set to "0", and binarization is performed (S2).

[0067] Next, the image pre-processing unit 11 performs pre-processing such as contraction processing, separation processing, and noise removal processing on the binary image ( S3 ). Furthermore, the contour information generating unit 12 extracts contours of a plurality of objects included in the binary image and generates contour information ( S4 ).

[0068] Next, the determination unit 13 detects the detection target object based on the contour shapes of the plurality of objects included in the binary image ( S5 ). The details of this object detection process will be described later.

[0069] Next, the control unit 10 records information related to the detected detection target object (uncomfortable substance) in the storage unit 20 , and displays the information on the output unit 60 , such as a display device ( S6 ).

[0070] Finally, the control unit 10 determines whether to end the image input (S7). For example, if the user instructs the end of the process via the input unit 50 (S7, Yes), the process is ended. Otherwise, if the user does not instruct the end of the process via the input unit 50 (S7, No), the process returns to step S2 and repeats the process of step S2 and thereafter.

[0071] (Detailed Processing Flow of Object Detection Process (S5))

[0072] Figure 6 It is used for Figure 5 The flowchart for explaining the details of the object detection process (S5) is shown in FIG. Figure 6 In the description, a long object is taken as a detection target object (inappropriate object).

[0073] Long objects include umbrellas and logs. Umbrellas contain metal parts, which can cause combustion in the incinerator to deteriorate and clog the ash discharge equipment of the incinerator, so they are considered as detection objects (unsuitable objects). In addition, logs can clog the inlet of the incinerator before incineration, so they are considered as detection objects (unsuitable objects).

[0074] First, the determination unit 13 obtains the first contour from the plurality of contours extracted by the contour information generation unit 12 (S51). Then, the edge number comparison unit 131 determines whether the number of edges is within a predetermined range (S52). Furthermore, when the process of step S51 is executed again, the determination unit 13 obtains the next contour from the plurality of contours.

[0075] If the number of sides of the object's contour is not within the specified range (S52, No), the object is not a candidate for the detection target object, and the processing enters step S68. In addition, if the number of sides of the object's contour is within the specified range (S52, Yes), the rectangle calculation unit 132 calculates the rectangle that the object's contour just enters (S53). In addition, the rectangle comparison unit 133 determines whether the value obtained by dividing the short side of the rectangle by the long side (short side / long side) is less than 0.35 (the second specified value) (S54). Here, 0.35 is a coefficient used to determine whether it is a long strip object, and the coefficient can also be changed according to the shape of the object to be detected. In addition, the rectangle that the object's contour just enters refers to a rectangle in which all four sides of the rectangle are connected to the contour of the object.

[0076] When the value obtained by dividing the short side of the rectangle by the long side is less than 0.35 (S54, Yes), the object is a candidate for the detection target object, and the process proceeds to step S59. In addition, when the value obtained by dividing the short side of the rectangle by the long side is greater than 0.35 (S54, No), it is determined whether the contour segmentation unit 14 can segment the contour of the object (S55).

[0077] Figure 7 is a diagram used to illustrate the case where a contour contains multiple objects. Figure 7 As shown in the left figure of FIG. 1 , two objects are extracted as one contour. The contour segmentation unit 14 is as follows Figure 7 As shown in the central figure of , extract the part represented by the straight line in the outline. Figure 7 In the central figure, a plurality of straight lines are indicated by dotted lines. When the number of parallel straight line pairs is plural, the contour segmentation unit 14 Figure 7 The outline is divided into multiple outlines as shown in the right figure of .

[0078] In addition, the contour segmentation unit 14 performs polygonal approximation on the contour, reduces the number of contours, and approximates the contour to a smaller number of contours. In this way, only the longer sides of the contour are left. At this moment, it is equivalent to Figure 7 In addition, when the contour segmentation unit 14 has a plurality of parallel straight line pairs, as shown in the central figure, Figure 7 The contour is divided into multiple contours as shown in the right figure of . In addition, in order to improve the detection effect, these two methods can also be used at the same time.

[0079] When the contour is divided into a plurality of contours by the contour segmentation unit 14, the processing of steps S56 to S67 is performed according to the number of divided contours. In step S56, the rectangle calculation unit 132 obtains the next contour from the plurality of contours divided by the contour segmentation unit 14. In addition, the rectangle calculation unit 132 calculates the rectangle into which the contour of the object just enters (S57). In addition, the rectangle comparison unit 133 determines whether the value obtained by dividing the short side of the rectangle by the long side is less than 0.35 (S58).

[0080] When the value obtained by dividing the short side of the rectangle by the long side is less than 0.35 (S58, Yes), the object is a candidate for the detection target object, and the process proceeds to step S59. On the other hand, when the value obtained by dividing the short side of the rectangle by the long side is greater than 0.35 (S58, No), the object is not a candidate for the detection target object, and the process proceeds to step S67.

[0081] In step S59, the length comparison unit 134 calculates the length of the contour (the length of the strip). For example, the length comparison unit 134 searches for the closest point in the contour for each of the four points of the rectangle calculated in step S53 or S57. Next, the length comparison unit 134 searches for a point for which depth information is valid, for example, a point set to "1" (or "255" when the image is in bytes) by binarization of the depth information (a point that is not set to be invalid).

[0082] In addition, since the contour information is obtained from the binary image, it is not limited to having depth information at the same pixel position as the contour point. Here, the x-axis and y-axis are the axes of the normal image. When the depth is set to the z-axis, if the internal parameters of the stereo camera are used, the x-value and y-value of the actual size can be obtained based on the x-value, y-value and z-value of the pixel in the actual size (for example, millimeters). That is, all the four points of the depth image obtained here can be transformed into coordinates of the actual size, and the longest distance between two of the four points (excluding the diagonal line) is set as the length of the long strip contour.

[0083] Next, the length comparison unit 134 determines whether the length of the long strip-shaped contour is within a predetermined range (S60). If the length of the long strip-shaped contour is not within the predetermined range (S60, No), the object is not a candidate for the detection target object, and the process proceeds to step S67. In addition, if the length of the long strip-shaped contour is within the predetermined range (S60, Yes), the area comparison unit 135 calculates the ratio of the area of ​​the rectangle to the area of ​​the corresponding contour (S61).

[0084] Next, the area comparison unit 135 determines whether the ratio of the area of ​​the rectangle to the area of ​​the corresponding contour is within a predetermined range (S62). When the ratio of the area of ​​the rectangle to the area of ​​the corresponding contour is not within the predetermined range (S62, No), the object is not a candidate for the detection target object, and the process proceeds to step S67.

[0085] In addition, when the ratio of the area of ​​the rectangle to the area of ​​the corresponding contour is within the range of the specified value (S62, Yes), the actual width comparison unit 136 calculates the actual width of the strip-shaped contour using the same method as described in step S59 (S63). In this case, the shortest distance between two of the four points is set as the actual width of the strip-shaped contour.

[0086] Next, the actual width comparison unit 136 determines whether the actual width of the elongated outline is within a predetermined range ( S64 ). If the actual width of the elongated outline is not within the predetermined range ( S64 , No), the object is not a candidate for the detection target object, and the process proceeds to step S67 .

[0087] In addition, if the actual width of the long strip-shaped contour is within the prescribed range (S64, yes), the flow comparison unit 137 determines whether the long side direction of the object contour is consistent with the inclination direction of the slope (whether it is parallel) (S65). When the long side direction of the object contour is consistent with the inclination direction of the slope (S65, yes), the contour is not a candidate for the detection target object, and the process proceeds to step S67.

[0088] In addition, when the long side direction of the object contour does not match the inclination direction of the slope ( S65 , No), the determination unit 13 sets the object of the contour as the detection target object (unsuitable object) ( S66 ).

[0089] In step S67, the determination unit 13 determines whether the processing of the plurality of contours segmented by the contour segmentation unit 14 is completed. In step S54, when it is determined that the contours include only one contour, the process proceeds to step 68.

[0090] When the processing of the plurality of contours segmented by the contour segmentation unit 14 is not completed (S67, No), the process returns to step S56 and the process thereafter is repeated. In addition, when the processing of the plurality of contours segmented by the contour segmentation unit 14 is completed (S67, Yes), it is determined whether the processing of all contours in the binary image is completed (S68).

[0091] If there is a contour that has not been processed (S68, No), the process returns to step S51 and the process thereafter is repeated. If there is no contour that has not been processed (S68, Yes), the object detection process (S5) is terminated.

[0092] In addition, the above description is an example of detecting an inappropriate object using a single image of the image. In order to improve accuracy, the same object can also be tracked on consecutive single images, and only when the same object is detected in multiple images, the object is judged as an inappropriate object.

[0093] In addition, through Figure 6 By appropriately changing the numerical values ​​used in the flowchart shown, it is also possible to detect other objects other than the above, for example, to detect other cars using images output from a camera installed on a car, or to detect ships based on satellite photos of the sea surface. For example, by setting the threshold of the value obtained by dividing the short side of a rectangle by the long side as a unique value for the car to be detected, it is also possible to detect a specific car.

[0094] (Effects of Information Processing Device 1)

[0095] As described above, according to the information processing device 1 of this embodiment, the determination unit 13 determines whether the object is a detection target based on the information obtained from the vertical length and horizontal length of the rectangle surrounding the object's outline in the outline information. Therefore, even if the lighting conditions change, a plurality of objects overlap or touch, etc., the detection target object can be properly detected.

[0096] Furthermore, the determination unit 13 determines whether the object is a detection target based on the number of sides constituting the object contour in the contour information and information obtained from the vertical and horizontal lengths of the rectangle surrounding the object contour. Therefore, the detection target object can be detected more appropriately.

[0097] In addition, the determination unit 13 uses one or more of the ratio of the vertical length to the horizontal length, the length of the vertical length to the horizontal length, the area of ​​the rectangle, and the actual width of the object as information obtained from the vertical length and the horizontal length of the rectangle surrounding the object contour to determine whether the object is a detection target. Therefore, the detection target object can be detected more appropriately.

[0098] When the long side direction of the contour does not match the inclination direction of the slope, the determination unit 13 detects the object of the contour as the detection target object. Therefore, even when liquid such as sewage is attached to the slope, it is possible to prevent erroneous determination as the detection target object.

[0099] Furthermore, the contour segmentation unit 14 approximates the contour using a plurality of straight lines, and segment the contour into a plurality of contours when there are a plurality of pairs of parallel straight lines. Therefore, even when contours of a plurality of objects are included in one contour, it is possible to appropriately segment the contour into a plurality of contours.

[0100] Furthermore, the contour segmentation unit 14 performs polygonal approximation on the contour and segments the contour into a plurality of contours when a plurality of parallel straight line pairs exist in the side of the contour longer than a predetermined value. Therefore, even when the contours of a plurality of objects are included in one contour, it can be appropriately segmented into a plurality of contours.

[0101] Furthermore, the image pre-processing unit 11 binarizes the depth information to generate a binarized image, and the outline information generating unit 12 performs shrinkage processing on the binarized image to generate outline information. Therefore, even when a plurality of objects overlap or touch each other, the outlines of different objects can be extracted separately.

[0102] When the depth information of a pixel in the image differs from the depth information of an adjacent pixel by a predetermined value or more, the image pre-processing unit 11 determines that the pixel is a pixel that does not constitute an outline. Therefore, even when multiple objects overlap or touch, the outlines of different objects can be extracted separately.

[0103] In each of the plurality of continuous images, the determination unit 13 determines whether the object is a detection target, and comprehensively determines whether the object is a detection target based on the determination results in each of the continuous images. Therefore, the detection target object can be detected more appropriately.

[0104] Furthermore, when the number of sides constituting the object contour in the contour information is within a predetermined range, the determination unit 13 determines that the object is a detection target. Therefore, the detection target object can be appropriately detected.

[0105] (Implementation method 2)

[0106] Figure 8 1 is a diagram for explaining the details of the control unit 10A of the information processing device according to the second embodiment. Figure 8 As shown in FIG. 1 , the control unit 10A includes: an outline information generating unit 12 and a determining unit 13A. In addition, the determining unit 13A includes: a rectangular image conversion unit 123, a rectangular image selection unit 124, a ratio calculation unit 125, and an object determining unit 126. Figure 4 The same components and functions as those of the control unit 10 of the information processing device 1 according to the first embodiment are denoted by the same reference numerals, and detailed description thereof will not be repeated.

[0107] The rectangular image conversion unit 123 converts the contour information generated by the contour information generating unit 12 into a rectangular group image composed of a plurality of rectangles. The rectangular image conversion unit 123 calculates the rectangle (including a rectangle and a square) into which the contours of the plurality of objects included in the contour information just fit, and converts the rectangle into the rectangular group image. For example, the method described in the above-mentioned non-patent document 2 can be used.

[0108] The rectangular image selection unit 124 selects a rectangular image to be processed from the rectangular group images generated by the rectangular image conversion unit 123. For example, the rectangular image selection unit 124 sequentially selects a rectangular image of an object closer to the 3D camera 2 from the rectangular group images and selects the rectangular image as a processing target.

[0109] The ratio calculation unit 125 calculates the aspect ratio of the rectangular image selected by the rectangular image selection unit 124. The method of calculating the aspect ratio of the rectangular image is the same as the method described in the first embodiment.

[0110] When the ratio calculated by the ratio calculation unit 125 satisfies a predetermined threshold condition, the object determination unit 126 determines that the rectangular image selected by the rectangular image selection unit 124 is a detection target object.

[0111] (Effects of the Information Processing Device of Embodiment 1)

[0112] As described above, according to the information processing device of this embodiment, when the aspect ratio of the rectangular image satisfies a predetermined threshold condition, the determination unit 13A determines that the rectangular image is a detection target object. Therefore, even when the lighting conditions change, or when multiple objects overlap or touch each other, the detection target object can be properly detected.

[0113] (Software-based implementation example)

[0114] The functions of the information processing device 1 (hereinafter referred to as “device”) can be realized by a program that causes a computer to function as the device and a program that causes a computer to function as each control module (particularly the control unit 10 and the control unit 10A) of the device.

[0115] In this case, as hardware for executing the above-mentioned program, the above-mentioned device includes a computer, and the computer includes: at least one control device (such as a processor) and at least one storage device (such as a memory). The above-mentioned program is executed by the control device and the storage device, thereby realizing each function described in the above-mentioned embodiments.

[0116] The program is not temporary and can be stored in one or more computer-readable recording media. The device may or may not have the recording medium. In the latter case, the program can be provided to the device via any transmission medium, whether wired or wireless.

[0117] In addition, part or all of the functions of the above-mentioned control modules can also be realized by logic circuits. For example, an integrated circuit having a logic circuit that functions as the above-mentioned control modules is also included in the scope of the present invention. In addition, for example, the functions of the above-mentioned control modules can also be realized by a quantum computer.

[0118] (Summarize)

[0119] The information processing device of method 1 of the present invention comprises: a contour information generating unit, which acquires an image including depth information and generates contour information of an object extracted from the image; and a judging unit, which judges whether the object is a detection target based on information in the contour information obtained from the vertical length and horizontal length of a rectangle surrounding the contour of the object.

[0120] The information processing device of mode 2 of the present invention is the information processing device described in mode 1, wherein the determination unit determines whether the object is a detection target based on the number of edges constituting the contour of the object in the contour information and the information obtained from the vertical length and the horizontal length of the rectangle surrounding the contour of the object.

[0121] The information processing device of mode 3 of the present invention is the information processing device described in mode 1 or 2, wherein the determination unit uses one or more of the ratio of the longitudinal length to the transverse length, the length of the longitudinal length to the transverse length, the area of ​​the rectangle, and the actual width of the object as information obtained from the longitudinal length and the transverse length of the rectangle surrounding the outline of the object to determine whether the object is a detection object.

[0122] The information processing device of mode 4 of the present invention is the information processing device described in any one of modes 1 to 3, wherein the image is an image of the object flowing on an inclined slope, and when the long side direction of the object contour in the contour information is inconsistent with the inclination direction of the slope, the determination unit determines the object of the contour as the detection target object.

[0123] The information processing device of mode 5 of the present invention is the information processing device described in any one of modes 1 to 4, and further includes a contour segmentation unit, which uses multiple straight lines to approximate the contour and divides the contour into multiple contours when there are multiple pairs of parallel straight lines.

[0124] The information processing device of mode 6 of the present invention is the information processing device described in any one of modes 1 to 4, and further comprises a contour segmentation unit, which performs polygonal approximation on the contour and divides the contour into multiple contours when there are multiple pairs of parallel straight lines in the side of the contour that is longer than a specified value.

[0125] The information processing device according to mode 7 of the present invention is the information processing device described in any one of modes 1 to 6, wherein the contour information generating unit binarizes the depth information to generate a binary image, and performs contraction processing on the binary image to generate the contour information.

[0126] The information processing device of mode 8 of the present invention is the information processing device described in any one of modes 1 to 7, and when the difference between the depth information of a pixel in the image and the depth information of an adjacent pixel is greater than a specified value, the contour information generating unit determines that the pixel is a pixel that does not constitute the contour.

[0127] The information processing device of mode 9 of the present invention is the information processing device described in any one of modes 1 to 8, wherein the image containing the depth information is a plurality of continuous images, and in each of the plurality of continuous images, the determination unit determines whether the object is the detection object, and the determination unit comprehensively determines whether the object is the detection object based on the determination results in each of the continuous images.

[0128] The information processing device of method 10 of the present invention comprises: a contour information generating unit, which acquires an image including depth information and generates contour information of an object extracted from the image; a rectangular image transforming unit, which transforms the contour information generated by the contour information generating unit into a rectangular group image composed of a plurality of rectangles; a rectangular image selecting unit, which selects a rectangular image as a processing object from the rectangular group image generated by the rectangular image transforming unit; a ratio calculating unit, which calculates the aspect ratio of the rectangular image selected by the rectangular image selecting unit; and an object determining unit, which determines that the rectangular image selected by the rectangular image selecting unit is a detection object when the ratio of the ratio calculating unit satisfies a predetermined threshold condition.

[0129] The information processing device of method 11 of the present invention comprises: a contour information generating unit, which acquires an image including depth information and generates contour information of an object extracted from the image; and a judging unit, which judges that the object is a detection object when the number of edges constituting the contour of the object in the contour information is within a specified range.

[0130] The information processing method of mode 12 of the present invention includes the following steps: acquiring an image containing depth information, generating contour information of an object extracted from the image; and determining whether the object is a detection target based on information in the contour information obtained from the vertical and horizontal lengths of a rectangle surrounding the contour of the object.

[0131] The recording medium of method 13 of the present invention is a computer-readable recording medium, which stores a computer program. When the program is executed by a processor, the following steps are performed: an image containing depth information is acquired, and contour information of an object extracted from the image is generated; based on the information in the contour information obtained from the vertical and horizontal lengths of the rectangle surrounding the contour of the object, it is determined whether the object is a detection object.

[0132] (Appendix)

[0133] The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope of the claims. Embodiments obtained by appropriately combining technical solutions disclosed in different embodiments are also included in the technical scope of the present invention.

Claims

1. An information processing device, characterized in that: have: a contour information generating unit that acquires an image including depth information and generates contour information of an object extracted from the image; and A determination unit determines whether the object is a detection target based on information obtained from a vertical length and a horizontal length of a rectangle surrounding the outline of the object in the outline information.

2. The information processing device according to claim 1, characterized in that The determination unit determines whether the object is a detection target based on the number of sides constituting the outline of the object in the outline information and information obtained from the vertical length and the horizontal length of a rectangle surrounding the outline of the object.

3. The information processing device according to claim 1 or 2, characterized in that: The determination unit uses one or more of the ratio of the longitudinal length to the transverse length, the length of the longitudinal length to the transverse length, the area of ​​the rectangle, and the actual width of the object as information obtained from the longitudinal length and the transverse length of the rectangle surrounding the outline of the object to determine whether the object is a detection object.

4. The information processing device according to claim 1 or 2, characterized in that: The image is an image of the object flowing on an inclined slope, When the long side direction of the object outline in the outline information does not coincide with the inclination direction of the slope, the determination unit determines that the object of the outline is the detection target object.

5. The information processing device according to claim 1 or 2, characterized in that: The device further includes a contour segmentation unit that approximates the contour using a plurality of straight lines and, when a plurality of parallel straight line pairs exist, segments the contour into a plurality of contours.

6. The information processing device according to claim 1 or 2, characterized in that: The device further includes a contour segmentation unit that performs polygonal approximation on the contour and, when a plurality of parallel straight line pairs exist in a side of the contour that is longer than a predetermined value, segments the contour into a plurality of contours.

7. The information processing device according to claim 1 or 2, characterized in that: The contour information generating unit binarizes the depth information to generate a binarized image, and performs contraction processing on the binarized image to generate the contour information.

8. The information processing device according to claim 1 or 2, characterized in that: When the difference between the depth information of a pixel in the image and the depth information of an adjacent pixel is equal to or larger than a predetermined value, the contour information generating unit determines that the pixel is a pixel that does not constitute the contour.

9. The information processing device according to claim 1 or 2, characterized in that: The image containing the depth information is a plurality of consecutive images, In each of the plurality of continuous images, the determination unit determines whether the object is the detection target. The determination unit comprehensively determines whether the object is the detection target based on the determination result in each of the continuous images.

10. An information processing device, characterized in that: have: a contour information generating unit, which acquires an image including depth information and generates contour information of an object extracted from the image; a rectangular image conversion unit that converts the outline information generated by the outline information generation unit into a rectangular group image composed of a plurality of rectangles; a rectangular image selection unit that selects a rectangular image as a processing object from among the rectangular group images generated by the rectangular image conversion unit; a ratio calculation unit that calculates an aspect ratio of the rectangular image selected by the rectangular image selection unit; and The object determination unit determines that the rectangular image selected by the rectangular image selection unit is a detection object when the ratio of the ratio calculation unit satisfies a predetermined threshold condition.

11. An information processing device, characterized in that: have: a contour information generating unit that acquires an image including depth information and generates contour information of an object extracted from the image; and A determination unit determines that the object is a detection target when the number of sides constituting the outline of the object in the outline information is within a predetermined range.

12. An information processing method, characterized in that: The process includes the following steps: Acquire an image containing depth information, and generate contour information of an object extracted from the image; Based on information obtained from the vertical length and horizontal length of a rectangle surrounding the outline of the object in the outline information, it is determined whether the object is a detection target.

13. A computer-readable recording medium storing a computer program, characterized in that: When the program is executed by a processor, the following steps are performed: Acquire an image containing depth information, and generate contour information of an object extracted from the image; Based on information obtained from the vertical length and horizontal length of a rectangle surrounding the outline of the object in the outline information, it is determined whether the object is a detection target.

Images