Object detection device, movement apparatus, object detection method, and program

The object detection device for moving devices uses a camera and depth sensor to accurately determine object depth and size, addressing the issue of overlapping objects and reducing false detection, thereby enhancing the safety and accuracy of navigation.

WO2024247435A1PCT designated stage expired Publication Date: 2024-12-05KONICA MINOLTA INC
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2024/009535
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-29
Filing Date
2024-03-12
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

Existing object detection systems for moving devices, such as automatic transport machines, face challenges in accurately identifying objects when multiple objects overlap, leading to potential misidentification of distance and type, particularly with unpredictable obstacles like children.

Method used

An object detection device equipped with a camera and depth sensor that acquires images and depth information, detects objects, specifies their depth, and determines if misidentification occurs based on the object's size, using skeletal point detection and depth variation analysis to adjust the field of view and reduce false detection.

Benefits of technology

The system effectively reduces false object detection by accurately determining the depth and size of objects, even in overlapping scenarios, ensuring safe navigation of moving devices.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2024009535_05122024_PF_FP_ABST
    Figure JP2024009535_05122024_PF_FP_ABST
Patent Text Reader

Abstract

Provided are an object detection device, a movement apparatus, an object detection method, and a program that make it possible to facilitate reduction in erroneous detection of objects. An object detection device (10) is provided with a control unit (11). The control unit (11) acquires a captured image and depth information within the viewing range of the captured image. The control unit (11) detects a prescribed object from the captured image. The control unit (11) identifies the depth of the prescribed object that has been detected. The control unit (11) determines, on the basis of the size of the prescribed object, whether or not said object is erroneously recognized.
Need to check novelty before this filing date? Find Prior Art

Description

Object detection device, moving device, object detection method and program

[0001] The present disclosure relates to an object detection device, a moving device, an object detection method, and a program.

[0002] Recently, there have been mobile devices, such as automatic guided vehicles, that can detect obstacles in their path. The mobile devices identify the location of the obstacle based on images captured by a camera or the like, and perform movement control to stop or bypass the obstacle. Patent Literature 1 discloses a robot that acquires information on the position and height of the obstacle to determine the type of obstacle and controls its own movement according to the type. When encountering an obstacle such as a child whose movements are difficult to predict, the robot increases the distance between itself and the obstacle and reduces the maximum movement speed of its own robot.

[0003] Japanese Patent Application Laid-Open No. 2008-200770

[0004] However, there are cases where multiple objects are positioned overlapping each other in a certain direction as viewed from the detection position, which poses a problem that the distance to the object or the type of the object may be misidentified.

[0005] An object of the present disclosure is to provide an object detection device, a moving device, an object detection method, and a program that can further reduce false detection of objects.

[0006] In order to achieve the above object, one aspect of the present disclosure is an object detection device including: an acquisition unit that acquires a captured image and information on depth in a field of view of the captured image; a detection unit that detects a predetermined object from the captured image; and an identification unit that identifies the depth of the detected predetermined object, wherein the identification unit determines whether the predetermined object has been mistakenly recognized based on a size of the predetermined object. Another aspect of the present disclosure is a mobile device including the object detection device; and an operation unit that moves the mobile device.

[0007] Another aspect of the present disclosure is an object detection method that acquires information about a captured image and the depth within the field of view of the captured image, detects a predetermined object from the captured image, and identifies the depth of the detected predetermined object, wherein the depth identification determines whether the predetermined object has been misidentified based on the size of the predetermined object.

[0008] Another aspect of the present disclosure is a program that causes a computer to function as an acquisition means for acquiring information about a captured image and the depth within the field of view of the captured image, a detection means for detecting a predetermined object from the captured image, and an identification means for identifying the depth of the detected predetermined object, wherein the identification means determines whether the predetermined object has been misidentified based on the size of the predetermined object.

[0009] According to the present disclosure, it is possible to further reduce false detection of an object.

[0010] FIG. 1 is a block diagram showing the functional configuration of a conveyance system of this embodiment. FIG. 2 is a diagram explaining the detection of a human body. FIG. 3 is a diagram explaining the detection of a human body. FIG. 4 is a diagram schematically showing a captured image and measurements by a depth sensor. FIG. 5 is a diagram schematically showing a captured image and measurements by a depth sensor. FIG. 6 is a flowchart showing a control procedure for object detection processing. FIG. 7 is a schematic diagram showing an example of a depth distribution. FIG. 8 is a diagram showing an example of setting a rectangular range. FIG. 9 is a flowchart showing another control procedure for object detection processing. FIG. 10 is a diagram showing another example of object detection.

[0011] An embodiment will be described below with reference to the drawings. Fig. 1 is a block diagram showing the functional configuration of a conveyance system 100 according to this embodiment. The conveyance system 100 includes an automatic conveyance machine 1 and a conveyance control device 4. The conveyance system 100 may include a plurality of automatic conveyance machines 1 for one conveyance control device 4.

[0012] The automatic guided vehicle 1, which is a moving device of this embodiment, has a carrier platform and is self-propelled. The automatic guided vehicle 1 transports an object placed on the carrier platform by self-propelling. The automatic guided vehicle 1 includes an object detection device 10, a camera 21 (photographing unit), a depth sensor 22 (depth measuring unit), a communication unit 23, a drive unit 24, and the like.

[0013] The camera 21 captures the direction of incident light to generate a captured image. The camera 21 may be capable of capturing color images in the visible light band. If the automated guided vehicle 1 can only move in limited directions, the camera 21 may have a specific field of view that includes the limited directions. The camera 21 may capture video or obtain still images at predetermined time intervals. Not all frame images of the video need necessarily be used for object detection. Some frame images may be extracted, and the extracted frame images may be treated in the same way as still images.

[0014] The depth sensor 22 measures the distance to an object located within the same range as the field of view or at least a portion thereof. Hereinafter, the distance from the depth sensor 22 to the object will be referred to as "depth." The depth sensor 22 may, for example, irradiate the object with laser infrared rays or ultrasonic waves and measure the distance to the object based on the reflected waves. The depth sensor 22 may be capable of outputting a map image showing the depth distribution. Alternatively, the depth sensor 22 may output numerical matrix data of simple measurement results.

[0015] The positions of the light receiving ports of the camera 21 and the depth sensor 22 are offset depending on the sizes of the camera 21 and the depth sensor 22. It is preferable that the camera 21 and the depth sensor 22 have light receiving ports at the same height on the main body (housing) of the automatic conveyor 1.

[0016] The object detection device 10 detects an object located in the direction of movement of the automatic conveyor 1 based on an image captured by the camera 21 and a measurement result from the depth sensor 22. The object may be, for example, a person. The object detection device 10 has a control unit 11 and a storage unit 12. The storage unit 12 includes a non-volatile memory such as a flash memory. A program 121 is stored in the non-volatile memory.

[0017] The control unit 11 acquires images captured by the camera 21 and depth information obtained by the depth sensor 22, and detects a predetermined object based on these. The control unit 11 has a hardware processor and memory. The hardware processor may be a CPU (Central Processing Unit). The memory is RAM (Random Access Memory), etc. The memory provides working memory space for the hardware processor and stores temporary data. The control unit 11 controls the movement status of the object detection device 10 in accordance with a control signal received by the communication unit 23 from the transport control device 4. Note that the control unit of the object detection device 10 and a control unit outside the object detection device 10 may be separate configurations (hardware processor and memory). The computer of this embodiment has at least the control unit 11. The obstacle detection operation by the control unit 11 will be described later.

[0018] The communication unit 23 controls the transmission and reception of data with an external device, in this case the transport control device 4, in accordance with a predetermined communication protocol. The communication protocol is, for example, a protocol for a wireless LAN (Local Area Network), etc. The communication unit 23 includes a network card and the like for controlling communication in accordance with this communication protocol.

[0019] The drive unit 24 outputs a signal to rotate the motor M and other components under the control of the control unit 11. This causes the motor M to rotate. In conjunction with the rotation of the motor M, movable parts such as the wheels, the direction of the wheels, and the direction of the main body operate, causing the automatic conveying machine 1 to move. The drive unit 24, the motor M, and other components constitute an operating unit that moves the automatic conveying machine 1 of this embodiment. The camera 21 and the depth sensor 22 may be included in the configuration of the object detection device 10.

[0020] The transport control device 4 receives object detection information from one or more automatic transport devices 1. Based on the received object detection information, the transport control device 4 transmits signals to the automatic transport devices 1 to control the position and operating state of each automatic transport device 1. The transport control device 4 includes a control unit 41, a storage unit 42, a communication unit 43, etc.

[0021] The control unit 41 has a hardware processor that performs arithmetic processing. The hardware processor may be a general-purpose CPU. The control unit 41 acquires the object detection result received from the automatic conveying machine 1 via the communication unit 43. The control unit 41 uses the acquired data to determine an operation change such as deceleration, stopping, or direction change (detouring) of the automatic conveying machine 1. The control unit 41 outputs a control signal indicating the determination content to the automatic conveying machine 1 via the communication unit 43.

[0022] The storage unit 42 includes a volatile memory (RAM) and a non-volatile memory. The RAM provides the CPU with working memory space and stores temporary data. The non-volatile memory stores programs, setting data, and the like. The programs include a program that controls the operation of the automatic conveyor 1. The non-volatile memory may be, for example, a flash memory and / or a hard disk drive (HDD).

[0023] The communication unit 43 controls communication with external devices. At least a part of the communication protocols that can be controlled by the communication unit 43 are the same as the communication protocols that can be controlled by the communication unit 23 of the automatic conveyance machine 1.

[0024] Next, the obstacle detection operation by the control unit 11 will be described. As described above, the control unit 11 detects a predetermined object based on the image captured by the camera 21 and the measurement results of the depth sensor 22. The predetermined object is, for example, a human body. The human body is detected, for example, by identifying multiple skeleton points and determining that the skeleton points match the structure of a human being.

[0025] The automated guided vehicle 1 detects an obstacle in the direction of movement while moving along a predetermined route or in a specified direction. The predetermined route may be determined, for example, by markings (signs) on the route. In this case, the automated guided vehicle 1 may detect the markings from images captured by the camera 21 and move by following the markings. Alternatively, the automated guided vehicle 1 may autonomously determine a direction of movement according to its current location based on the results of positioning using a satellite positioning system. When an obstacle is detected, the automated guided vehicle 1 operates under the control of the transport control device 4 to avoid colliding with the obstacle. For example, the automated guided vehicle 1 may slow down or temporarily stop until the obstacle moves, or may avoid the obstacle by taking a detour route.

[0026] It is often difficult to identify the spatial position of an object from a two-dimensional image captured by the camera 21. When an unexpected object is detected in the image captured by the camera 21, the automated guided vehicle 1 determines the distance to the object based on the measurement results from the depth sensor 22. This allows it to determine whether the object is on the route.

[0027] 2A and 2B are diagrams illustrating the detection of a human body B. As shown in FIG. 2A , the human body B is detected along with its posture by identifying the skeletal positions of the arm, leg, waist, and neck joints, as well as the eyes, ears, and nose. Arm joints include the wrist, elbow, and shoulder. Leg joints include the ankle and knee. Some of these joints may be invisible due to the orientation of the human body or other overlapping objects. Skeletal detection may use a skeletal detection algorithm based on conventionally known image recognition technology using a machine learning model or the like. The detected skeletal positions may be a combination different from the above depending on the algorithm used. Note that in skeletal detection, some parts of the human body may not be identified, for example, the feet may be hidden and not visible. In this case, the position of the unidentified parts may be appropriately estimated based on the size of other parts other than the invisible parts.

[0028] As shown in FIG. 2B , the distance h (number of pixels) in the height direction of the human body B along the skeletal points roughly corresponds to the height. For example, the distance h can be calculated as the sum of the distances between the eyes and neck, the neck and waist, the waist and knees, and the knees and ankles. This allows for the approximate calculation of the height of a person who is crouching or squatting. When the human body B is standing upright, the distance h is approximately the same as the linear distance between the skeletal points of the head (eyes) and the skeletal points of the ankles (ankles). Since precise height determination is not necessary, a certain degree of error in the distance h is acceptable. The angle of view α corresponding to the distance h can be obtained using the distance h, the depth D of the human body B, and the configuration information of the camera 21. The height H corresponding to the actual height can be calculated from the depth D and the angle of view α. Alternatively, the vertical width H0 at the position of the depth D can be obtained from the depth D and the configuration information of the camera. The height H can be calculated using the width H0, the number of vertical pixels h0 in the image, and the distance h.

[0029] If the height H obtained as described above is not within a range appropriate for a human body, the detected object is not a human body. In other words, the detection of the human body B is a misidentification. Even when multiple objects including a human body are detected collectively as human bodies, a height H that is inappropriate for a human body may be obtained.

[0030] For example, multiple objects may overlap in an image. If the multiple objects are not properly separated, the size and depth of the objects may be misidentified. In particular, a human body B to be detected may be located behind another object, with only a portion of it visible. In this case, the human body B, which does not directly affect the movement of the automated guided vehicle 1, may be mistakenly recognized as being close to the automated guided vehicle 1 due to the overlapping object. Furthermore, depending on the positional relationship between the multiple objects and the automated guided vehicle 1, the parallax between the camera 21 and the depth sensor 22 may be large. Therefore, the depth of the human body B detected by the camera 21 may be mistakenly recognized as the depth of a closer object. The control unit 11 adjusts the field of view of the camera 21 and the depth sensor 22 to separate multiple objects at different distances.

[0031] 3A and 3B are diagrams schematically illustrating a captured image and measurements by the depth sensor 22. In the captured image shown in FIG. 3A , a pipe P is located on the right side of the direction of travel of the automated guided vehicle 1, extending from near the ceiling to the floor. A fixed object extending along a route, such as the pipe P, may be used as a marker indicating the route. A human body B is standing, partially hidden by the pipe P. Once the range of human body B is identified, the distance from the automated guided vehicle 1 to the closest point within the range of human body B (the shortest distance) is identified as the distance to human body B. If the pipe P is not accurately excluded from the range of human body B, the distance to the pipe P will be mistakenly recognized as the distance to human body B. In this case, the pipe P is recognized as an obstacle that is much taller than the person's height. Therefore, it can be determined that the object detection device 10 has mistakenly identified a person.

[0032] As shown in FIG. 3B , the light receiving apertures of the camera 21 and the depth sensor 22 are not located at the same position. Therefore, when viewing the same position of human body B, human body B may be visible on one side, but a pipe P may be located in front of human body B on the other side. In this case, the position of the pipe P may be mistakenly recognized as the position of human body B. On the other hand, the pipe P may cause the depth distribution of obstacles along the horizontal direction to be recognized as having significant irregularities. Therefore, based on the irregularities, the object detection device 10 can determine that it has mistakenly identified a person. In this way, the object detection device 10 needs to adjust the field of view between the camera 21 and the depth sensor 22.

[0033] 4 is a flowchart showing a control procedure by the control unit 11 of the object detection process executed by the object detection device 10. This object detection process, including the object detection method of this embodiment, is continuously executed based on the program 121 while the automatic conveyor 1 is moving.

[0034] The control unit 11 acquires image data from the camera 21 (S1). The control unit 11 acquires depth distribution data at a timing corresponding to the image data from the depth sensor 22 (S2). Processes S1 and S2 are operations of the control unit 11 as an acquisition unit, and correspond to the acquisition means in the program 121.

[0035] The control unit 11 detects portions corresponding to a human body from the acquired image data and identifies skeletal points of the human body (S3). Process S3 is an operation of the control unit 11 as a detection unit, and corresponds to the detection means in the program 121. The control unit 11 identifies the depth of a position corresponding to each skeletal point from the depth distribution data (S4). Process S4 is an operation of the control unit 11 as an identification unit, and corresponds to the identification means in the program 121.

[0036] The control unit 11 calculates the distance h based on the coordinates (pixels) of each skeleton point. The control unit 11 identifies the smallest value (closest point) among the depths of each skeleton point to determine the depth D. The control unit 11 calculates the angle of view α based on the depth D and the distance h. The control unit 11 calculates the height H of the human body based on the depth D and the angle of view α (S5).

[0037] The control unit 11 determines whether the height is within a reference range (S6). The reference range corresponds to the height of a human being and is the first reference range in this embodiment. This lower limit value may be set to include infants and elementary school students who are able to walk alone, or may be set assuming only adults. Whether or not infants and the like are taken into consideration may be set depending on the purpose and usage environment of the automatic transport vehicle 1. The upper limit value may be set appropriately assuming a tall person.

[0038] If it is determined that the height is within the reference range (S6; YES), the control unit 11 causes the communication unit 23 to transmit position information of the human body to the transport control device 4 (S7). Then, the processing of the control unit 11 returns to S1. If it is determined that the height is not within the reference range (S6; NO), the control unit 11 determines that the identification of the human body was a mistake and cancels the identification result (S7). Then, the processing of the control unit 11 returns to S1. Note that the control unit 11 may determine that the detected object is an object other than a human body and cause the communication unit 23 to transmit its position information to the transport control device 4.

[0039] As described above, the object detection device 10 may be able to distinguish misidentification by focusing on the width of the human body. When a part of the human body B overlaps another object (the pipe P), the depth distribution in the area determined to be a human body from the captured image changes discontinuously depending on the pipe P.

[0040] FIG. 5 is a schematic diagram showing an example of a depth distribution. This depth distribution indicates depths along a horizontal line in the image. In this embodiment, if the variation (smallness) of the depth distribution does not meet a standard, it is determined to be a misidentification. For example, if a range of width w is recognized as a human body, the depth distribution Dp of a pipe P is mixed in with the depth distribution Db of the actual human body B. A standard range of variation, i.e., the maximum allowable depth Dth, is determined for the smallest depth D0 within this range. Within the width w, only the portion of width wp showing the depth distribution Dp of the pipe P exhibits a depth less than the maximum depth Dth. Even within the actual human body portion, only the portion of width wb exhibits a continuous, similar depth distribution Db. As a result, the depth variation across the entire width w is significantly large. Therefore, the object detection device 10 determines that the human body has not been accurately recognized.

[0041] In determining this variation, for example, the object detection device 10 first determines the range of the human body and then determines whether there is a large horizontal variation in depth within the range of the human body. Alternatively, an outer frame including the human body may be set, for example, as a rectangular range, and the variation within the rectangular range may be determined. Figure 6 shows an example of setting a rectangular range F.

[0042] To determine whether or not an object is a human body, for example, the system determines whether the horizontal depth continues within an appropriate range within the area or rectangular area recognized as a human body. For example, using the point with the smallest depth D0 as a reference, it determines whether the portion with a depth equal to or less than the maximum depth Dth continues continuously within a range of a length equivalent to the width of the human body. The difference between the maximum depth Dth and the depth D0 corresponds to the second reference range in this embodiment. Note that the human body may be oriented sideways with respect to the field of view of the depth sensor 22. Therefore, the reference range for determining whether or not an object is a human body may be a range that covers both the horizontal and vertical width of the human body. The range for determining whether or not an object is a human body corresponds to the reference width in this embodiment. Alternatively, the orientation of the human body may be identified based on skeletal detection, and the second reference range and reference width may be determined according to that orientation. The reference range may be determined, for example, as a predetermined ratio of the second reference range.

[0043] In the above, the human body is detected by a skeleton detection algorithm, but if the height H and / or the orientation of the human body is not to be specified, it is not necessary to specify skeleton points. The human body may be detected by an object detection algorithm that simply detects a predetermined object using a known image recognition algorithm.

[0044] 7 is a flowchart showing another control procedure for the object detection process. This object detection process includes steps S13 to S18 instead of steps S3 to S6 in the object detection process described above. Steps S1, S2, S7, and S8 are the same in all object detection processes, so they are denoted by the same reference numerals and their explanations are omitted.

[0045] After step S2, the control unit 11 detects and identifies an object, in this case a human body (S13). Identification of the object may be performed using a conventionally known image recognition algorithm, a contour detection algorithm, or the like.

[0046] The control unit 11 sets a rectangular area that includes the identified object (S14). The rectangular area may extend along both the vertical and horizontal directions of the image and circumscribe the object. The control unit 11 identifies a point within the rectangular area with the smallest depth (distance) (S15). The control unit 11 sets the depth to the identified point (smallest depth) as the reference distance.

[0047] The control unit 11 extracts points (pixels) within the rectangular area whose depths are within the reference range from the reference distance (S16).

[0048] The control unit 11 detects the portion where the extracted pixels are continuously arranged in the horizontal direction and identifies the maximum width of the continuous arrangement (S17). That is, this maximum width is the width of the portion that is within the second reference range. The control unit 11 determines whether the maximum width is larger than the reference width (S18). The reference width may be set to, for example, 20-30 cm, assuming the normal width and thickness of the human body.

[0049] If it is determined that the maximum width is greater than the reference width (S18; YES), the process of the control unit 11 proceeds to S7. If it is determined that the maximum width is not greater than (equal to) the reference width (S18; NO), the process of the control unit 11 proceeds to S8.

[0050] FIG. 8 is a diagram illustrating another example of object detection. The object detection device 10 may use another technique using a machine learning model for image recognition, such as semantic segmentation, to segment a captured image into multiple regions. In this case, the floor, partitions, walls, ceiling, fluorescent lights, and the like are individually identified as regions. The control unit 11 detects a specific object by determining whether each of the multiple segmented regions corresponds to a specific object (human body). At the same time, field of view adjustment is performed between the camera 21 and the depth sensor 22, and the depth of the human body B is identified.

[0051] As described above, the object detection device 10 of this embodiment includes a control unit 11. The control unit 11, as an acquisition unit, acquires a captured image and depth information within the field of view of the captured image. The control unit 11, as a detection unit, detects a predetermined object, in this case, a human body, from the captured image. The control unit 11, as an identification unit, identifies the depth of the detected predetermined object. The control unit 11, as an identification unit, determines whether the predetermined object has been misidentified based on the size of the predetermined object. In this way, the object detection device 10 obtains the size of the object, such as its height and / or width, from not only the captured image but also the depth information. If this size is unnatural compared to the expected size of the identified object, the object detection device 10 can determine that the object has been misidentified. Therefore, the object detection device 10 can reduce the misidentification of a predetermined object.

[0052] The predetermined object is a human body. The control unit 11, which functions as a detection unit, can detect multiple skeletal points of the human body using a human skeleton detection algorithm. The control unit 11, which functions as an identification unit, calculates the height of the human body using the positions and depths of the multiple skeletal points, and determines that the human body has been misidentified if the calculated height is outside a first reference range. By identifying the skeletal points, the posture of the human body can be obtained. Therefore, the object detection device 10 can more accurately determine the height of the human body according to its height. In other words, even if a person is crouching or bending over, the height according to their height can be obtained with high accuracy.

[0053] The control unit 11 as the identification unit may determine whether a human body has been mistakenly recognized based on the variation in depth of multiple skeleton points. When another object is present in front of the human body, the depth distribution of the detected object varies greatly. Therefore, the object detection device 10 can determine whether a human body has been mistakenly recognized based on this depth distribution.

[0054] The control unit 11 as an identification unit determines that the predetermined object has been misidentified if the depth variation of the detected predetermined object does not satisfy a standard. Therefore, the object detection device 10 can more accurately determine whether the predetermined object has been misidentified by appropriately combining the image captured by the camera 21 and the measurement results of the depth sensor 22.

[0055] The control unit 11, which functions as a detection unit, detects a human body using a predetermined object detection algorithm. The control unit 11, which functions as an identification unit, identifies the minimum depth D0 within the range of the human body. If the width of a portion of the human body within a second reference range from the minimum depth D0 (a depth equal to or less than Dth) is equal to or less than the reference width, the control unit 11 determines that a human body has been mistakenly recognized. In this way, the object detection device 10 determines whether the recognized object has a surface shape and width appropriate for a human body. This allows the object detection device 10 to more accurately determine whether a human body has been mistakenly recognized.

[0056] Alternatively, when a human body is detected, the control unit 11 sets a rectangular range F that includes the human body in the captured image. The control unit 11 identifies the minimum depth D0 within the rectangular range F. The control unit 11 determines whether the width of the portion of the rectangular range F that is within a reference range from the minimum depth D0 (depth equal to or less than Dth) is equal to or less than a reference width. If the width is equal to or less than the reference width, the control unit 11 determines that a human body has been mistakenly recognized. In this way, the range for identifying the width may be delimited by a rectangle. This simplifies the width calculation process, thereby shortening the calculation time and reducing the calculation load.

[0057] Alternatively, the control unit 11 as a detection unit may segment the range of the predetermined object by semantic segmentation. The control unit 11 determines whether the predetermined object has been misrecognized based on the depth variation within the segmented range of the predetermined object. In this way, the predetermined object may not only be extracted from the captured image, but also the captured image may be segmented into regions. Even in this case, by appropriately aligning the depth distribution with the depth distribution and obtaining the depth distribution of the range of the predetermined object, it is possible to appropriately determine whether the predetermined object has been misrecognized.

[0058] The object detection device 10 may include a camera 21 that takes a photograph and generates a captured image, and a depth sensor 22 that measures depth. That is, the object detection device 10 may detect a predetermined object based on an image that it has taken and a depth that it has measured. This enables the object detection device 10 to detect objects that may become obstacles with greater accuracy than conventional devices, and in particular, reduces misidentification.

[0059] The automatic conveying machine 1, which is a moving device of this embodiment, includes the above-described object detection device 10, and a driving unit 24 and a motor M, which are operating units for moving the automatic conveying machine 1. Such an automatic conveying machine 1 can move appropriately while reducing the false recognition of objects that obstruct movement.

[0060] The object detection method of this embodiment acquires a captured image and depth information within the field of view of the captured image, detects a predetermined object from the captured image, and identifies the depth of the detected predetermined object. The depth identification determines whether the predetermined object has been misidentified based on the size of the predetermined object. This object detection method determines the size of the object, such as its height and / or width, from not only the captured image but also the depth information. If this size is unnatural compared to the expected size of the identified object, it is determined to be a misidentification. Therefore, this object detection method can reduce the misidentification of a predetermined object.

[0061] Furthermore, by installing and executing the program 121 for object detection on a computer, it is possible to easily and more accurately reduce the misidentification of a predetermined object.

[0062] The present disclosure is not limited to the above-described embodiment, and various modifications are possible. For example, both the determination of misrecognition based on the height H and the determination of misrecognition based on the width w may be performed. Furthermore, at least one of these may be used in combination with a determination using a machine learning model such as semantic segmentation.

[0063] In the above embodiment, the camera 21 captures a color image, but this is not limiting. As long as a predetermined object can be appropriately identified, the camera 21 may capture an image in a limited number of colors (wavelength bands).

[0064] In the above embodiment, the image captured by the camera 21 is aligned with the depth distribution of the depth sensor 22 before determining whether the object is a predetermined object (human body), but this is not limited to this. The object detection device 10 may determine the possibility of misidentification without alignment, and if there is a possibility of misidentification, align the image and then make a determination again.

[0065] In the above embodiment, the detection target has been described as a human body. For example, if the movement path of the automatic conveying machine 1 is determined in advance and no obstacles are left on this movement path, only a human body needs to be detected. However, the detection target is not limited to a human body. Of the multiple automatic conveying machines 1, an object other than the automatic conveying machine 1 itself may be detected. Alternatively, a wagon, a step stool or ladder, a chair, or the like having a predetermined shape may be detectable. The object detection device 10 may detect multiple of these.

[0066] On the other hand, although the pipe P has been described above as another object that may overlap in front of the predetermined target, it is not limited to this. The other object may be a chair, a desk or table leg, a step or ladder, a rope or tape, a pillar or stick, etc.

[0067] In the above description, the transport control device 4 controls the movement of one or more automatic transport devices 1. However, the transport control device 4 may also control the movement positions of the multiple automatic transport devices 1 in accordance with the relative positions of the multiple automatic transport devices 1.

[0068] In the above description, the automatic conveying machine 1 and the conveyance control device 4 are separate devices, but this is not limited to this. The automatic conveying machine 1 may perform all operations, including control operations, by itself. Furthermore, the camera 21 and the depth sensor 22 may not be built into the automatic conveying machine 1, and external devices may be attached to the automatic conveying machine 1.

[0069] Furthermore, in the above description, nonvolatile memory such as flash memory is cited as an example of a computer-readable medium for storing the program 121 related to object detection control of the present disclosure. However, the computer-readable medium is not limited to these. Other computer-readable media include other nonvolatile memories such as HDDs, EEPROMs, and MRAMs. Alternatively, the program 121 may be stored on a portable recording medium such as a CD-ROM or DVD disk. Furthermore, a carrier wave is also applicable to the present disclosure as a medium for providing the program data via a communication line. Furthermore, the specific configurations, processing operation contents, and procedures shown in the above embodiments may be modified as appropriate without departing from the spirit of the present disclosure. The scope of the present invention includes the scope of the invention set forth in the claims and their equivalents.

[0070] The present disclosure can be used in an object detection device, a moving device, an object detection method, and a program.

[0071] REFERENCE SIGNS LIST 1 Automatic conveyor 10 Object detection device 11 Control unit 12 Memory unit 121 Program 21 Camera 22 Depth sensor 23 Communication unit 24 Driving unit 4 Conveyance control device 41 Control unit 42 Memory unit 43 Communication unit 100 Conveyance system B Human body F Rectangular area M Motor P Pipe

Claims

1. An object detection device comprising: an acquisition unit that acquires a captured image and information about the depth within the field of view of the captured image; a detection unit that detects a specified object from the captured image; and an identification unit that identifies the depth of the detected specified object, wherein the identification unit determines whether or not the specified object has been misrecognized based on the size of the specified object.

2. The object detection device of claim 1, wherein the specified object is a human body, the detection unit detects multiple skeletal points of the human body using a human skeletal detection algorithm, and the identification unit calculates the height of the human body using the positions and depths of the multiple skeletal points, and if the height is outside a first reference range, determines that the human body has been misidentified.

3. The object detection device according to claim 2, wherein the identification unit determines whether the human body has been mistakenly recognized based on the variation in depth of the multiple skeleton points.

4. The object detection device according to claim 1, wherein the identification unit determines that the specified object has been misidentified if the variation in depth of the detected specified object does not satisfy a criterion.

5. The object detection device of claim 4, wherein the specified object is a human body, the detection unit detects the human body using a specified object detection algorithm, and the identification unit identifies a minimum depth within the range of the human body, and if the width of a portion of the human body that is within a second reference range from the minimum depth is equal to or less than a reference width, determines that the human body has been misidentified.

6. The object detection device of claim 5, wherein, when the identification unit detects a human body, it sets a rectangular range in the captured image that contains the human body, identifies a minimum depth within the rectangular range, and if the width of the portion of the rectangular range that is within the reference range from the minimum depth is less than the reference width, it determines that the human body has been misidentified.

7. The object detection device of claim 1, wherein the detection unit divides the range of the specified object by semantic segmentation and determines whether the specified object has been misidentified based on the variation in depth within the range of the specified object.

8. The object detection device according to claim 1, comprising: an imaging unit that captures an image and generates the captured image; and a depth measurement unit that measures the depth.

9. A moving device comprising: an object detection device according to any one of claims 1 to 8; and an operating unit that moves the device itself.

10. An object detection method comprising: acquiring information on a captured image and the depth within the field of view of the captured image; detecting a specified object from the captured image; and identifying the depth of the detected specified object, wherein, in identifying the depth, it is determined whether or not the specified object has been misrecognized based on the size of the specified object.

11. A program that causes a computer to function as an acquisition means for acquiring information about a captured image and the depth within the field of view of the captured image, a detection means for detecting a specified object from the captured image, and an identification means for identifying the depth of the detected specified object, wherein the identification means determines whether or not the specified object has been misidentified based on the size of the specified object.

Citation Information

Patent Citations

  • Weight detection method, human body characteristic parameter detection method and device

    CN113534189A

  • Moving object identifying device for vehicle

    JP1998100820A

  • Object-detecting device

    JP2006010652A

  • Biodetection method and device, electronic device, and storage medium

    JP2020523665A

  • Detection system, detection device, terminal device, display control device, detection method, and program

    JP2022048830A