Obstacle avoidance methods, object detection methods and devices, and robotic vacuum cleaners

Abstract

This invention provides an obstacle avoidance method, an object detection method and apparatus, and a robotic vacuum cleaner, which can effectively detect small objects. The object detection method includes the following steps: acquiring a grayscale image and a pair of matching images corresponding to the grayscale image; performing edge detection on the grayscale image to obtain the edge position and edge change angle of the object in the grayscale image; performing disparity matching on the pair of matching images through a conventional matching window to obtain a reference disparity for all pixels in the matching images; determining the size and orientation of a novel matching window based on the edge position and edge change angle of the object; performing disparity matching on the pair of matching images through the novel matching window to obtain an updated disparity for the edge pixels of the object in the matching images; performing disparity fusion between the updated disparity and the reference disparity to obtain the final matching disparity for all pixels in the matching images; and obtaining a corresponding depth image through depth calculation.

Description

Technical Field

[0001] This invention relates to the field of stereo matching ranging technology, and in particular to an obstacle avoidance method, an object detection method and device, and a sweeping robot. Background Technology

[0002] Obstacle avoidance for robots, such as robotic vacuum cleaners, has become a recurring topic. Initially, robots typically used rudimentary distance sensing methods, relying on shock-absorbing rubber materials combined with random movement paths to navigate indoors. Upon detecting an obstacle, they would randomly change direction until encountering the next obstacle, then change direction again. However, this method was simple, crude, only suitable for extremely small indoor spaces, and inefficient. Later, with the development of 3D structured light technology, this active optical ranging method was applied to robot obstacle avoidance, achieving significant progress.

[0003] However, current binocular and monocular structured light ranging technologies still have relatively low accuracy in detecting small objects. Especially when a robot vacuum is cleaning, it cannot detect small objects such as wires, pens, or chopsticks on the ground very well, making it difficult to avoid obstacles effectively. This can easily cause various abnormal problems for the robot vacuum, such as getting tangled in wires or sucking in pens or chopsticks, which can damage the internal structure of the robot vacuum. Summary of the Invention

[0004] One advantage of this invention is that it provides an obstacle avoidance method, an object detection method and apparatus, and a sweeping robot, which can effectively detect small objects, providing richer information for robot obstacle avoidance and other applications, making it easier to avoid small obstacles.

[0005] Another advantage of the present invention is that it provides an obstacle avoidance method, an object detection method and apparatus, and a sweeping robot. In one embodiment of the present invention, the object detection method can improve the detection accuracy of small objects.

[0006] Another advantage of the present invention is that it provides an obstacle avoidance method, an object detection method and apparatus, and a sweeping robot. In one embodiment of the present invention, the object detection method can significantly reduce the computational complexity of detecting small objects.

[0007] Another advantage of the present invention is that it provides an obstacle avoidance method, an object detection method and apparatus, and a sweeping robot, wherein, in one embodiment of the present invention, the object detection method can significantly improve depth resolution.

[0008] Another advantage of the present invention is that it provides an obstacle avoidance method, an object detection method and device, and a sweeping robot. In one embodiment of the present invention, the object detection method can use gradient calculation to determine the placement direction of small objects so as to determine the direction of a new matching window based on the placement angle of the small objects, thereby increasing the proportion of small objects in the new matching window.

[0009] Another advantage of the present invention is that it provides an obstacle avoidance method, an object detection method and apparatus, and a sweeping robot. In one embodiment of the present invention, the object detection method can use the disparity of a conventional matching window to determine the disparity search range of a novel matching window, so as to solve the problem of high computational complexity caused by re-performing a large-scale disparity search.

[0010] Another advantage of the present invention is that it provides an obstacle avoidance method, an object detection method and apparatus, and a sweeping robot, wherein, in one embodiment of the present invention, the object detection method can use edge detection to determine the position of small objects.

[0011] Another advantage of the present invention is that it provides an obstacle avoidance method, an object detection method and apparatus, and a sweeping robot. In one embodiment of the present invention, the object detection method can determine the size of a novel matching window based on the area where the small object is located, which helps to increase the proportion of small objects in the novel matching window.

[0012] Another advantage of this invention is that it provides an obstacle avoidance method, an object detection method and apparatus, and a robotic vacuum cleaner, wherein, to achieve the above objectives, expensive materials or complex structures are not required. Therefore, this invention successfully and effectively provides a solution that not only offers a simple obstacle avoidance method, object detection method and apparatus, and robotic vacuum cleaner, but also increases the practicality and reliability of the obstacle avoidance method, object detection method and apparatus, and robotic vacuum cleaner.

[0013] To achieve at least one of the above-mentioned advantages or other advantages and objectives of the present invention, the present invention provides an object detection method, comprising the steps of:

[0014] Obtain a grayscale image and a pair of matching images corresponding to that grayscale image;

[0015] Edge detection is performed on the grayscale image to obtain the edge positions and edge change angles of objects in the grayscale image;

[0016] The pair of matched images is subjected to disparity matching through a regular matching window to obtain the reference disparity of all pixels in the matched image;

[0017] Based on the edge position of the object in the grayscale image and the edge change angle of the object, the size and orientation of the novel matching window are determined.

[0018] The novel matching window is used to perform disparity matching on the pair of matching images to obtain the updated disparity of the object edge pixels in the matching images;

[0019] The updated disparity is fused with the reference disparity to obtain the final matching disparity for all pixels in the matching image; and

[0020] Based on the final matching disparity, the depth is calculated to obtain the corresponding depth image.

[0021] According to one embodiment of this application, the object detection method further includes the following steps:

[0022] Based on the edge position of the object in the grayscale image, the reference disparity corresponding to the edge pixel of the object in the matching image is used as the center to determine the disparity search range corresponding to the edge pixel of the object in the matching image, so that the new matching window performs disparity matching within the disparity search range.

[0023] According to an embodiment of this application, the step of performing edge detection on the grayscale image to obtain the edge position and edge change angle of an object in the grayscale image includes the following steps:

[0024] Gradient calculation is performed on the pixel values ​​of all pixels in the grayscale image to calculate the horizontal and vertical gradients of each pixel in the grayscale image.

[0025] Based on the horizontal and vertical gradients of each pixel in the grayscale image, the location of the object boundary in the grayscale image is determined, and the pixel values ​​within the predetermined area containing the object boundary are marked as label values; and

[0026] Based on the horizontal and vertical gradients of the pixels with the marked values ​​in the grayscale image, the placement angle of the object in the grayscale image is calculated, which is used as the angle of change of the object's edge in the grayscale image.

[0027] According to an embodiment of this application, the step of determining the size and orientation of a novel matching window based on the edge position and edge change angle of the object in the grayscale image includes the following steps:

[0028] Based on the edge change angle of the object in the grayscale image, adjust the orientation of the novel matching window to adapt the orientation of the novel matching window to the object in the grayscale image; and

[0029] Based on the region where the boundary of the object in the grayscale image is located, the size of the new matching window is adjusted so that the area of ​​the grayscale image closer to the camera uses a larger new matching window, and the area of ​​the grayscale image farther from the camera uses a smaller new matching window.

[0030] According to one embodiment of this application, in the step of adjusting the orientation of the novel matching window according to the edge change angle of the object in the grayscale image so that the orientation of the novel matching window adapts to the object in the grayscale image:

[0031] The rotation angle of the new matching window relative to the conventional matching window is equal to the placement angle of the object in the grayscale image.

[0032] According to one embodiment of this application, in the step of adjusting the size of the novel matching window based on the region where the boundary position of the object in the grayscale image is located, so that a larger novel matching window is used for regions of the grayscale image closer to the camera, and a smaller novel matching window is used for regions of the grayscale image farther from the camera:

[0033] The grayscale image is divided into regions a, b, and c from bottom to top. The height of the novel matching window corresponding to region a is half the height of the regular matching window; the height and width of the novel matching window corresponding to region b are 2 / 3 of the height and width of the novel matching window corresponding to region a; and the height and width of the novel matching window corresponding to region c are 2 / 3 of the height and width of the novel matching window corresponding to region b.

[0034] According to another aspect of this application, this application further provides an object detection device, comprising components communicatively connected to each other:

[0035] The acquisition module is used to acquire a grayscale image and a pair of matching images corresponding to that grayscale image;

[0036] The edge detection module is used to perform edge detection on the grayscale image to obtain the edge position and edge change angle of the object in the grayscale image;

[0037] The standard disparity matching module is used to perform disparity matching on the pair of matching images through the standard matching window to obtain the reference disparity of all pixels in the matching image;

[0038] A novel window confirmation module is used to determine the size and orientation of a novel matching window based on the edge position of the object in the grayscale image and the edge change angle of the object.

[0039] A novel disparity matching module is used to perform disparity matching on the pair of matched images through the novel matching window to obtain the updated disparity of the object edge pixels in the matched image;

[0040] A disparity fusion module is used to fuse the updated disparity with the reference disparity to obtain the final matching disparity for all pixels in the matching image; and

[0041] The depth calculation module is used to calculate the depth based on the final matched disparity to obtain the corresponding depth image.

[0042] According to one embodiment of this application, the edge detection module includes a gradient calculation module, a boundary determination module, and an angle calculation module that are communicatively connected to each other. The gradient calculation module is used to perform gradient calculation on the pixel values ​​of all pixels in the grayscale image to calculate the horizontal and vertical gradients of each pixel in the grayscale image. The boundary determination module is used to determine the object boundary position in the grayscale image based on the horizontal and vertical gradients of each pixel in the grayscale image to mark the object boundary and the pixel values ​​within a predetermined area in the grayscale image as marker values. The angle calculation module is used to calculate the placement angle of the object in the grayscale image based on the horizontal and vertical gradients of the pixels with the pixel values ​​as marker values, as the angle of change of the object edge in the grayscale image.

[0043] According to one embodiment of this application, the novel window confirmation module includes a direction adjustment module and a size adjustment module that are communicatively connected to each other. The direction adjustment module is used to adjust the direction of the novel matching window according to the edge change angle of the object in the grayscale image, so that the direction of the novel matching window is adapted to the object in the grayscale image. The size adjustment module is used to adjust the size of the novel matching window according to the region where the boundary position of the object in the grayscale image is located, so that the region closer to the camera uses a larger novel matching window and the region farther away from the camera uses a smaller novel matching window.

[0044] According to one embodiment of this application, the object detection device further includes a search range confirmation module communicatively connected to the conventional disparity matching module and the novel disparity matching module. The search range confirmation module is used to determine the disparity search range in the matching image corresponding to the object edge pixels based on the edge position of the object in the grayscale image, with the reference disparity corresponding to the object edge pixels in the matching image as the center. The novel disparity matching module is further used to perform disparity matching on the pair of matching images within the disparity search range corresponding to the object edge pixels through the novel matching window, so as to obtain the updated disparity of the object edge pixels in the matching image.

[0045] According to another aspect of this application, this application further provides an obstacle avoidance method, including the steps of:

[0046] Based on the depth image obtained by any of the object detection methods described above, the contour of the object is constructed to obtain the contour information and depth information of the object; and

[0047] Based on the object's contour and depth information, the next direction of movement is calculated to avoid the object.

[0048] According to another aspect of this application, this application further provides a depth computing apparatus, comprising:

[0049] Structured light camera module, used to acquire image data; and

[0050] A data processing module, communicatively connected to the structured light camera module, is used to perform the steps of any of the above-described object detection methods based on image data from the structured light camera module to perform depth calculation.

[0051] According to another aspect of this application, this application further provides a robotic vacuum cleaner, comprising:

[0052] The robot vacuum cleaner itself; and

[0053] The aforementioned depth calculation device is configured on the body of the sweeping robot and is used to provide depth images for the sweeping robot body. Attached Figure Description

[0054] Figure 1 This is a schematic flowchart of an object detection method according to an embodiment of this application;

[0055] Figure 2 A flowchart illustrating the edge detection step in the object detection method according to the above embodiments of this application is shown.

[0056] Figure 3 A flowchart illustrating the novel window confirmation step in the object detection method according to the above embodiments of this application is shown.

[0057] Figure 4 A specific example of the object detection method according to the above embodiments of this application is shown;

[0058] Figure 5 An example of calculating the object placement angle in the object detection method according to the above embodiments of this application is shown;

[0059] Figure 6A An example of a conventional matching window in the object detection method according to the above embodiments of this application is shown;

[0060] Figure 6B An example of a small matching window in the object detection method according to the above embodiments of this application is shown;

[0061] Figure 6C An example of a novel matching window in the object detection method according to the above embodiments of this application is shown;

[0062] Figure 7 An example of the design of the disparity search range in the object detection method according to the above embodiments of this application is shown;

[0063] Figure 8 An example of novel matching window resizing in the object detection method according to the above embodiments of this application is shown;

[0064] Figure 9 This is a block diagram of an object detection device according to an embodiment of this application;

[0065] Figure 10 This is a flowchart illustrating an circumvention method according to an embodiment of this application;

[0066] Figure 11 This is a block diagram of a depth computing device according to an embodiment of this application;

[0067] Figure 12 This is a block diagram of a robotic vacuum cleaner according to one embodiment of this application.

[0068] Key component symbols: 200, Object detection device; 210, Acquisition module; 220, Edge detection module; 221, Gradient calculation module; 222, Boundary determination module; 223, Angle calculation module; 230, Conventional parallax matching module; 240, Novel window confirmation module; 241, Direction adjustment module; 242, Size adjustment module; 250, Novel parallax matching module; 260, Parallax fusion module; 270, Depth calculation module; 280, Search range confirmation module; 400, Depth calculation device; 410, Structured light camera module; 420, Data processing module; 500, Robot vacuum cleaner; 510, Robot vacuum cleaner body.

[0069] The above description of the main component symbols, together with the accompanying drawings and specific embodiments, provides a more detailed explanation of the present invention. Detailed Implementation

[0070] The following description is intended to disclose the present invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art. The basic principles of the invention defined in the following description can be applied to other embodiments, modifications, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the invention.

[0071] Those skilled in the art should understand that, in the disclosure of this invention, the terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the above terms should not be construed as limiting this invention.

[0072] In this invention, the term "a" in the claims and specification should be understood as "one or more," that is, in one embodiment, the number of an element can be one, while in another embodiment, the number of the element can be multiple. Unless explicitly indicated in the disclosure of this invention that the number of the element is only one, the term "a" should not be construed as unique or single, and the term "a" should not be construed as a limitation on the quantity.

[0073] In the description of this invention, it should be understood that terms such as "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, terms such as "connected" or "linked" should be interpreted broadly. For example, it can refer to a fixed connection, a detachable connection, or an integral connection; it can refer to a mechanical connection or an electrical connection; it can refer to a direct connection or an indirect connection through a medium. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0074] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0075] To address the issue that existing home robots based on binocular and monocular structured light cannot detect small objects on the ground, the applicant has found that the main reasons are the inability to determine the location of small objects and low depth resolution. The low depth resolution is primarily due to the small object being submerged in the background because its parallax is close to that of the background and its small size within the matching window. Therefore, this application creatively proposes an obstacle avoidance method, an object detection method and device, and a robotic vacuum cleaner, which can effectively detect small objects, providing richer information for robot obstacle avoidance and other applications, facilitating the avoidance of small obstacles.

[0076] Specifically, refer to the accompanying drawings in the specification of this application. Figures 1 to 8 According to one embodiment of this application, an object detection method is provided, which may include the steps of:

[0077] S110: Obtain a grayscale image and a pair of matching images corresponding to the grayscale image;

[0078] S120: Perform edge detection on the grayscale image to obtain the edge position and edge change angle of the object in the grayscale image;

[0079] S130: Perform disparity matching on the pair of matching images through a regular matching window to obtain the reference disparity of all pixels in the matching image;

[0080] S140: Based on the edge position of the object in the grayscale image and its edge change angle, determine the size and orientation of the new matching window;

[0081] S150: Perform disparity matching on the pair of matched images through the new matching window to obtain the updated disparity of the object edge pixels in the matched images;

[0082] S160: Perform disparity fusion between the updated disparity and the reference disparity to obtain the final matching disparity for all pixels in the matching image; and

[0083] S170: Based on the final matching disparity, calculate the depth to obtain the corresponding depth image.

[0084] It is worth noting that the object detection method described in this application can not only determine the location of small objects and their edge change angles through edge detection, but also determine the size and direction of the novel matching window based on the edge position and edge change angle of the small objects, so as to increase the proportion of small objects in the novel matching window; at the same time, it further fuses the updated disparity obtained through the novel matching window with the reference disparity obtained through the conventional matching window to obtain the final matching disparity and depth image, which helps to effectively detect small objects and provides richer information for applications such as robot obstacle avoidance, making it easier to avoid small obstacles.

[0085] More specifically, such as Figure 1 As shown, the object detection method of this application may further include the following step between step S130 and step S150:

[0086] S180: Based on the edge position of the object in the grayscale image, and taking the reference disparity corresponding to the edge pixel of the object in the matching image as the center, determine the disparity search range corresponding to the edge pixel of the object in the matching image, so that the new matching window performs disparity matching within the disparity search range.

[0087] In this way, when performing disparity matching through the new matching window, not only can the disparity search range of small objects be reduced to decrease the amount of computation, but the matching error caused by the disparity search range being too large can also be reduced.

[0088] For example, such as Figure 4 As shown, taking an image acquired via structured light as the input image as an example, in step S110, the acquired images are a grayscale image and a speckle image simultaneously acquired by the structured light camera. It is understood that the structured light camera of this application can be, but is not limited to, a monocular structured light module or a binocular structured light module. When the structured light camera is implemented as a monocular structured light module, the input image is a grayscale image and a scene speckle image simultaneously acquired by the monocular structured light module, as well as a reference speckle image pre-stored in the monocular structured light module. That is, the scene speckle image and the reference speckle image are a pair of matched images. When the structured light camera is implemented as a binocular structured light module, the input image is a grayscale image and a pair of scene speckle images simultaneously acquired by the binocular structured light module. That is, the pair of scene speckle images are a pair of matched images. This application will not elaborate further on this.

[0089] It is worth noting that, such as Figure 6AAs shown, the conventional matching window in this application is typically implemented as a square window; however, small objects in the grayscale image are usually not horizontally placed, and their length is much greater than their width, resulting in a very small proportion of the small objects within the square window, thus increasing the matching error of the small objects. In this case, as... Figure 6B As shown, even if the proportion of small objects in the window is increased by reducing the size of the regular matching window to form a smaller matching window, if the small matching window is too small, there will be no more obvious features, which will also lead to the problem of increased matching error of small objects.

[0090] However, as Figure 6C As shown, the size and orientation of the novel matching window in this application can be determined based on the edge position and edge change angle of the object in the grayscale image. This results in a significantly larger proportion of small objects within the novel matching window compared to the conventional matching window, and the feature information of small objects within the novel matching window is also much greater than that within the smaller matching window. Therefore, the novel matching window in this application can significantly improve the matching accuracy of small objects. To further improve the matching accuracy of small objects, the orientation of the novel matching window should match the edge change angle of the object in the grayscale image as closely as possible; the size of the novel matching window should match the visual size of the object in the grayscale image as closely as possible.

[0091] Optionally, such as Figure 2 As shown, step S120 of the object detection method of this application may include the following steps:

[0092] S121. Perform gradient calculation on the pixel values ​​of all pixels in the grayscale image to calculate the horizontal and vertical gradients of each pixel in the grayscale image.

[0093] S122. Based on the horizontal and vertical gradients of each pixel in the grayscale image, determine the location of the object boundary in the grayscale image, and mark the object boundary and the pixel values ​​within a predetermined area in the grayscale image as marker values; and

[0094] S123. Based on the horizontal and vertical gradients of the pixels with the pixel value of the marker in the grayscale image, calculate the placement angle of the object in the grayscale image, and use it as the change angle of the object's edge in the grayscale image.

[0095] It is worth noting that in step S122 of this application, the predetermined area range may be implemented as a pixel area range such as 3*3 or 5*5; the marker value may be implemented as 1.

[0096] Furthermore, pixels corresponding to the horizontal gradient Gx and vertical gradient Gy with larger absolute values ​​are selected from all pixels in the grayscale image. The locations of these selected pixels are the object boundaries in the grayscale image. Then, the pixel values ​​within a 3x3 or 5x5 pixel area centered on the selected pixels are all assigned a value of 1, thus completing the marking of the object edges. At this point, if... Figure 5 As shown, based on the horizontal and vertical gradients of the pixels with marked values ​​in the grayscale image, the placement angle α of the object in the grayscale image can be calculated using the arctangent function arctan(Gy / Gx). This determines the change angle of the object's edge in the grayscale image, preparing for the design of the next step, a novel matching window. It is understood that in step S150 of this application, the object edge pixels in the matching image refer to pixels with a pixel value marked as 1 in the grayscale image; since the objects in the grayscale image are small, the object edge pixels can usually cover the object area.

[0097] Optionally, such as Figure 3 As shown, step S140 of the object detection method of this application may include the following steps:

[0098] S141: Adjust the orientation of the new matching window according to the edge change angle of the object in the grayscale image, so that the orientation of the new matching window adapts to the object in the grayscale image; and

[0099] S142: Adjust the size of the new matching window according to the region where the boundary position of the object in the grayscale image is located, so that the region closer to the camera in the grayscale image uses a larger new matching window, and the region farther away from the camera in the grayscale image uses a smaller new matching window.

[0100] It is worth noting that the novel matching window of this application is preferably implemented as a rectangular window, and the orientation adjustment of the novel matching window is achieved by rotating the novel matching window. For example, when the placement angle α of the object in the grayscale image is calculated, the novel matching window only needs to be rotated relative to the conventional matching window by the same angle α; that is, the rotation angle of the novel matching window relative to the conventional matching window is equal to the placement angle α of the object in the grayscale image, to ensure that the orientation of the novel matching window is adapted to the object in the grayscale image, so that the proportion of small objects in the novel matching window is significantly increased. It can be understood that in this application, the orientation of the novel matching window being adapted to the object in the grayscale image may refer to the fact that part of the edge of the novel matching window is parallel to the edge of the object in the grayscale image.

[0101] Furthermore, according to the imaging principle, the closer an object is to the camera, the larger its image; the farther an object is from the camera, the smaller its image. Therefore, in order to further increase the proportion of small objects in the novel matching window, in step S142 of this application, when the boundary position of the object in the grayscale image is in a region close to the camera, the size of the novel matching window is increased; when the boundary position of the object in the grayscale image is in a region far from the camera, the size of the novel matching window is decreased.

[0102] For example, taking the operation scenario of a robotic vacuum cleaner as an example, the grayscale image typically displays a ground image. Therefore, the lower the area in the grayscale image, the closer it is to the camera, and the higher the area, the farther away it is from the camera. For example, as... Figure 8 As shown, the grayscale image can be divided into three regions from bottom to top: region a, region b, and region c. Region a is closest to the camera and has the largest image of the object, so a larger novel matching window can be used for disparity matching. Region c is farthest from the camera and has the smallest image of the object, so a smaller novel matching window is needed for disparity matching.

[0103] Optionally, the height of the new matching window corresponding to region a can be half the height of the regular matching window; the height and width of the new matching window corresponding to region b can be 2 / 3 of the height and width of the new matching window corresponding to region a; and the height and width of the new matching window corresponding to region c can be 2 / 3 of the height and width of the new matching window corresponding to region b. It can be understood that, assuming the width and height of the regular matching window are both w, then the height of the new matching window corresponding to region a is w / 2 and the width is w; the height of the new matching window corresponding to region b is w / 3 and the width is 2w / 3; and the height of the new matching window corresponding to region c is 2w / 9 and the width is 4w / 9.

[0104] It is worth noting that after selecting a matching window (a conventional matching window or / and a novel matching window), performing disparity matching on a pair of matching images through the matching window to obtain the corresponding disparity is a fundamental method in this field, and this application will not elaborate on it further. However, since small objects exist in the background, based on the principle of disparity matching, the disparity of small objects and the background is relatively close. Therefore, the object detection method described in this application can refer to the disparity obtained through the conventional matching window to determine the disparity range where the small object is located, thereby enabling the novel matching window to search only within the disparity range where the small object is located, which helps to reduce the error caused by excessive disparity search.

[0105] In other words, during the disparity search process, due to the similarity between adjacent pixels, the disparity of the pixel to be calculated is very close to the disparity of its surrounding pixels that has already been calculated. Therefore, by referring to the disparity of the surrounding pixels, the disparity search range of small objects can be determined. This not only reduces the disparity search range of small objects and thus reduces the amount of computation, but also reduces the matching error caused by a disparity search range that is too large.

[0106] For example, such as Figure 7 As shown, the disparity search range for a square black pixel can refer to the disparity d of the previous pixel. That is, with the disparity d of the previous pixel as the center, the number of pixels L and R offset to the left and right by a small margin can be used to implement the disparity search range as (dL, d+R). In other words, in step S180 of this application, the corresponding reference disparity d (i.e., the disparity of the pixel with a pixel value of 1) is selected based on the pixel with a pixel value of 1 in the grayscale image; thus, the disparity search range is determined to be (dL, d+R). In this way, in step S150 of this application, the new matching window only needs to perform disparity matching within the range of (dL, d+R) to obtain the updated disparity of the object edge pixels in the matched image.

[0107] According to the above examples of this application, in step S160, the updated disparity can be directly used to replace the reference disparity of the corresponding pixel in the matching image to obtain the final matching disparity. Of course, in other examples of this application, it is also possible to first determine whether the updated disparity is abnormal, then replace the reference disparity of the corresponding pixel with the updated disparity that is determined to be normal, while retaining the reference disparity corresponding to the abnormal updated disparity; or, a weighted model can be used to fuse the updated disparity and the reference disparity to take into account both the reference disparity and the updated disparity during disparity fusion. This application will not elaborate further on this aspect.

[0108] Furthermore, in step S170 of this application, performing depth calculation based on disparity to obtain a depth image is a fundamental method in this field, and the specific process will not be described in detail here.

[0109] It is worth mentioning that, attached Figure 9 An object detection apparatus 200 according to one embodiment of this application is shown, which may include objects communicatively connected to each other:

[0110] The acquisition module 210 is used to acquire a grayscale image and a pair of matching images corresponding to the grayscale image;

[0111] The edge detection module 220 is used to perform edge detection on the grayscale image to obtain the edge position of the object in the grayscale image and its edge change angle;

[0112] The conventional disparity matching module 230 is used to perform disparity matching on the pair of matching images through a conventional matching window to obtain the reference disparity of all pixels in the matching image;

[0113] The novel window confirmation module 240 is used to determine the size and orientation of the novel matching window based on the edge position of the object in the grayscale image and its edge change angle.

[0114] The novel disparity matching module 250 is used to perform disparity matching on the pair of matching images through the novel matching window to obtain the updated disparity of the object edge pixels in the matching image;

[0115] The disparity fusion module 260 is used to fuse the updated disparity with the reference disparity to obtain the final matching disparity of all pixels in the matching image; and

[0116] The depth calculation module 270 is used to calculate the depth based on the final matching disparity to obtain the corresponding depth image.

[0117] In one example of this application, such as Figure 9 As shown, the edge detection module 220 may include a gradient calculation module 221, a boundary determination module 222, and an angle calculation module 223 that are communicatively connected to each other. The gradient calculation module 221 is used to perform gradient calculation on the pixel values ​​of all pixels in the grayscale image to calculate the horizontal and vertical gradients of each pixel in the grayscale image. The boundary determination module 222 is used to determine the object boundary position in the grayscale image based on the horizontal and vertical gradients of each pixel in the grayscale image to mark the object boundary and the pixel values ​​within the predetermined area where it is located as marker values. The angle calculation module 223 is used to calculate the placement angle of the object in the grayscale image based on the horizontal and vertical gradients of the pixels with the pixel values ​​as marker values, as the angle of change of the object edge in the grayscale image.

[0118] In one example of this application, such as Figure 9 As shown, the novel window confirmation module 240 may include a direction adjustment module 241 and a size adjustment module 242 that are communicatively connected to each other. The direction adjustment module 241 is used to adjust the direction of the novel matching window according to the edge change angle of the object in the grayscale image, so that the direction of the novel matching window is adapted to the object in the grayscale image. The size adjustment module 242 is used to adjust the size of the novel matching window according to the area where the boundary position of the object in the grayscale image is located, so that the area closer to the camera uses a larger novel matching window and the area farther away from the camera uses a smaller novel matching window.

[0119] According to the above embodiments of this application, as Figure 9 As shown, the object detection device 200 may further include a search range confirmation module 280 communicatively connected to the conventional disparity matching module 230 and the novel disparity matching module 250. This module is used to determine the disparity search range in the matching image corresponding to the object's edge pixels, based on the object's edge position in the grayscale image and centered on the reference disparity corresponding to the object's edge pixels. Simultaneously, the novel disparity matching module 250 is further used to perform disparity matching on the pair of matching images within the disparity search range corresponding to the object's edge pixels using the novel matching window, to obtain the updated disparity of the object's edge pixels in the matching image.

[0120] It is worth mentioning that, attached Figure 10 An obstacle avoidance method according to an embodiment of this application is shown, which may include the steps of:

[0121] S310: Based on the depth image obtained by the above object detection method, construct the contour of the object to obtain the contour information and depth information of the object; and

[0122] S320: Calculate the next movement direction based on the object's contour and depth information in order to avoid the object.

[0123] It is worth noting that the obstacle avoidance method described in this application can be applied to robots, but is not limited to this application. In this case, the depth information of the object (such as an obstacle) is the distance information between the robot and the object, so that the robot can calculate its next movement direction based on the outline information of the obstacle and the distance information between the robot and the object, so as to avoid the obstacle and thus achieve the obstacle avoidance function.

[0124] It is worth mentioning that, attached Figure 11 A depth calculation apparatus 400 according to an embodiment of this application is shown, which may include a structured light camera module 410 and a data processing module 420 communicatively connected to each other. The structured light camera module 410 is used to acquire image data input to the data processing module 420, and the data processing module 420 is used to perform the steps in the object detection method described above based on the image data to perform depth calculation.

[0125] It is worth mentioning that, attached Figure 12 A robotic vacuum cleaner 500 according to an embodiment of this application is shown. It may include a robotic vacuum cleaner body 510 and the aforementioned depth calculation device 400. The depth calculation device 400 is configured on the robotic vacuum cleaner body 510 and is used to provide a depth image to the robotic vacuum cleaner body 510 so that the robotic vacuum cleaner body 510 can perform obstacle avoidance operations based on the depth image, thereby giving the robotic vacuum cleaner 500 a better obstacle avoidance function.

[0126] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0127] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method of object detection, characterized by, The method comprises the steps of: acquiring a gray image and a pair of matching images corresponding to the gray image; when the gray image is acquired by a monocular structured light module, the pair of matching images corresponding to the gray image are a scene speckle image acquired by the monocular structured light module and a reference speckle image stored in the monocular structured light module in advance; when the gray image is acquired by a binocular structured light module, the pair of matching images corresponding to the gray image are a pair of scene speckle images acquired by the binocular structured light module at the same time; performing edge detection on the gray image to obtain an edge position of an object in the gray image and an edge change angle of the object; performing disparity matching on the pair of matching images by using a conventional matching window to obtain reference disparity of all pixels in the matching images; determining a size and a direction of a new matching window based on the edge position of the object in the gray image and the edge change angle of the object; performing disparity matching on the pair of matching images by using the new matching window to obtain updated disparity of object edge pixels in the matching images; performing disparity fusion on the updated disparity and the reference disparity to obtain final matching disparity of all pixels in the matching images; and calculating depth based on the final matching disparity to obtain a corresponding depth image. The object detection method further comprises the step of:

2. The object detection method according to claim 1, wherein determining a disparity search range corresponding to the object edge pixels in the matching image based on the edge position of the object in the gray image, with the reference disparity corresponding to the object edge pixels in the matching image as the center, so that the new matching window performs disparity matching in the disparity search range. The step of performing edge detection on the gray image to obtain an edge position of an object in the gray image and an edge change angle of the object comprises the steps of:

3. The object detection method according to claim 1 or 2, characterized in that, performing gradient calculation on pixel values of all pixels in the gray image to calculate horizontal direction gradient and vertical direction gradient of each pixel in the gray image; determining an object boundary position in the gray image based on the horizontal direction gradient and the vertical direction gradient of each pixel in the gray image, to mark pixel values in a predetermined area range of the object boundary and in the gray image as a marker value; and calculating a placement angle of the object in the gray image as a change angle of the object edge in the gray image according to the horizontal direction gradient and the vertical direction gradient of the pixels with the marker value in the gray image. The step of determining a size and a direction of a new matching window based on the edge position of the object in the gray image and the edge change angle of the object comprises the steps of:

4. The object detection method according to claim 3, wherein adjusting the direction of the new matching window according to the edge change angle of the object in the gray image, so that the direction of the new matching window is adapted to the object in the gray image; and adjusting the size of the new matching window according to the area where the object boundary position in the gray image is located, so that a larger new matching window is used in a region closer to the camera in the gray image, and a smaller new matching window is used in a region farther away from the camera in the gray image. ​ ​ 5. The object detection method according to claim 4, wherein In the step of adjusting the direction of the new matching window according to the edge change angle of the object in the gray image, the direction of the new matching window is adjusted to adapt to the object in the gray image, and in the step of adjusting the size of the new matching window according to the area where the boundary position of the object in the gray image is located, the larger the area is closer to the camera, and the smaller the area is farther from the camera. The rotation angle of the new matching window relative to the regular matching window is equal to the placement angle of the object in the gray image.

6. The object detection method of claim 4, wherein In the step of adjusting the size of the new matching window according to the area where the boundary position of the object in the gray image is located, the larger the area is closer to the camera, and the smaller the area is farther from the camera. The gray image is divided into area a, area b and area c from bottom to top, the height of the new matching window corresponding to the area a is half of the height of the regular matching window; the height and width of the new matching window corresponding to the area b are 2 / 3 of the height and width of the new matching window corresponding to the area a; the height and width of the new matching window corresponding to the area c are 2 / 3 of the height and width of the new matching window corresponding to the area b.

7. An object detection device, characterized by The system comprises: An acquisition module is configured to acquire a gray image and a pair of matching images corresponding to the gray image; when the gray image is acquired by a monocular structured light module, the pair of matching images corresponding to the gray image are a scene speckle image acquired by the monocular structured light module and a reference speckle image pre-stored in the monocular structured light module; when the gray image is acquired by a binocular structured light module, the pair of matching images corresponding to the gray image are a pair of scene speckle images simultaneously acquired by the binocular structured light module; An edge detection module is configured to perform edge detection on the gray image to obtain the edge position of the object in the gray image and the edge change angle of the object; A regular disparity matching module is configured to perform disparity matching on the pair of matching images by a regular matching window to obtain the reference disparity of all pixels in the matching image; A new window confirmation module is configured to determine the size and direction of a new matching window based on the edge position of the object in the gray image and the edge change angle of the object; A new disparity matching module is configured to perform disparity matching on the pair of matching images by the new matching window to obtain the updated disparity of the object edge pixels in the matching image; A disparity fusion module is configured to perform disparity fusion on the updated disparity and the reference disparity to obtain the final matching disparity of all pixels in the matching image; and A depth calculation module is configured to calculate depth based on the final matching disparity to obtain a corresponding depth image. ​ 8. The object detection apparatus according to claim 7, characterized by The edge detection module comprises a gradient calculation module, a boundary determination module and an angle calculation module which are communicatively connected with each other. The gradient calculation module is configured to calculate the gradient of the pixel value of all pixels in the gray image to calculate the horizontal and vertical gradients of each pixel in the gray image. The boundary determination module is configured to determine the boundary position of the object in the gray image based on the horizontal and vertical gradients of each pixel in the gray image, and mark the pixel value of the object boundary and the predetermined area range where the object boundary is located in the gray image as a marker value. The angle calculation module is configured to calculate the placement angle of the object in the gray image according to the horizontal and vertical gradients of the pixels with the marker value in the gray image, and take the placement angle as the change angle of the object edge in the gray image.

9. The object detection apparatus according to claim 8, characterized by The new window confirmation module comprises a direction adjustment module and a size adjustment module which are communicatively connected with each other. The direction adjustment module is configured to adjust the direction of the new matching window according to the edge change angle of the object in the gray image, so that the direction of the new matching window is adapted to the object in the gray image. The size adjustment module is configured to adjust the size of the new matching window according to the area where the boundary position of the object in the gray image is located, so that a larger new matching window is used in the area closer to the camera, and a smaller new matching window is used in the area farther away from the camera.

10. The object detection apparatus according to any one of claims 7 to 9, characterized by, The object detection device further comprises a search range confirmation module communicatively connected with the conventional disparity matching module and the new disparity matching module. The search range confirmation module is configured to determine the disparity search range of the object edge pixel in the matching image based on the edge position of the object in the gray image, with the reference disparity corresponding to the object edge pixel in the matching image as the center. The new disparity matching module is further configured to perform disparity matching on the pair of matching images in the disparity search range corresponding to the object edge pixel through the new matching window, to obtain the updated disparity of the object edge pixel in the matching image.

11. Obstacle avoidance method, characterized in that, The steps comprise: Based on the depth image obtained by the object detection method as claimed in any one of claims 1 to 6, the contour of the object is constructed to obtain the contour information and depth information of the object; and According to the contour information and depth information of the object, the next motion direction is calculated to avoid the object. The steps comprise:

12. A depth computing device, characterized by A structured light camera module is configured to obtain image data; and A data processing module is communicatively connected to the structured light camera module and configured to perform the steps of the object detection method as claimed in any one of claims 1 to 6 based on the image data from the structured light camera module to perform depth calculation. The steps comprise: A robot body; and 13. A robot vacuum cleaner characterised in that The depth calculation device as claimed in claim 12 is configured to the robot body to provide the robot body with a depth image. The steps comprise: A robot body; and The depth calculation device as claimed in claim 12 is configured to the robot body to provide the robot body with a depth image.

Images