3D point cloud target detection method and system based on monocular structured light

Abstract

The present invention discloses a 3D point cloud target detection method and system based on monocular structured light, the method comprising: obtaining a monocular structured light image, the monocular structured light image comprising a two-dimensional image, a depth image and a three-dimensional point cloud image; detecting the area of ​​the target object in the two-dimensional image according to the shape contour template of the target object; obtaining the projection area of ​​the target object in the depth image according to the area of ​​the target object in the two-dimensional image; performing point cloud data processing on the three-dimensional point cloud image according to the projection area of ​​the target object in the depth image, and obtaining the precise area of ​​the target object in the three-dimensional point cloud image. The technical solution of the present invention can improve the detection speed and accuracy of 3D point cloud targets.

Description

Technical Field

[0001] The present invention relates to the field of image detection, and in particular to a 3D point cloud target detection method and system based on monocular structured light. Background Art

[0002] At present, in order to detect and locate the target object in real time, 3D point cloud target detection in the industry usually adopts a one-stage detection method or a two-stage detection method.

[0003] The first-stage detection method is to directly use 3D point cloud data for detection. The second-stage detection method is to first project the two-dimensional detection results into the 3D point cloud to roughly determine the location of the object, and then obtain more accurate regression information based on the designed point cloud network. Its advantage is high detection accuracy, and its disadvantage is slow speed. At present, the two-stage detection method is mainly used without considering the detection speed.

[0004] In industrial environments, the objects being measured in the basket may have unsatisfactory point cloud data due to factors such as light, angle, material, and occlusion, which affects the accuracy and speed of target detection. Summary of the invention

[0005] The main purpose of the present invention is to provide a 3D point cloud target detection method and system based on monocular structured light, aiming to improve the detection speed and accuracy of 3D point cloud targets.

[0006] In an embodiment of the present invention, a 3D point cloud target detection method based on monocular structured light is proposed, which is characterized by comprising:

[0007] Acquire a monocular structured light image, wherein the monocular structured light image includes a two-dimensional image, a depth image, and a three-dimensional point cloud image;

[0008] Detecting a region of the target object in the two-dimensional image according to a shape contour template of the target object;

[0009] Acquire a projection area of ​​the target object in the depth image according to the area of ​​the target object in the two-dimensional image;

[0010] Point cloud data processing is performed on the three-dimensional point cloud image according to the projection area of ​​the target object in the depth image to obtain the precise area of ​​the target object in the three-dimensional point cloud image.

[0011] In an embodiment of the present invention, the 3D point cloud target detection method based on monocular structured light further includes:

[0012] A detection frame of the target object is displayed in the three-dimensional point cloud image to obtain the position and posture of the target object.

[0013] In the embodiment of the present invention, a minimum bounding box algorithm is used to obtain the position and posture of the target object.

[0014] In an embodiment of the present invention, the 3D point cloud target detection method based on monocular structured light further includes:

[0015] The target object is modeled in advance to obtain a two-dimensional shape contour template of the target object.

[0016] In an embodiment of the present invention, point cloud data processing is performed on the three-dimensional point cloud image, including:

[0017] The point cloud data in the projection area is used to extract the boundary of the target object through the Euclidean clustering algorithm, and the boundary is accurately segmented to obtain the precise area of ​​the target object in the three-dimensional point cloud image.

[0018] In an embodiment of the present invention, a 3D point cloud target detection system based on monocular structured light is also provided, which includes:

[0019] A monocular structured light camera module, used to obtain a monocular structured light image, wherein the monocular structured light image includes a two-dimensional image, a depth image, and a three-dimensional point cloud image;

[0020] A two-dimensional image detection module, used to detect the area of ​​the target object in the two-dimensional image according to the shape contour template of the target object;

[0021] A depth image detection module, used to obtain a projection area of ​​the target object in the depth image according to the area of ​​the target object in the two-dimensional image;

[0022] The point cloud data processing module is used to perform point cloud data processing on the three-dimensional point cloud image according to the projection area of ​​the target object in the depth image, and obtain the precise area of ​​the target object in the three-dimensional point cloud image.

[0023] In an embodiment of the present invention, the 3D point cloud target detection system based on monocular structured light further includes:

[0024] The position and posture detection module is used to display the detection frame of the target object in the three-dimensional point cloud image and obtain the position and posture of the target object.

[0025] In the embodiment of the present invention, the position and posture detection module uses a minimum bounding box algorithm to obtain the position and posture of the target object.

[0026] In an embodiment of the present invention, the 3D point cloud target detection system based on monocular structured light further includes:

[0027] The modeling module is used to model the target object to obtain a two-dimensional shape contour template of the target object.

[0028] In the embodiment of the present invention, the point cloud data processing module performs point cloud data processing on the three-dimensional point cloud image, including:

[0029] The point cloud data in the projection area is used to extract the boundary of the target object through the Euclidean clustering algorithm, and the boundary is accurately segmented to obtain the precise area of ​​the target object in the three-dimensional point cloud image.

[0030] Compared with the prior art, the 3D point cloud target detection method and system based on monocular structured light of the present invention first detects the area of ​​the target object in the two-dimensional image according to the shape contour template of the target object, then obtains the projection area of ​​the target object in the depth image according to the area of ​​the target object in the two-dimensional image, and finally performs point cloud data processing on the three-dimensional point cloud image according to the projection area of ​​the target object in the depth image to obtain the precise area of ​​the target object in the three-dimensional point cloud image, and detects the image data collected by the monocular structured light camera module through a three-stage method, which can accurately obtain the position and posture of the target object, and at the same time can improve the detection speed of the target object, save time for subsequent sorting processes in industry, and improve production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is the process of a 3D point cloud target detection method based on monocular structured light according to an embodiment of the present invention.

[0032] Figure 2 Schematic diagram of the structure of a 3D point cloud target detection system based on monocular structured light according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention is described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the following specific embodiments are only used to explain the present invention and do not constitute a limitation to the present invention.

[0034] like Figure 1 As shown, in an embodiment of the present invention, a 3D point cloud target detection method based on monocular structured light is proposed, which includes steps S1-S5. The steps are described below respectively.

[0035] Step S1: Acquire a monocular structured light image.

[0036] It should be noted that the monocular structured light image includes a two-dimensional image, a depth image and a three-dimensional point cloud image, which are reconstructed after being photographed by a monocular structured light camera device.

[0037] Step S2: Detecting the area of ​​the target object in the two-dimensional image according to the shape contour template of the target object.

[0038] It should be noted that the shape contour template of the target object is obtained by pre-modeling the target object in the monocular structured light two-dimensional image. Since the size and shape of the same target object under the monocular structured light imaging system may vary in an industrial environment, a deformable template algorithm is used in this embodiment to model the target object to obtain the shape contour template of the target object. That is, the target object is manually annotated, and then these annotated images are transferred and trained using the SSD neural network model to obtain a model that can identify the target object.

[0039] Step S3: acquiring a projection area of ​​the target object in the depth image according to the area of ​​the target object in the two-dimensional image.

[0040] It should be noted that the approximate projection area of ​​the target object in the depth image can be obtained according to the regional pixel coordinates of the target object in the two-dimensional image. At this time, the area of ​​the target object obtained is not detailed enough.

[0041] Step S4: performing point cloud data processing on the three-dimensional point cloud image according to the projection area of ​​the target object in the depth image to obtain the precise area of ​​the target object in the three-dimensional point cloud image.

[0042] Specifically, point cloud data processing is performed on the three-dimensional point cloud image, including point cloud clustering and point cloud segmentation, that is, the point cloud data of the area corresponding to the projection area is used to extract the boundary of the target object through the Euclidean clustering algorithm, and it is accurately segmented to obtain the precise area of ​​the target object in the three-dimensional point cloud image.

[0043] Step S5: Displaying a detection frame of the target object in the three-dimensional point cloud image to obtain the position and posture of the target object.

[0044] In the embodiment of the present invention, a minimum bounding box algorithm is used to obtain the position and posture of the target object, that is, the minimum bounding box area of ​​the segmented point cloud area is calculated, and then the length and width, center position and posture of the target object are estimated.

[0045] Through the above steps, the data collected by the monocular structured light is detected through a three-stage detection method, which can accurately obtain the position and posture of the target object. At the same time, the detection speed of the target object can be improved, which saves time for subsequent sorting processes in the industry and improves production efficiency.

[0046] like Figure 2As shown, corresponding to the above-mentioned 3D point cloud target detection method based on monocular structured light, in an embodiment of the present invention, a 3D point cloud target detection system based on monocular structured light is also provided, which includes a monocular structured light camera module 1, a modeling module 2, a two-dimensional image detection module 3, a depth image detection module 4, a point cloud data processing module 5 and a position and posture detection module 6. They are described below respectively.

[0047] The monocular structured light camera module 1 is used to obtain a monocular structured light image, and the monocular structured light image includes a two-dimensional image, a depth image and a three-dimensional point cloud image.

[0048] The modeling module 2 is used to model the target object and obtain a two-dimensional shape contour template of the target object. In this embodiment, the modeling module 2 uses a deformable template algorithm to model the target object and obtain a shape contour template of the target object.

[0049] The two-dimensional image detection module 3 is used to detect the area of ​​the target object in the two-dimensional image according to the shape contour template of the target object.

[0050] The depth image detection module 4 is used to obtain a projection area of ​​the target object in the depth image according to the area of ​​the target object in the two-dimensional image.

[0051] The point cloud data processing module 5 is used to perform point cloud data processing on the three-dimensional point cloud image according to the projection area of ​​the target object in the depth image, and obtain the precise area of ​​the target object in the three-dimensional point cloud image. Specifically, the point cloud data processing module 5 extracts the boundary of the target object from the point cloud data in the projection area through the Euclidean clustering algorithm, and accurately segments it to obtain the precise area of ​​the target object in the three-dimensional point cloud image.

[0052] The position and posture detection module 6 is used to display the detection frame of the target object in the three-dimensional point cloud image and obtain the position and posture of the target object. In the embodiment of the present invention, the position and posture detection module 6 uses a minimum bounding box algorithm to obtain the position and posture of the target object, that is, to calculate the minimum bounding box area of ​​the segmented point cloud area, and then estimate the length and width, center position and posture of the target object.

[0053] In summary, the 3D point cloud target detection method and system based on monocular structured light of the present invention are adopted. First, the area of ​​the target object in the two-dimensional image is detected according to the shape contour template of the target object. Then, the projection area of ​​the target object in the depth image is obtained according to the area of ​​the target object in the two-dimensional image. Finally, the three-dimensional point cloud image is processed according to the projection area of ​​the target object in the depth image to obtain the precise area of ​​the target object in the three-dimensional point cloud image. The image data collected by the monocular structured light camera module is detected by a three-stage method, so that the position and posture of the target object can be accurately obtained. At the same time, the detection speed of the target object can be improved, which saves time for subsequent sorting processes in industry and improves production efficiency.

[0054] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A 3D point cloud target detection method based on monocular structured light, characterized in that: include: Acquire a monocular structured light image, wherein the monocular structured light image includes a two-dimensional image, a depth image, and a three-dimensional point cloud image; Detecting a region of the target object in the two-dimensional image according to a shape contour template of the target object; Acquire a projection area of ​​the target object in the depth image according to the area of ​​the target object in the two-dimensional image; Performing point cloud data processing on the three-dimensional point cloud image according to the projection area of ​​the target object in the depth image to obtain the precise area of ​​the target object in the three-dimensional point cloud image; Point cloud data processing is performed on the three-dimensional point cloud image, including point cloud clustering and point cloud segmentation. The point cloud data of the area corresponding to the projection area is extracted through the Euclidean clustering algorithm to extract the boundary of the target object, which is accurately segmented to obtain the precise area of ​​the target object in the three-dimensional point cloud image.

2. The 3D point cloud target detection method based on monocular structured light according to claim 1, characterized in that: Also includes: A detection frame of the target object is displayed in the three-dimensional point cloud image to obtain the position and posture of the target object.

3. The 3D point cloud target detection method based on monocular structured light according to claim 2, characterized in that: The minimum bounding box algorithm is used to obtain the position and posture of the target object.

4. The 3D point cloud target detection method based on monocular structured light according to claim 1, characterized in that: Also includes: The target object is modeled in advance to obtain a two-dimensional shape contour template of the target object.

5. The 3D point cloud target detection method based on monocular structured light according to claim 1, characterized in that: Performing point cloud data processing on the three-dimensional point cloud image includes: The point cloud data in the projection area is used to extract the boundary of the target object through the Euclidean clustering algorithm, and the boundary is accurately segmented to obtain the precise area of ​​the target object in the three-dimensional point cloud image.

6. A 3D point cloud target detection system based on monocular structured light, characterized in that: include: A monocular structured light camera module, used to obtain a monocular structured light image, wherein the monocular structured light image includes a two-dimensional image, a depth image, and a three-dimensional point cloud image; A two-dimensional image detection module, used to detect the area of ​​the target object in the two-dimensional image according to the shape contour template of the target object; A depth image detection module, used to obtain a projection area of ​​the target object in the depth image according to the area of ​​the target object in the two-dimensional image; A point cloud data processing module, used to perform point cloud data processing on the three-dimensional point cloud image according to the projection area of ​​the target object in the depth image, and obtain the precise area of ​​the target object in the three-dimensional point cloud image; Point cloud data processing is performed on the three-dimensional point cloud image, including point cloud clustering and point cloud segmentation. The point cloud data of the area corresponding to the projection area is extracted through the Euclidean clustering algorithm to extract the boundary of the target object, which is accurately segmented to obtain the precise area of ​​the target object in the three-dimensional point cloud image.

7. The 3D point cloud target detection system based on monocular structured light according to claim 6, characterized in that: Also includes: The position and posture detection module is used to display the detection frame of the target object in the three-dimensional point cloud image and obtain the position and posture of the target object.

8. The 3D point cloud target detection system based on monocular structured light according to claim 7, characterized in that: The position and posture detection module uses a minimum bounding box algorithm to obtain the position and posture of the target object.

9. The 3D point cloud target detection system based on monocular structured light according to claim 6, characterized in that: Also includes: The modeling module is used to model the target object to obtain a two-dimensional shape contour template of the target object.

10. The 3D point cloud target detection system based on monocular structured light according to claim 6, characterized in that: The point cloud data processing module performs point cloud data processing on the three-dimensional point cloud image, including: The point cloud data in the projection area is used to extract the boundary of the target object through the Euclidean clustering algorithm, and the boundary is accurately segmented to obtain the precise area of ​​the target object in the three-dimensional point cloud image.

Images