A cargo drawing automatic generation method based on a 3D structured light depth camera

Abstract

The application discloses a kind of cargo drawing automatic generation method based on 3D structured light depth camera, first using 3D structured light depth camera to photograph measured goods, obtain depth map, then calculate the world coordinates of all depth values contained in depth map pixel point, obtain the depth map after pre-processing, then the depth map after pre-processing is converted into gray scale chart, then the gray scale chart is denoised and contour is extracted, contour chart is obtained, contour chart is collated with depth map, contour point is reversely tracked, corresponding point from depth map is found, so as to calculate the related size of the contour of measured goods in contour chart using the internal parameter matrix of 3D structured light depth camera, finally the related size of the contour of measured goods is marked in contour chart, and engineering drawing icon is drawn, so as to obtain cargo drawing.The application combines the world coordinates calculated from depth map and the related technology of image processing, realizes the collection of depth map and the automatic generation of cargo size drawing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a cargo drawing automatic generation method based on a 3D structured light depth camera. BACKGROUND

[0002] With the rapid development of the logistics industry, especially the rapid development of express logistics in recent years, the logistics flow of express cargo in express company transfer stations is increasing day by day. When the express cargo is classified and arranged, the length, width and height dimensions of the cargo need to be measured, and then the parameters of the cargo are input into the system to achieve the purpose of monitoring the cargo in all aspects.

[0003] At present, there are many ways to draw the drawings of the cargo dimensions. The first way is to obtain the dimension information of the cargo by manual size pulling, and then draw the drawings, which is suitable for the case of less cargo. The shortcomings are large data error, high labor intensity and low operation efficiency. The second way is to use static cargo measurement equipment. The cargo is placed in the measurement area by manual operation, and the dimension information of the cargo is quickly obtained by the size measurement equipment, and then the drawings are drawn. But the shortcoming is that it cannot meet the needs of large-batch large-cargo data acquisition. The third way is to use a forklift to carry the cargo to a designated area, and then use a visual measurement lens to collect the dimension information of the cargo, and then draw the drawings. This way not only has high measurement difficulty, but also has large data error and low efficiency when measuring large cargo information. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a cargo drawing automatic generation method based on a 3D structured light depth camera. The depth map collected by the 3D structured light depth camera is combined with the world coordinates calculated from the depth map and the related technology of image processing to realize the collection of the depth map and the automatic generation of the cargo dimension drawing.

[0005] The technical scheme of the present application is as follows:

[0006] A cargo drawing automatic generation method based on a 3D structured light depth camera, specifically comprising the following steps:

[0007] (1) Depth map preprocessing: first, a 3D structured light depth camera is used to shoot the measured cargo and the image background is denoised to obtain a depth map a1. Then, the world coordinates of all pixel points containing depth values in the depth map a1 are calculated according to the intrinsic matrix of the 3D structured light depth camera. Finally, the ground noise in the world coordinates is removed to obtain a preprocessed depth map a2.

[0008] (2) generating a cargo drawing: first, convert the depth map a2 into a gray scale map b1, then denoise and contour extraction are performed on the gray scale map b1 to obtain a contour map b3, the contour map b3 is compared with the depth map a2, the contour points are reversely tracked to find the corresponding points projected from the depth map a2, so as to calculate the related dimensions of the measured cargo contour in the contour map b4 by using the intrinsic matrix of the 3D structured light depth camera, finally, the related dimensions of the measured cargo contour are marked in the contour map b4, and engineering drawing icons are drawn to obtain a contour marked drawing b5.

[0009] In the step (1), the 3D structured light depth camera captures the measured cargo multiple times at a specified shooting position, thereby obtaining multiple depth maps, and according to the relative distance between the multiple depth map shooting positions, the original depth map a0 is spliced, and according to the horizontal distance between the 3D structured light depth camera and the measured cargo, the background noise before and after the measured cargo in the original depth map a0 is removed, thereby obtaining the depth map a1.

[0010] In the step (1), according to the intrinsic matrix of the 3D structured light depth camera, the world coordinates of all pixel points containing depth values in the depth map a1 are calculated, and the calculation formula of the world coordinates is shown in formula (1).

[0011]

[0012] In formula (1), u and v are pixel coordinates of a pixel point, z is a depth value, f x , s, f y , c x , c y are factory fixed values in the intrinsic matrix of the 3D structured light depth camera, and x, y and z are calculated world coordinates.

[0013] In the step (1), the ground noise with a height in the range of 0-20 mm in the world coordinates is removed to obtain the preprocessed depth map a2.

[0014] In the step (2), the specific way of converting the depth map a2 into a gray scale map b1 is: reading the depth value information in the depth map a2, expanding in the front view direction, and expanding the color value of each pixel point to 0-255, thereby obtaining the gray scale map b1.

[0015] In the step (2), the specific way of denoising the gray scale map b1 is: performing dilation processing on the gray scale map b1 to expand the boundary contour of the cargo, bridge the broken gap, and then using median filtering to denoise the image to obtain the gray scale map b2.

[0016] The gray scale image b2 is subjected to contour extraction using a Sobel operator to obtain a contour image b3, the contour image b3 is subjected to fine contour curve perfection to obtain a contour image b4 with levels, and the contour image b4 is compared with the depth image a2 to perform reverse tracking on the contour points to find corresponding points projected from the depth image a2, so that the related dimensions of the measured cargo contour in the contour image b4 are calculated using an intrinsic matrix of the 3D structured light depth camera.

[0017] The specific way of fine contour curve perfection of the contour image b3 is that the results of the coarse detection in the contour image b3 are analyzed and compared with the gray scale image b1 to adjust the edge points, and the found contours are graded and stored in a tree structure, so that the edges of the internal shape of the measured cargo are also subjected to contour drawing, and finally the contour image b4 with levels is obtained.

[0018] The specific way of the step (2) of marking the related dimensions of the measured cargo contour into the contour image b4 and drawing engineering drawing icons to obtain the contour marked drawing b5 is that the smallest circumscribed rectangle of each contour in the contour image b4 is found to position the marked points, then the engineering drawing icons are drawn on the contour image b4, and the related dimensions of the measured cargo contour are marked at the specified marked points, so that the contour marked drawing b5 is obtained.

[0019] Advantages of the present application:

[0020] The present application uses a 3D structured light depth camera to collect a depth image, performs image processing on the depth image, so as to quickly collect the dimension information of the measured cargo contour, mark the related dimensions of the measured cargo contour into a contour image, draw engineering drawing icons, and automatically generate cargo drawings. Compared with the traditional method, the present application has the advantages of low cost, high efficiency and high precision, can greatly shorten the time required for traditional drawing, and provides a reference for subsequent cargo allocation, warehouse planning, transportation scheduling and other processes, and realizes fine management. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 is a depth image a1 before ground noise removal in the embodiment of the present application.

[0022] Figure 2 is a depth image a2 before ground noise removal in the embodiment of the present application.

[0023] Figure 3 is a contour image b4 with levels in the embodiment of the present application.

[0024] Figure 4 is a table of marked values and three measurement values collected in the specific embodiment of the present application. DETAILED DESCRIPTION

[0025] Clearly, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present application.

[0026] A cargo drawing automatic generation method based on a 3D structured light depth camera, characterized in that it specifically comprises the following steps:

[0027] (1), depth map preprocessing:

[0028] S1, first place the 3D structured light depth camera on the support frame and connect it with the computer;

[0029] S2, the 3D structured light depth camera repeatedly shoots the measured cargo at the specified shooting position, thereby obtaining multiple depth maps, and according to the relative distance between the multiple depth map shooting positions, splicing to form an original depth map a0;

[0030] S3, according to the horizontal distance between the 3D structured light depth camera and the measured cargo, remove the background noise before and after the measured cargo in the original depth map a0, thereby obtaining a depth map a1, at this time the depth map a1 still contains the ground background noise in the same distance range as the measured cargo (see Figure 1 );

[0031] S4, according to the intrinsic matrix of the 3D structured light depth camera, calculate the world coordinates of all pixel points containing depth values in the depth map a1, the calculation formula of the world coordinates is shown in formula (1):

[0032]

[0033] In formula (1), u and v are pixel coordinates of the pixel point, z is a depth value, f x , s, f y , c x , c y are factory fixed values in the intrinsic matrix of the 3D structured light depth camera, x, y, z are the calculated world coordinates;

[0034] S5, remove the ground noise with a height in the range of 0-20mm in the world coordinates to obtain a preprocessed depth map a2 (see Figure 2 );

[0035] (2), generate cargo drawing:

[0036] S6, convert the depth map a2 into a gray scale map b1, that is, first read the depth value information in the depth map a2, then expand in the orthographic direction, and finally The color value of each pixel is expanded to 0-255, thereby obtaining a gray scale image b1;

[0037] S7, the gray scale image b1 is dilated, i.e. using a 3*3 structure element, scanning each pixel, using the structure element and the binary image covered thereby to perform an "and" operation, if all are 0, the pixel of the structure image is 0, otherwise 1, so that the boundary profile of the goods is expanded, and the gaps of the broken bridge are bridged; then using median filtering to remove noise, especially the salt and pepper noise which has the greatest impact on the profile formation, i.e. selecting the pixel values of the pixel points and the surrounding adjacent pixel points (the filter size is set to 5*5) in the digital image or digital sequence, sorting these pixel values, and then taking the pixel value located in the middle position as the pixel value of the current pixel point, so that the surrounding pixel values approach the true value, thereby eliminating isolated noise points and enhancing display processing, to obtain a gray scale image b2;

[0038] S8, the gray scale image b2 is subjected to profile extraction using a Sobel operator, to obtain a profile image b3;

[0039] The Sobel operator is a discrete difference operator used to operate on the approximate value of the gray scale of the image brightness function; the operator contains two groups of 3x3 matrices, which are horizontal and vertical, and plane convolution is performed on the image using the two groups of matrices, so that the horizontal and vertical brightness difference approximations are obtained; if A represents the original image, G x and G y respectively represent the image gray scale values subjected to horizontal and vertical edge detection, and their formulas are as follows:

[0040]

[0041] The horizontal and vertical gray scale values of each pixel of the image are combined by the following formula to calculate the size of the point gray scale G, and the formula is as follows:

[0042] S9, the profile image b3 is reprocessed, the gray scale image b1 is taken as the axis according to the threshold value 150, and those less than 150 are set to 0, and those greater than 150 are set to 255, so that the boundary points can be more intuitively found by the difference changes of the pixel points, the results of the rough detection are analyzed in the fine improvement of the profile curve, and the edge points are adjusted by comparison with the gray scale image b1, and the found profile is graded and stored according to the tree structure, so that the edge of the internal shape of the measured goods is also profiled, and finally a profile image b4 with levels is obtained; the profile image b4 with levels is as shown in Figure 3 , the numbers outside the brackets represent the profile names, and the numbers inside the brackets represent the hierarchical order, Figure 3 the profile 0(0) is outside the largest square profile in the figure, and 1(1) is inside the largest square profile;

[0043] S10, the contour map b4 is compared with the depth map a2, the contour points are reversely tracked, the corresponding points projected from the depth map a2 are searched, and the related size of the measured cargo contour in the contour map b4 is calculated by using the internal parameter matrix of the 3D structured light depth camera;

[0044] S11, the minimum positive circumscribed rectangle of each contour in the contour map b4 is searched to locate the labeling point, then the engineering drawing icon is drawn on the contour map b4, and the related size of the measured cargo contour is labeled at the specified labeling point, so that the contour labeling drawing b5 is obtained; specifically, the geometric moment of the image is used to calculate the area of the contour enclosed graph, and by adjusting the related parameters, the contour that is too large or too small is discarded and no longer labeled, wherein the zero-order moment of the image is used to calculate the area of the contour enclosed graph, and the calculation formula is shown in formula (2):

[0045]

[0046] In formula (2), M 00 is the zero-order moment of the image, I represents the contour curve track, and u and v are pixel coordinates of the pixel point; inappropriate threshold selection will lead to meaningless and complicated drawing; for each qualified contour in the selection range, the minimum positive circumscribed rectangle is calculated respectively. max min max min That is, the four vertices of the rectangle are (u max ,v max )(u min ,v max )(u max ,v min )(u min ,v min ), the midpoint coordinates can be calculated after the vertex coordinates are obtained, and the related size of the measured cargo contour in step S10 can be labeled in the contour map b4, and the engineering drawing icon is drawn to obtain the contour labeling drawing b5. Specific embodiments

[0048] The 3D structured light depth camera fixed on the support frame is used to collect the depth map, and the depth maps are spliced according to the relative distance between different shooting positions; according to the above steps, the contour map b4 is compared with the depth map a2, the contour points are reversely tracked, the corresponding points projected from the depth map a2 are searched, and the related size of the measured cargo contour in the contour map b4 is calculated by using the internal parameter matrix of the 3D structured light depth camera, and the labeling value and the three measurement values collected during the test are shown in Figure 4 .

[0049] ​​​While embodiments of the application have been shown and described, it is to be understood that the embodiments described are merely exemplary of the principles and application of the present application. Numerous modifications and adaptions can be effected without departing from the spirit and scope of the present application, which is not limited to the exact construction and arrangement described. It is intended, therefore, to cover all modifications and adaptions that fall within the scope of the claims and their equivalents.

Claims

1. A 3D structured light depth camera based automatic generation method of cargo drawing, characterized in that: Specifically comprising the following steps: (1) Depth map preprocessing: first, a 3D structured light depth camera is used to shoot the measured goods and the image background is denoised to obtain a depth map a1, then the world coordinates of all pixel points containing depth values in the depth map a1 are calculated according to the intrinsic matrix of the 3D structured light depth camera, and finally the ground noise in the world coordinates is removed to obtain a preprocessed depth map a2; (2) Generating a goods drawing: first, the depth map a2 is converted into a gray scale map b1, then the gray scale map b1 is denoised and the contour is extracted to obtain a contour map b3, the results of rough detection in the contour map b3 are analyzed and compared with the gray scale map b1, the edge points are adjusted, and the found contour is graded and stored in a tree structure, so that the edges of the internal shape of the measured goods are also contoured, and finally a contour map b4 with levels is obtained, the contour map b4 is compared with the depth map a2, the contour points are traced in reverse, the corresponding points projected from the depth map a2 are found, and the related dimensions of the measured goods contour in the contour map b4 are calculated by using the intrinsic matrix of the 3D structured light depth camera, and finally the smallest circumscribed rectangle of each contour in the contour map b4 is found to locate the marking points, then engineering drawing icons are drawn on the contour map b4, and the related dimensions of the measured goods contour are marked at the specified marking points, so as to obtain a contour marked drawing b5.

2. The method for automatically generating cargo drawings based on a 3D structured light depth camera according to claim 1, characterized in that: In the step (1), the 3D structured light depth camera shoots the measured goods multiple times at a specified shooting position to obtain multiple depth maps, and according to the relative distance between the multiple depth map shooting positions, the original depth map a0 is spliced to form, and then according to the horizontal distance between the 3D structured light depth camera and the measured goods, the background noise before or after the measured goods in the original depth map a0 is removed to obtain the depth map a1. 3.The method of claim 1, wherein: In the step (1), the world coordinates of all pixel points containing depth values in the depth map a1 are calculated according to the intrinsic matrix of the 3D structured light depth camera, and the calculation formula of the world coordinates is shown in formula (1): In formula (1), u, v are pixel coordinates of a pixel point, z is a depth value, f x , s, f y , c x , c y are all factory fixed values in a 3D structured light depth camera intrinsic matrix, and x, y, z are calculated world coordinates.

4. The method of claim 1, wherein the method further comprises: In the step (1), the ground noise in the world coordinates with a height in the range of 0-20mm is removed to obtain the preprocessed depth map a2.

5. The method of claim 1, wherein the method further comprises: In the step (2), the specific way of converting the depth map a2 into a gray scale map b1 is: reading the depth value information in the depth map a2, expanding in the front view direction, and expanding the color value of each pixel point to 0-255, so as to obtain the gray scale map b1.

6. The method of claim 1, wherein the method further comprises: In the step (2), the specific way of denoising the gray scale map b1 is: performing inflation processing on the gray scale map b1 to expand the boundary contour of the goods and bridge the broken gap, and then using median filtering to denoise the image to obtain a gray scale map b2.

7. The method of claim 6, wherein the method further comprises: The gray-scale image b2 is subjected to contour extraction using a Sobel operator to obtain a contour image b3, the contour image b3 is subjected to fine perfection of a contour curve to obtain a contour image b4 with levels, the contour image b4 is compared with the depth image a2, contour points are reversely tracked, corresponding points projected from the depth image a2 are searched, and thus the related dimensions of the measured cargo contour in the contour image b4 are calculated by using an intrinsic matrix of the 3D structured light depth camera.

Images