A method for volume measurement of thin packages based on multi-frame depth image probability processing

Abstract

The application discloses a kind of thin parcel volume measurement method based on multi-frame depth image probability processing, belong to parcel volume measurement technical field.The depth information of scene is extracted dynamically by depth camera acquisition;The difference of depth value between adjacent points is used to dynamically update the probability of points in the scene, applied to various specifications of parcels existing in the scene, to ensure that various specifications of parcels can be detected and measured;Since the depth information of very thin parcel and the depth information of surrounding have very small difference, by the processing mode of probability updating of each point by multi-frame depth map, very thin parcel can be effectively extracted and measured;For various types of parcels, the results are accurate, have good noise immunity, can be used for depth camera with large precision fluctuation;For large volume parcel, accurate results can be obtained by processing with few frames, with the advantage of short time.

Description

Technical Field

[0001] This invention relates to the field of package volume measurement technology, and specifically to a method for measuring the volume of thin packages based on multi-frame depth image probability processing. Background Technology

[0002] With the rapid development of various industries, warehousing demands are increasing in terms of efficiency, refined management, and operational accuracy, which traditional labor can no longer fully meet. In particular, the rapid development of e-commerce and consumers' growing expectations for timely logistics have led to higher requirements for throughput and operational efficiency at logistics centers, making the rapid acquisition of volume information for packages of various sizes especially crucial.

[0003] Traditional volume calculation methods involve reconstructing the package in 3D from below the camera, then calculating the package's length, width, and height to derive its volume. This method is inefficient and prone to errors.

[0004] Meanwhile, the large error in depth values ​​obtained by ToF depth cameras makes it difficult to extract, identify, and measure small packages. Therefore, a method for measuring the volume of thin packages based on probabilistic processing of multi-frame depth images is proposed. Summary of the Invention

[0005] The technical problem to be solved by this invention is: how to solve the problem of volume measurement of packages of various specifications in the prior art, especially the problem that the measured package volume is too small to be identified, and provides a method for measuring the volume of thin packages based on multi-frame depth image probability processing.

[0006] The present invention solves the above-mentioned technical problems through the following technical solution, and the present invention includes the following steps:

[0007] S1: Dynamically acquire regular depth maps of the package using a vertically shooting depth camera. Each depth map includes H*W pixels.

[0008] S2: Update the occupancy probability of each pixel in the depth map;

[0009] S3: Traverse all pixels in the depth map, calculate the score of each pixel, determine whether it is an edge pixel, and all edge pixels form an edge region.

[0010] S4: Extract the minimum bounding rectangle for the edge region. The minimum bounding rectangle corresponds to the region that encloses the outer contour.

[0011] S5: If the smallest bounding rectangle is extracted, proceed to steps S6 to S8; otherwise, repeat steps S1 to S4.

[0012] S6: Calculate the depth values ​​of the pixels inside and outside the smallest bounding rectangle region respectively, and the difference between the two is the height of the bounding rectangle.

[0013] S7: Find the corresponding bounding pixel point of the smallest bounding rectangle;

[0014] S8: Perform straight line fitting on the boundary pixels to obtain four boundary lines and calculate four intersection points. Calculate the three-dimensional distance between two adjacent intersection points in the spatial coordinate system to obtain the length and width values ​​of the package. Then, combine the height obtained in step S6 to calculate the volume of the package.

[0015] Furthermore, in step S2, each pixel i is defined as follows:

[0016] Each pixel is divided into two states: occupied or idle;

[0017] Let P(i) represent the probability of occupancy. Indicates the probability of being idle.

[0018] Use P t (i) represents the probability of pixel i occupying the depth map in frame t. This represents the idle probability of pixel i in the depth map of frame t;

[0019] Use H t (i) represents the logarithmic probability of pixel i in the depth map of frame t, which is the ratio of the occupied probability to the idle probability of pixel i in the depth map of frame t, and then the logarithm of the ratio:

[0020]

[0021] Use L t (i) represents the cumulative log probability of pixel i in the previous t frames, i.e.:

[0022] L t (i)=H t (i)+H t-1 (i)+…+H1(i)+H0(i)=H t (i)+L t-1 (i).

[0023] Furthermore, the specific process in step S2 is as follows:

[0024] S21: Calculate the occupancy probability P of the depth map of each pixel i in frame t. t (i), the calculation process is as follows:

[0025] S211: In the depth map of the t-th frame, the depth value of pixel i is subtracted from the depth values ​​of its c*c neighboring pixels, and the average of all the differences is taken as delta(i), where c is a positive integer;

[0026] S212: Set the depth difference threshold to Th, then the depth score of pixel i is delta_h(i) = delta(i) / Th;

[0027] S213: Substitute the depth score into the following probability formula:

[0028]

[0029] That is, to obtain the occupancy probability P of pixel i in the depth map of frame t. t (i) is a probability value between 0 and 1;

[0030] S22: Calculate the cumulative log probability of pixel i in the depth map of each frame. The calculation process is as follows:

[0031] S221: Initially, the occupancy probability and idle probability of a pixel are both 0.5. Then, the cumulative logarithmic probability of pixel i in frame 0 is:

[0032]

[0033] S222: Assume that in the depth map of frame t-1, the cumulative logarithmic probability of pixel i is L. t-1 (i), then in the depth map of frame t, as the occupancy probability P of pixel point i in each frame increases... t (i) Update: The cumulative logarithmic probability of pixel i is:

[0034]

[0035] S23: Determine whether pixel i is a high point, a low point, or a flat point:

[0036] Set the threshold for identifying a high point to a positive number T1. If the cumulative logarithmic probability of pixel i is L... t (i) If it is greater than T1, then pixel i is determined to be a high point; set the threshold for judging it as a low point to a negative number T2, T2 = -T1, if the cumulative log probability of pixel i is L t (i) If it is less than T2, then pixel i is determined to be a low point; otherwise, pixel i is determined to be a flat point; where the high point and the low point are located at the wrapping edge.

[0037] Furthermore, in step S3, for each pixel m(u, v), u is the x-coordinate of pixel m in the depth map, and v is the y-coordinate of pixel m in the depth map. The specific process of step S3 is as follows:

[0038] S31: Take d*d pixels in the neighborhood of pixel m in the depth map, and divide the d*d pixels into two rectangular regions with the horizontal coordinate u as the center line.

[0039] Calculate the absolute value of the difference between the number of high points and low points within the rectangular region:

[0040] ΔNUM up =|NH up -NL up |

[0041] Among them, NH up NL represents the number of high points within the upper rectangular area. up This indicates the number of low points within the upper rectangular area;

[0042] Calculate the absolute value of the difference between the number of high points and low points within the rectangular region:

[0043] ΔNUM down =|NH down -NL down |

[0044] Among them, NH down NL represents the number of high points within the upper rectangular area. down This indicates the number of low points within the upper rectangular area;

[0045] The sum of the two is ΔSUM up_down =(ΔNUM) up +ΔNUM down );

[0046] S32: Take d*d pixels in the neighborhood of pixel m in the depth map, and divide the d*d pixels into two rectangular regions with the vertical coordinate v as the center line.

[0047] Calculate the absolute value of the difference between the number of high points and low points within the left rectangular area:

[0048] ΔNUM left =|NH left -NL left |

[0049] Among them, NH left NL represents the number of high points within the upper rectangular area. left This indicates the number of low points within the upper rectangular area;

[0050] Calculate the absolute value of the difference between the number of high points and low points within the right rectangular region:

[0051] ΔNUM right =|NH right -NL right |

[0052] Among them, NH right NL represents the number of high points within the upper rectangular area. right This indicates the number of low points within the upper rectangular area;

[0053] The sum of the two is ΔSUM left_right =(ΔNUM) left +ΔNUM right );

[0054] S33: Take ΔSUM up_down With ΔSUM left_right The larger value in the range is taken as the score of pixel m;

[0055] When the score is greater than the set threshold T3, pixel m is determined to be an edge point, and the score is assigned to pixel m. All edge points in the depth map form an edge region.

[0056] Furthermore, the specific process in step S6 is as follows:

[0057] S61: Obtain the pixel coordinates of the four vertices P1, P2, P3, and P4 of the minimum bounding rectangle corresponding to the outer contour of the package, with side lengths P1P2 = L1 and P1P4 = L2 respectively. Here, P1 and P3 are two intersection points on a pair of opposite diagonals, and P2 and P4 are two intersection points on another pair of opposite diagonals.

[0058] S62: Take the midpoints M1 and M2 of opposite sides P1P4 and P2P3 respectively, and then take the first and third quartering points N1 and N2 on line segment M1M2 respectively, where M1N1 = M2N2; for each pixel p on line segment N1N2, traverse the pixels within the range e*L2 along both directions of vector P2P3, record the depth values ​​of all traversed pixels, and take the median of these depth values ​​to obtain the depth value inside the smallest bounding rectangle region, which is the distance H from the camera to the wrapped upper surface. up , where e is a positive number;

[0059] S63: For each pixel p on line segment P1P2, traverse the pixels of length f*L2 along the direction of vector P3P2 to obtain the pixel corresponding to edge P1P2, where f is a positive number.

[0060] S64: Following the operation of step S63, traverse line segments P2P3, P3P4, and P4P1 along the directions of vector P2P1, vector P3P2, and vector P1P2 respectively to obtain the pixel points corresponding to line segments P2P3, P3P4, and P4P1 respectively.

[0061] S65: Record the depth values ​​of all pixels during the traversal process in steps S63 and S64, and take the median of these depth values ​​to obtain the depth value outside the smallest bounding rectangle region, which is the distance H between the supporting surface and the camera. down ;

[0062] S66: The distance H between the support surface and the camera. down Subtract the distance H between the top surface of the package and the camera. up That is, the height of the package.

[0063] Furthermore, the specific process of step S7 is as follows:

[0064] S71: Obtain the pixel coordinates of the four vertices P1, P2, P3, and P4 of the minimum bounding rectangle corresponding to the outer contour of the package, with side lengths P1P2 = L1 and P1P4 = L2 respectively. Here, P1 and P3 are two intersection points on a pair of opposite diagonals, and P2 and P4 are two intersection points on another pair of opposite diagonals.

[0065] S72: For each pixel p on line segment P1P2, traverse along vector P2P3 towards the interior, with a traversal length of g*L2, where g is a positive number. During the traversal, find the pixel q corresponding to the first local maximum score, where the score of pixel q satisfies the following condition:

[0066]

[0067] Among them, D pre D represents the score of the previous pixel of the current pixel. cur D represents the score of the current pixel. next T represents the score of the next pixel after the current pixel. b This indicates that a threshold value is set.

[0068] Starting with pixel q, continue traversing along the P2P3 direction to find pixel k corresponding to the local minimum score, where the score of pixel k satisfies the following condition:

[0069]

[0070] Among them, D pre D represents the score of the previous pixel of the current pixel. cur D represents the score of the current pixel. next T represents the score of the next pixel after the current pixel. s This indicates that a threshold value is set.

[0071] When two pixels k1 and k2 that satisfy the local minimum conditions of the above scores are found in sequence, pixel k2 is the boundary pixel.

[0072] If only one pixel k1 is found that satisfies the local minimum condition of the above score, then that pixel is a boundary pixel.

[0073] If no pixel point satisfies the local minimum condition of the above score, then the pixel point p has no corresponding boundary pixel point. All the boundary pixel points found are the boundary pixel points corresponding to line segment P1P2.

[0074] S73: Following the operation of step S72, traverse line segments P2P3, P3P4, and P4P1 along the directions of vector P2P1, vector P3P2, and vector P1P2 respectively, and find the boundary pixel points corresponding to edges P2P3, P3P4, and P4P1.

[0075] Furthermore, the specific process of step S8 is as follows:

[0076] S81: Perform line fitting on the boundary pixels corresponding to each edge to obtain the line equations of the four boundary lines;

[0077] S82: Calculate the intersection of two adjacent straight lines, which are the vertices of the rectangle that encloses the top surface, resulting in four vertices;

[0078] S83: Calculate the two-dimensional distance L between any two adjacent points among the four vertices. 2D And convert it into a three-dimensional distance L 3D This means obtaining the length and width of the package, and then calculating its volume.

[0079] Furthermore, in step S83, the two-dimensional distance is converted into a three-dimensional distance using the following formula:

[0080]

[0081] Where fx is the effective focal length of the camera, and L 3D For three-dimensional distance, L 2D H is a two-dimensional distance. up The distance between the top surface of the package and the camera.

[0082] The present invention has the following advantages over the prior art:

[0083] 1) The thin package volume measurement method based on multi-frame depth image probability processing of the present invention dynamically extracts the depth information of the scene by acquiring the depth information through a depth camera; it dynamically updates the probability of points in the scene by utilizing the difference in depth values ​​between adjacent points, and is applied to the scene where there are packages of various sizes, ensuring that packages of various sizes can be detected and measured.

[0084] 2) The thin package volume measurement method based on multi-frame depth image probability processing of the present invention can effectively extract and measure the thin package because the depth information of the extremely thin package has a very small difference from the depth information of the surrounding area. By updating the probability of each point through multi-frame depth images, the extremely thin package can be effectively extracted.

[0085] 3) The thin package volume measurement method based on multi-frame depth image probability processing of the present invention adopts multi-frame stabilization processing for various types of packages, with accurate results and good noise resistance, and can be used for depth cameras with large accuracy fluctuations.

[0086] 4) The thin package volume measurement method based on multi-frame depth image probability processing of the present invention is applicable to extracting packages with a height of at least 2cm and measuring their volume when the depth image is obtained by using a TOF depth camera, because the depth error is large.

[0087] 5) The thin package volume measurement method based on multi-frame depth image probability processing of the present invention can obtain accurate results with very few frames for packages with large volume, and has the advantage of short processing time. Attached Figure Description

[0088] Figure 1 This is a flowchart illustrating the method for measuring package volume in an embodiment of the present invention;

[0089] Figure 2 This is a schematic diagram of the structure wrapped around the light projection of the vision camera in an embodiment of the present invention;

[0090] Figure 3 This is a diagram showing the results of finding occupied and free points in an embodiment of the present invention;

[0091] Figure 4 This is a result image of finding edge regions in an embodiment of the present invention;

[0092] Figure 5 This is a schematic diagram of finding points inside the package in an embodiment of the present invention;

[0093] Figure 6 This is a schematic diagram of finding external points of the package in an embodiment of the present invention;

[0094] Figure 7 This is a display diagram in which the value of each pixel in this embodiment of the invention is displayed from black to white according to the range of 0 to 255;

[0095] Figure 8 This is a schematic diagram of finding the edge points of the package in an embodiment of the present invention;

[0096] Figure 9This is a pixel score curve diagram of case one in the process of finding the wrapping boundary point in the embodiment of the present invention;

[0097] Figure 10 This is a pixel score curve diagram of case two in the process of finding the wrapping boundary point in the embodiment of the present invention. Detailed Implementation

[0098] The embodiments of the present invention are described in detail below. These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. However, the scope of protection of the present invention is not limited to the following embodiments.

[0099] like Figure 1 As shown, this embodiment provides a technical solution: a method for measuring the volume of a thin package based on multi-frame depth image probability processing, specifically including the following steps:

[0100] like Figure 2 As shown, the figure is a schematic diagram of the structure wrapped around the light projection of the vision camera in this measurement embodiment;

[0101] S1: Dynamically acquire a regular depth map of the package using a vertically shooting depth camera. The depth map consists of H*W pixels.

[0102] The depth images obtained by the depth camera are arranged in a regular row-by-row manner to form a regular depth image, with a resolution of 640*480.

[0103] S2: Update the occupancy probability of each pixel in the depth map;

[0104] Specifically, each pixel i is defined as follows:

[0105] Each pixel is divided into two states: occupied or idle;

[0106] Let P(i) represent the probability of occupancy. This represents the idle probability, because each pixel has only these two states, so

[0107] Use P t (i) represents the probability of pixel i occupying the depth map in frame t. This represents the idle probability of pixel i in the depth map of frame t;

[0108] Use H t (i) represents the logarithmic probability of pixel i in the depth map of frame t, which is the ratio of the occupied probability to the idle probability of pixel i in the depth map of frame t, and then the logarithm of the ratio:

[0109]

[0110] Use L t (i) represents the cumulative log probability of pixel i in the previous t frames, i.e.:

[0111] L t (i)=H t (i)+H t-1 (i)+…+H1(i)+H0(i)=H t (i)+L t-1 (i)

[0112] Calculate the occupancy probability P of the depth map of each pixel i in frame t. t (i), the specific calculation process is as follows:

[0113] In the depth map of frame t, the depth value of pixel i is subtracted from the depth values ​​of its c*c (5*5) neighboring pixels, and the average of all the differences is taken as delta(i).

[0114] If the depth difference threshold is set to Th, then the depth score of a pixel is delta_h(i) = delta(i) / Th;

[0115] Substitute the depth score into the following probability formula:

[0116]

[0117] That is, to obtain the occupancy probability P of pixel i in the depth map of frame t. t (i) is a probability value between 0 and 1. The cumulative log probability of pixel i in the depth map of each frame is calculated as follows:

[0118] Initially, the occupied probability and idle probability of a pixel are both 0.5, meaning the cumulative logarithmic probability of pixel i in frame 0 is:

[0119]

[0120] Assume that in the depth map of frame t-1, the cumulative log probability of pixel i is L. t-1 (i), then in the depth map of frame t, as the occupancy probability P of pixel point i in each frame increases... t (i) Update: The cumulative logarithmic probability of pixel i is:

[0121]

[0122] Determine the significance of pixel i: high point, low point, flat point:

[0123] The threshold for identifying a high point is set to a positive number T1 (0.5 in this embodiment). If the cumulative logarithmic probability L of pixel i is... t(i) If it is greater than T1, then pixel i is determined to be the high point;

[0124] Conversely, the threshold for identifying a low point is set to a negative number T2 (in this embodiment, it is -0.5), T2 = -T1. If the cumulative logarithmic probability L of pixel i is... t (i) If it is less than T2, then pixel i is determined to be a low point;

[0125] Otherwise, pixel i is determined to be a flat point;

[0126] The high and low points are located at the edges of the package.

[0127] like Figure 3 As shown, the white dots represent high points, and the black dots represent low points.

[0128] S3: Traverse all pixels in the depth map, calculate the score of each pixel, and determine whether it is an edge pixel. All edge pixels form an edge region. For each pixel m(u, v), u is the x-coordinate of pixel m in the depth map, and v is the y-coordinate of pixel m in the depth map.

[0129] S31: Take d*d (15*15) pixels in the neighborhood of pixel m in the depth map, and divide the d*d (15*15) pixels into two rectangular regions with the x-coordinate equal to u as the center line;

[0130] Calculate the absolute value of the difference between the number of high points and low points within the rectangular region:

[0131] ΔNUM up =|NH up -NL up |

[0132] Among them, NH up NL represents the number of high points within the upper rectangular area. up This indicates the number of low points within the upper rectangular area;

[0133] Calculate the absolute value of the difference between the number of high points and low points within the rectangular region:

[0134] ΔNUM down =|NH down -NL down |

[0135] Among them, NH down NL represents the number of high points within the upper rectangular area. down This indicates the number of low points within the upper rectangular area;

[0136] The sum of the two is ΔSUM up_down =(ΔNUM) up +ΔNUM down );

[0137] S32: Take d*d (15*15) pixels in the neighborhood of pixel m in the depth map, and divide the d*d (15*15) pixels into two rectangular regions with the vertical coordinate v as the center line;

[0138] Calculate the absolute value of the difference between the number of high points and low points within the left rectangular area:

[0139] ΔNUM left =|NH left -NL left |

[0140] Among them, NH left NL represents the number of high points within the upper rectangular area. left This indicates the number of low points within the upper rectangular area;

[0141] Calculate the absolute value of the difference between the number of high points and low points within the right rectangular region:

[0142] ΔNUM right =|NH right -NL right |

[0143] Among them, NH right NL represents the number of high points within the upper rectangular area. right This indicates the number of low points within the upper rectangular area;

[0144] The sum of the two is ΔSUM left_right =(ΔNUM) left +ΔNUM right );

[0145] S33: Take ΔSUM up_down With ΔSUM left_right The larger value in the range is taken as the score of pixel m;

[0146] When the score is greater than the set threshold T3 (70 in this embodiment), pixel m is determined to be an edge point, and the score is assigned to pixel m. All edge points in the depth map form an edge region.

[0147] like Figure 4 As shown, white dots are edge points, and black dots are non-edge points.

[0148] S4: Extract the minimum bounding rectangle for the edge region, which corresponds to the region that encloses the outer contour.

[0149] Optionally, the findContours function in OpenCV can be used to extract the minimum bounding rectangle of the edge region.

[0150] S5: If the smallest bounding rectangle is extracted, proceed to steps S6 to S8; otherwise, repeat steps S1 to S4.

[0151] S6: Calculate the depth values ​​of the pixels inside and outside the smallest bounding rectangle region respectively, and the difference between the two is the height value of the bounding rectangle.

[0152] Obtain the pixel coordinates of the four vertices P1, P2, P3, and P4 of the minimum bounding rectangle corresponding to the outer contour of the package, with side lengths P1P2 = L1 and P1P4 = L2, where P1 and P3 are two intersection points on one pair of diagonals, and P2 and P4 are two intersection points on another pair of diagonals.

[0153] like Figure 5 As shown, take the midpoints M1 and M2 of opposite sides P1P4 and P2P3 respectively, and then take the first and third four-part points N1 and N2 on line segment M1M2 respectively, where M1N1 = M2N2;

[0154] For each pixel p on line segment N1N2, traverse the pixels within the range e*L2 along both the positive and negative directions of vector P2P3, record the depth values ​​of all traversed pixels, and take the median of these depth values ​​to obtain the depth value inside the smallest bounding rectangle region, which is the distance H from the camera to the bounding surface. up , where e is a positive number, and in this embodiment, it takes the value of 0.3.

[0155] like Figure 6 As shown, for each pixel p on line segment P1P2, the pixels of length f*L2 are traversed along the direction of vector P3P2 to obtain the pixel corresponding to edge P1P2, where f is a positive number and is 0.2 in this embodiment.

[0156] Similarly, obtain the pixels corresponding to the other three sides, record the depth values ​​of all pixels during the traversal, and take the median of these depth values ​​to obtain the depth value outside the smallest bounding rectangle region, which is the distance H between the supporting surface and the camera. down .

[0157] The distance H between the support surface and the camera. down Subtract the distance H between the top surface of the package and the camera. up That is, the height of the package.

[0158] S7: Find the corresponding bounding pixel point of the minimum bounding rectangle;

[0159] like Figure 7As shown, each pixel is displayed with a score ranging from 0 to 255, from black to white. As the process moves from the outside of the package to the inside, the score is lower at the intersection of the package support surface and the side, as well as at the intersection of the package top surface and the side. This corresponds to two darker line segments in the package edge area of ​​the image compared to their neighboring points.

[0160] like Figure 8 As shown, obtain the pixel coordinates of the four vertices P1, P2, P3, and P4 of the minimum bounding rectangle corresponding to the outer contour of the package, with side lengths P1P2 = L1 and P1P4 = L2, where P1 and P3 are two intersection points on a pair of opposite diagonals, and P2 and P4 are two intersection points on another pair of opposite diagonals.

[0161] For each pixel p on line segment P1P2, traverse along vector P2P3 towards the interior, with a traversal length of g*L2, where g is a positive number (0.3 in this embodiment). During the traversal, find the pixel q corresponding to the first local maximum score, where the score of pixel q satisfies the following condition:

[0162]

[0163] Among them, D pre D represents the score of the previous pixel of the current pixel. cur D represents the score of the current pixel. next T represents the score of the next pixel after the current pixel. b This indicates that a threshold value is set, which is 10 in this embodiment.

[0164] Starting with pixel q, continue traversing along the P2P3 direction to find pixel k corresponding to the local minimum score, where the score of pixel k satisfies the following condition:

[0165]

[0166] Among them, D pre D represents the score of the previous pixel of the current pixel. cur D represents the score of the current pixel. next T represents the score of the next pixel after the current pixel. s This indicates that a threshold value is set, which is 10 in this embodiment.

[0167] Scenario 1: For example Figure 8 As shown in section (a), when two pixels k1 and k2 that satisfy the local minimum conditions of the above scores are found in sequence, pixel k2 is the boundary pixel, and the score change of the corresponding pixel is as follows. Figure 9 As shown;

[0168] Scenario 2: For example Figure 8As shown in section (b), if only one pixel k1 is found that satisfies the local minimum condition of the above score, then this pixel is a boundary pixel, and the score of the corresponding pixel is as follows. Figure 10 As shown;

[0169] If no pixel point satisfies the local minimum condition of the above score, then the pixel point p has no corresponding boundary pixel point. All the boundary pixel points found are the boundary pixel points corresponding to line segment P1P2.

[0170] Similarly, traverse line segments P2P3, P3P4, and P4P1 along the directions of vector P2P1, vector P3P2, and vector P1P2 respectively to find the boundary pixels corresponding to edges P2P3, P3P4, and P4P1.

[0171] S8: Perform straight line fitting on the boundary pixels to obtain four boundary lines and calculate four intersection points. Calculate the three-dimensional distance between two adjacent intersection points in the spatial coordinate system to obtain the length and width of the package, and thus calculate the volume of the package.

[0172] By fitting straight lines to the boundary pixels corresponding to each edge, the straight line equations of the four boundary lines are obtained.

[0173] Calculate the intersection of two adjacent straight lines, which are the vertices of the rectangle that encloses the top surface, thus obtaining four vertices;

[0174] Calculate the two-dimensional distance L between any two adjacent points among the four vertices. 2D And convert it into a three-dimensional distance L 3D This yields the length and width of the package, allowing for the calculation of its volume. The two-dimensional distance is converted to three-dimensional distance using the following formula:

[0175]

[0176] Where fx is the effective focal length of the camera, and L 3D For three-dimensional distance, L 2D H is a two-dimensional distance. up The distance between the top surface of the package and the camera.

[0177] In summary, the package volume measurement method described in the above embodiments dynamically extracts the scene's depth information using a depth camera; it dynamically updates the probability of points in the scene by utilizing the differences in depth values ​​between adjacent points, making it applicable to packages of various sizes in the scene and ensuring that packages of all sizes can be detected and measured; since the depth information of extremely thin packages differs very little from the surrounding depth information, the method of updating the probability of each point using multi-frame depth maps can effectively extract and measure extremely thin packages; for various types of packages, the multi-frame stabilization processing method yields accurate results and has good noise resistance, making it suitable for depth cameras with large accuracy fluctuations; for larger packages, accurate results can be obtained with very few frames, offering the advantage of short processing time.

[0178] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for measuring the volume of a thin package based on probabilistic processing of multi-frame depth images, characterized in that, Includes the following steps: S1: Dynamically acquire regular depth maps of the package using a vertically shooting depth camera. Each depth map includes H*W pixels. S2: Update the occupancy probability of each pixel in the depth map; S3: Traverse all pixels in the depth map, calculate the score of each pixel, determine whether it is an edge pixel, and all edge pixels form an edge region. S4: Extract the minimum bounding rectangle for the edge region. The minimum bounding rectangle corresponds to the region that encloses the outer contour. S5: If the smallest bounding rectangle is extracted, proceed to steps S6 to S8; otherwise, repeat steps S1 to S4. S6: Calculate the depth values ​​of the pixels inside and outside the smallest bounding rectangle region respectively, and the difference between the two is the height of the bounding rectangle. S7: Find the corresponding bounding pixel point of the smallest bounding rectangle; S8: Perform straight line fitting on the boundary pixels to obtain four boundary lines and calculate four intersection points. Calculate the three-dimensional distance between two adjacent intersection points in the spatial coordinate system to obtain the length and width values ​​of the package. Then, combine the height obtained in step S6 to calculate the volume of the package. In step S2, each pixel i is defined as follows: Each pixel is divided into two states: occupied or idle; Let P(i) represent the probability of occupancy. Indicates the probability of being idle. Use P t (i) represents the probability of pixel i occupying the depth map in frame t. This represents the idle probability of pixel i in the depth map of frame t; Use H t (i) represents the logarithmic probability of pixel i in the depth map of frame t, which is the ratio of the occupied probability to the idle probability of pixel i in the depth map of frame t, and then the logarithm of the ratio: Use L t (i) represents the cumulative log probability of pixel i in the previous t frames, i.e.: L t (i)=H t (i)+H t-1 (i)+…+H1(i)+H0(i)=H t (i)+L t-1 (and).

2. The method for measuring the volume of a thin package based on multi-frame depth image probability processing according to claim 1, characterized in that: The specific process in step S2 is as follows: S21: Calculate the occupancy probability P of the depth map of each pixel i in frame t. t (i), the calculation process is as follows: S211: In the depth map of the t-th frame, the depth value of pixel i is subtracted from the depth values ​​of its c*c neighboring pixels, and the average of all the differences is taken as delta(i), where c is a positive integer; S212: Set the depth difference threshold to Th, then the depth score of pixel i is delta_h(i) = delta(i) / Th; S213: Substitute the depth score into the following probability formula: That is, to obtain the occupancy probability P of pixel i in the depth map of frame t. t (i) is a probability value between 0 and 1; S22: Calculate the cumulative log probability of pixel i in the depth map of each frame. The calculation process is as follows: S221: Initially, the occupancy probability and idle probability of a pixel are both 0.

5. Then, the cumulative logarithmic probability of pixel i in frame 0 is: S222: Assume that in the depth map of frame t-1, the cumulative logarithmic probability of pixel i is L. t-1 (i), then in the depth map of frame t, as the occupancy probability P of pixel point i in each frame increases... t (i) Update: The cumulative logarithmic probability of pixel i is: S23: Determine whether pixel i is a high point, a low point, or a flat point: Set the threshold for identifying a high point to a positive number T1. If the cumulative logarithmic probability of pixel i is L... t (i) If it is greater than T1, then pixel i is determined to be a high point; set the threshold for judging it as a low point to a negative number T2, T2 = -T1, if the cumulative log probability of pixel i is L t (i) If it is less than T2, then pixel i is determined to be a low point; otherwise, pixel i is determined to be a flat point; where the high point and the low point are located at the wrapping edge.

3. The method for measuring the volume of a thin package based on multi-frame depth image probability processing according to claim 2, characterized in that: In step S3, for each pixel m(u, v), u is the x-coordinate of pixel m in the depth map, and v is the y-coordinate of pixel m in the depth map. The specific process of step S3 is as follows: S31: Take d*d pixels in the neighborhood of pixel m in the depth map, and divide the d*d pixels into two rectangular regions with the horizontal coordinate u as the center line. Calculate the absolute value of the difference between the number of high points and low points within the rectangular region: ΔNUM up =|NH up -NL up | Among them, NH up NL represents the number of high points within the upper rectangular area. up This indicates the number of low points within the upper rectangular area; Calculate the absolute value of the difference between the number of high points and low points within the rectangular region: ΔNUM down =|NH down -NL down | Among them, NH down NL represents the number of high points within the upper rectangular area. down This indicates the number of low points within the upper rectangular area; The sum of the two is ΔSUM up_down =(ΔNUM) up +ΔNUM down ); S32: Take d*d pixels in the neighborhood of pixel m in the depth map, and divide the d*d pixels into two rectangular regions with the vertical coordinate v as the center line. Calculate the absolute value of the difference between the number of high points and low points within the left rectangular area: ΔNUM left =|NH left -NL left | Among them, NH left NL represents the number of high points within the upper rectangular area. left This indicates the number of low points within the upper rectangular area; Calculate the absolute value of the difference between the number of high points and low points within the right rectangular region: ΔNUM right =|NH right -NL right | Among them, NH right NL represents the number of high points within the upper rectangular area. right This indicates the number of low points within the upper rectangular area; The sum of the two is ΔSUM left_right =(ΔNUM) left +ΔNUM right ); S33: Take ΔSUM up_down With ΔSUM left_right The larger value in the range is taken as the score of pixel m; When the score is greater than the set threshold T3, pixel m is determined to be an edge point, and the score is assigned to pixel m. All edge points in the depth map form an edge region.

4. The method for measuring the volume of a thin package based on multi-frame depth image probability processing according to claim 1, characterized in that: The specific process in step S6 is as follows: S61: Obtain the pixel coordinates of the four vertices P1, P2, P3, and P4 of the minimum bounding rectangle corresponding to the outer contour of the package, with side lengths P1P2 = L1 and P1P4 = L2 respectively. Here, P1 and P3 are two intersection points on a pair of opposite diagonals, and P2 and P4 are two intersection points on another pair of opposite diagonals. S62: Take the midpoints M1 and M2 of opposite sides P1P4 and P2P3 respectively, and then take the first and third quartering points N1 and N2 on line segment M1M2 respectively, where M1N1 = M2N2; for each pixel p on line segment N1N2, traverse the pixels within the range e*L2 along both directions of vector P2P3, record the depth values ​​of all traversed pixels, and take the median of these depth values ​​to obtain the depth value inside the smallest bounding rectangle region, which is the distance H from the camera to the wrapped upper surface. up , where e is a positive number; S63: For each pixel p on line segment P1P2, traverse the pixels of length f*L2 along the direction of vector P3P2 to obtain the pixel corresponding to edge P1P2, where f is a positive number. S64: Following the operation of step S63, traverse line segments P2P3, P3P4, and P4P1 along the directions of vector P2P1, vector P3P2, and vector P1P2 respectively to obtain the pixel points corresponding to line segments P2P3, P3P4, and P4P1 respectively. S65: Record the depth values ​​of all pixels during the traversal process in steps S63 and S64, and take the median of these depth values ​​to obtain the depth value outside the smallest bounding rectangle region, which is the distance H between the supporting surface and the camera. down ; S66: The distance H between the support surface and the camera. down Subtract the distance H between the top surface of the package and the camera. up That is, the height of the package.

5. A method for measuring the volume of a thin package based on multi-frame depth image probability processing according to claim 1 or 4, characterized in that: The specific process of step S7 is as follows: S71: Obtain the pixel coordinates of the four vertices P1, P2, P3, and P4 of the minimum bounding rectangle corresponding to the outer contour of the package, with side lengths P1P2 = L1 and P1P4 = L2 respectively. Here, P1 and P3 are two intersection points on a pair of opposite diagonals, and P2 and P4 are two intersection points on another pair of opposite diagonals. S72: For each pixel p on line segment P1P2, traverse along vector P2P3 towards the interior, with a traversal length of g*L2, where g is a positive number. During the traversal, find the pixel q corresponding to the first local maximum score, where the score of pixel q satisfies the following condition: Among them, D pre D represents the score of the previous pixel of the current pixel. cur D represents the score of the current pixel. next T represents the score of the next pixel after the current pixel. b This indicates that a threshold value is set. Starting with pixel q, continue traversing along the P2P3 direction to find pixel k corresponding to the local minimum score, where the score of pixel k satisfies the following condition: Among them, D pre D represents the score of the previous pixel of the current pixel. cur D represents the score of the current pixel. next T represents the score of the next pixel after the current pixel. s This indicates that a threshold value is set. When two pixels k1 and k2 that satisfy the local minimum conditions of the above scores are found in sequence, pixel k2 is the boundary pixel. If only one pixel k1 is found that satisfies the local minimum condition of the above score, then that pixel is a boundary pixel. If no pixel point satisfies the local minimum condition of the above score, then the pixel point p has no corresponding boundary pixel point. All the boundary pixel points found are the boundary pixel points corresponding to line segment P1P2. S73: Following the operation of step S72, traverse line segments P2P3, P3P4, and P4P1 along the directions of vector P2P1, vector P3P2, and vector P1P2 respectively, and find the boundary pixel points corresponding to edges P2P3, P3P4, and P4P1.

6. The method for measuring the volume of a thin package based on multi-frame depth image probability processing according to claim 5, characterized in that: The specific process of step S8 is as follows: S81: Perform line fitting on the boundary pixels corresponding to each edge to obtain the line equations of the four boundary lines; S82: Calculate the intersection of two adjacent straight lines, which are the vertices of the rectangle that encloses the top surface, resulting in four vertices; S83: Calculate the two-dimensional distance L between any two adjacent points among the four vertices. 2D And convert it into a three-dimensional distance L 3D This means obtaining the length and width of the package, and then calculating its volume.

7. The method for measuring the volume of a thin package based on multi-frame depth image probability processing according to claim 6, characterized in that: In step S83, the two-dimensional distance is converted into a three-dimensional distance using the following formula: Where fx is the effective focal length of the camera, and L 3D For three-dimensional distance, L 2D H is a two-dimensional distance. up The distance between the top surface of the package and the camera.

Images