A method for measuring the volume of multiple packages based on vertical shooting of a depth camera
Through vertical shooting of the depth camera, the difference in the edge area and depth value of the package is obtained, and the package volume is calculated, which solves the problems of large or high volume of existing equipment and low contact measurement efficiency, and achieves fast and accurate multi-package volume measurement, which is suitable for complex scenarios , with high precision and anti-interference performance.
Patent Information
- Application Number
- CN202210647611.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-08
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-06-08
AI Technical Summary
Existing non-contact measurement equipment is large in size or expensive, and has low contact measurement efficiency, making it difficult to quickly and accurately obtain the package volume, especially in daily logistics scenarios.
The vertical shooting method based on the depth camera is adopted to obtain the edge area of the package through the depth map, extract the minimum external rectangle, calculate the length, width and height of the package, and calculate the volume using the difference in depth values. It is suitable for multiple parallel or stacked packages without calibration.
It realizes rapid, accurate and convenient measurement of multiple package volumes, reduces equipment costs, adapts to complex scenarios, has strong anti-interference, high measurement accuracy, stable and reliable, and good real-time performance.
Smart Images

Figure CN115077379B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of package volume measurement, and in particular to a method for measuring the volumes of multiple packages based on vertical photography by a depth camera. Background Art
[0002] Whether in industrial production or daily life, people often need to obtain the size and volume information of cartons, and there are many measurement methods available, from the original direct contact measurement of rulers to non-contact measurement using sound waves, electromagnetic waves, etc. Various measurement technologies are constantly developing and emerging. However, the high cost, large size and poor convenience of various measurement tools limit the widespread application of many high-efficiency non-contact measurement equipment.
[0003] In the robot packaging task, it packs customer orders of any shape and weight into a single shipping box. The robot reads the container and target object dimensions in real time and optimizes the executable packaging plan, which can save a lot of costs for large warehouses.
[0004] In the management of logistics warehouses, the volume measurement system collects and uploads size and volume information, enabling the system to select the best packaging method, more accurately estimate delivery costs, accurately classify picked items, improve order execution accuracy, and improve automated warehouse management.
[0005] Therefore, whether it is a single package or multiple packages arranged side by side or stacked together, how to accurately and conveniently obtain its volume has important research value.
[0006] In the logistics industry, tens of millions of logistics parcels need to be collected, sent, and transferred every day. People need to quickly obtain the size and volume of the parcels to calculate transportation costs, optimize storage space, etc. However, most non-contact measurement equipment needs to be used in conjunction with professional platforms such as conveyor belts and shipping shelves. For example, sensors such as linear lasers and infrared light curtains are used to calculate the volume of the parcel to be measured by accumulating the two-dimensional cross-sectional area on a uniform conveyor belt; the parcel is fixed on a specific shelf, and a camera at a fixed height above the shelf is used to take a picture, and the parcel volume is calculated based on the proportion of parcel pixels in the image. These devices are either bulky or expensive, and are often only placed in distribution and transfer centers and warehouses. Logistics personnel who collect parcels in the streets and alleys can only use tools such as tape measures to perform contact measurements on the parcels, which is inefficient. To this end, a method for measuring the volume of multiple parcels based on vertical shooting with a depth camera is proposed. Summary of the invention
[0007] The technical problem to be solved by the present invention is: how to solve the problem of bulky or high cost of non-contact measurement equipment in the prior art and the problem of low efficiency of contact measurement, and provide a method for measuring the volume of multiple packages based on vertical shooting with a depth camera.
[0008] The present invention solves the above technical problems through the following technical solutions, and the present invention comprises the following steps:
[0009] S1: Use a vertically-shot depth camera to obtain a depth map of the scene containing the package;
[0010] S2: According to the depth values of all pixels in the depth map, the depth values of each pixel and its neighboring pixels are compared to obtain the pixels whose depth value changes reach the set threshold, and such pixels are assigned as edge points. All edge points constitute the wrapped edge area;
[0011] S3: extract the minimum circumscribed rectangle of all inner contours in the edge area of the package, and then process the minimum circumscribed rectangle corresponding to each package in order of the distance from the upper surface of the package to the depth camera from small to large;
[0012] S4: searching for boundary pixel points corresponding to the upper surface of the package for the current minimum circumscribed rectangular area;
[0013] S5: Perform straight line fitting on the boundary pixel points to obtain four boundary lines and calculate the four intersection points of the four boundary lines, respectively calculate the distances between adjacent intersection points, and obtain the length and width of the package;
[0014] S6: respectively calculating the depth values of the inner and outer points of the current minimum circumscribed rectangular area, performing difference processing to obtain the height value of the package, and thus calculating the volume of the package;
[0015] S7: Determine whether there is any unprocessed minimum bounding rectangle. If so, repeat S4 to S6. If not, the volume measurement of all packages in the scene is completed.
[0016] Furthermore, in step S1, the resolution of the depth map is H*W, where H and W are both positive integers.
[0017] Furthermore, the specific process in step S2 is as follows:
[0018] S21: Based on the principle that the depth values of the upper surface of the package and the supporting surface of the package are different, all pixel points are traversed, and the average value of the depth value difference between each pixel point and the pixel points in the set (including but not limited to 5*5) neighborhood is calculated, and the pixel points whose average value of the difference is greater than the set threshold a are obtained. All such pixel points constitute the change area points;
[0019] S22: Traverse the changed area points again, calculate the number of changed area points in the set neighborhood of each pixel point, obtain the pixel points whose number is greater than the set threshold b as the wrapped edge points, and all the wrapped edge points constitute the wrapped edge area.
[0020] Furthermore, the specific process in step S3 is as follows:
[0021] S31: extracting the minimum bounding rectangles of all inner contours of the wrapped edge area, removing the minimum bounding rectangles whose side lengths are less than c pixels due to noise points, and the total number of all minimum bounding rectangles is recorded as M;
[0022] S32: measuring the volume of each package in order from the smallest to the largest distance from the upper surface of the package to the depth camera;
[0023] S33: Calculate the median of the depth values of all pixels on the contour corresponding to each minimum bounding rectangle, obtain the median array of the contours corresponding to all minimum bounding rectangles, and then process all the extracted minimum bounding rectangles in order from small to large in the median array according to the principle that the upper surface of the package is closest to the depth camera.
[0024] Furthermore, the specific process in step S4 is as follows:
[0025] S41: Obtain pixel coordinates P1, P2, P3, and P4 of four vertices of the minimum circumscribed rectangle corresponding to the contour in the wrapped edge area, with side lengths of P1P2=L1 and P1P4=L2, respectively, where P1 and P3 are two intersection points on a pair of diagonals, and P2 and P4 are two intersection points on another pair of diagonals;
[0026] S42: for each pixel point p on the line segment P1P2, search along the direction of vector P3P2 until the first non-edge area point is reached and stop, which is recorded as point q. The traversal length is recorded as L. Then search along the direction of vector P2P3 within the length range of L for the boundary pixel point.
[0027] S43: In the process of searching along the direction of vector P2P3, if the depth difference between the traversed pixel point and the previous pixel point is greater than t, then this pixel point is the boundary pixel point corresponding to pixel point p, that is, the boundary pixel point set corresponding to edge P1P2 is obtained;
[0028] S44: For line segment P1P4, line segment P2P3, and line segment P3P4, adopt steps S42 and S43 to obtain the boundary pixel point sets corresponding to edge P1P4, edge P2P3, and edge P3P4.
[0029] Furthermore, the specific process in step S5 is as follows:
[0030] S51: performing straight line fitting on the boundary pixel point set corresponding to each edge to obtain straight line equations of four boundary lines;
[0031] S52: Taking the intersection of two adjacent straight lines as the vertices of the rectangle wrapping the upper surface, that is, obtaining four vertices;
[0032] S53: Obtain the length and width of the package according to the three-dimensional distances between adjacent vertices.
[0033] Furthermore, in step S53, the two-dimensional distance value between two adjacent pixel points in the four vertices in the plane is calculated and converted into a three-dimensional distance value in space, that is, the length and width of the package are obtained, thereby calculating the volume of the package; wherein the two-dimensional distance value is converted into a three-dimensional distance value by the following formula:
[0034]
[0035] Among them, L 3D is the three-dimensional distance value, L 2D is the two-dimensional distance value, fx is the effective focal length of the depth camera, and H is the actual distance value from the package surface to the depth camera.
[0036] Furthermore, the specific process in step S5 is as follows:
[0037] S61: Obtain pixel coordinates P1, P2, P3, and P4 of four vertices of the minimum circumscribed rectangle corresponding to the contour in the wrapped edge area, with side lengths of P1P2=L1 and P1P4=L2, respectively, where P1 and P3 are two intersection points on a pair of diagonals, and P2 and P4 are two intersection points on another pair of diagonals;
[0038] S62: Take the midpoints M1 and M2 of the opposite sides P1P4 and P2P3 respectively, and for each pixel point p on the line segment M1M2, traverse the pixel points in the range of f*L2 along the positive and negative directions of the vector P2P3 (f is generally 0.3-0.5, which corresponds to the inner area of the upper surface of the package), stop traversing when the edge area of the package is traversed, record the depth values of all the traversed pixel points, take the median of the depth values of all the above pixel points, and obtain the depth value inside the minimum circumscribed rectangular area, which is the actual distance value from the upper surface of the package to the depth camera;
[0039] S63: For each pixel point p on the line segment P1P2, traverse the length g*L2 (g is generally 0.2-0.4, i.e., the area outside the upper surface of the package) along the direction of the vector P3P2. Stop traversing when encountering the minimum circumscribed rectangle corresponding to other packages and the edge area of the package. Record the depth values of the pixels in all non-edge areas. Similarly, obtain the depth values of the pixels in all non-edge areas corresponding to the other three sides. The depth values of the pixels in all non-edge areas form an array Qh.
[0040] S64: Perform k-means clustering processing on the obtained array Qh to obtain the actual distance value from the package support surface to the depth camera and the real distance value from the package support surface to the depth camera;
[0041] S65: Subtract the actual distance value from the upper surface of the package to the depth camera from the actual distance value from the supporting surface of the package to the depth camera and take the absolute value. The calculation result is the height of the package. The volume of the package is calculated in combination with the length and width of the package.
[0042] Furthermore, in step S64, the number of clusters is set to two. When the difference between the center values of the two clusters is less than the threshold value Tk, it indicates that the entire area of the measured package falls on its support surface. The median of the depth values of all pixels in the array Qh is taken as the actual distance value from the package support surface to the depth camera.
[0043] When the difference between the center values of the two clusters is greater than the threshold Tk, it indicates that part of the measured package area falls on its support surface, and the other part falls on the lower plane. Since the distance from the package support surface to the depth camera is smaller than the distance from the lower plane to the depth camera at this time, the class with smaller k-means cluster center value corresponds to the points on the package support surface, and the class with larger k-means cluster center value corresponds to the points on the lower plane. Therefore, the median of all depth values of the smaller k-means cluster is taken as the actual distance value from the package support surface to the depth camera.
[0044] Compared with the prior art, the present invention has the following advantages:
[0045] (1) The method for measuring the volume of multiple packages based on vertical photography by a depth camera of the present invention extracts all packages to be measured through the depth information obtained by the depth camera, and then measures the volume of the packages without calibration;
[0046] (2) The method for measuring the volume of multiple packages based on vertical photography by a depth camera of the present invention utilizes the difference in depth values between the upper surface of the package and the supporting surface of the package in the depth map to extract the edge area of the package; the minimum circumscribed rectangle of the edge area of the package is extracted, which can be applied to a scene where there are multiple packages to be measured at the same time, especially to the scene where there are multiple parallel or stacked packages, and ensures that the packages do not affect each other, and has good anti-interference performance;
[0047] (3) The method for measuring the volume of multiple packages based on vertical photography by a depth camera of the present invention sequentially calculates all packages in the measurement scene by sorting the distance from the upper surface of the package to the camera from small to large, so that it can adapt to complex scenes and ensure that all packages are detected and measured;
[0048] (4) The method for measuring the volume of multiple packages based on vertical photography of a depth camera of the present invention obtains the boundary points of each package and fits straight lines, and calculates the distances of adjacent intersections of four straight lines respectively to obtain the length and width of the package; by respectively calculating the distance from the upper surface of each package to the depth camera and the distance from the support surface of the package to the depth camera, the height of the package is obtained by subtracting the two, thereby obtaining the volume result, with high measurement accuracy, stability and reliability;
[0049] (5) The method for measuring the volume of multiple packages based on vertical photography by a depth camera of the present invention processes the two-dimensional depth data extracted by the depth camera throughout the entire step process, and there is no algorithm with high time complexity. Therefore, the volume measurement of multiple packages at the same time is fast, time-saving, and real-time performance is good. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1 is a schematic flow chart of the steps of a method for measuring the volumes of multiple packages in Embodiment 1 of the present invention;
[0051] Figure 2 is a schematic diagram of the installation position structure of the depth camera in the first embodiment of the present invention;
[0052] Figure 3 This is a schematic diagram of finding the minimum circumscribed rectangle corresponding to the wrapped edge area and the inner contour in the first embodiment of the present invention;
[0053] Figure 4 This is a schematic diagram of finding the boundary points on the upper surface of a package in the first embodiment of the present invention;
[0054] Figure 5 is a schematic diagram of finding a surface point on a package in the first embodiment of the present invention;
[0055] Figure 6 is a schematic diagram of finding a wrapping support point in the first embodiment of the present invention;
[0056] Figure 7 It is a schematic diagram of the structure of multiple packages projected by visual camera light in the second embodiment of the present invention;
[0057] Figure 8 This is a diagram of measurement results of multiple packages in Example 2 of the present invention after being processed by the measurement method in Example 1.
[0058] In the figure: 1, depth camera; 2, package A; 3, package B; 4, package C; 21, upper surface; 22, side; 23, support surface; 101, outer contour of the package edge area; 102, package edge area; 103, real boundary of the package; 104, contour within the package edge area; 105, minimum circumscribed rectangle of the contour within the package edge area. DETAILED DESCRIPTION
[0059] The following is a detailed description of an embodiment of the present invention. This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and a specific operation process are given, but the protection scope of the present invention is not limited to the following embodiment.
[0060] Embodiment 1
[0061] like Figure 1 As shown, this embodiment provides a technical solution: a method for measuring the volume of multiple packages based on vertical photography by a depth camera. The packages referred to in this embodiment are commonly used rectangular packages. The method for measuring the volume of multiple packages specifically includes the following steps:
[0062] S1: Use a vertically-shot depth camera to obtain depth map information of the scene containing the package;
[0063] Among them, Figure 2 As shown, the depth camera 1 is set directly above the packages A2, B3, and C4, and its shooting direction is perpendicular to the packages A2, B3, and C4. The package A2 is stacked directly above the package B3, and the packages B3 and C4 are placed on the same supporting surface;
[0064] In this embodiment, the data processing control module in the depth camera 1 controls the opening and closing of the depth camera to obtain a depth image.
[0065] It should be noted that the depth image obtained by the depth camera 1 is regularly arranged row by row to form a regular depth image, and the resolution is selected to be 640*480.
[0066] S2: According to the depth values of all pixels in the depth map, the depth values of each pixel and its neighboring pixels are compared to obtain the pixels whose depth value changes reach the set threshold, and such pixels are assigned as edge points. All edge points constitute the wrapped edge area;
[0067] Among them, since the depth camera cannot capture the side 22 of the package, the depth value difference of the pixel points within the side 22 of the package is relatively large. Since the depth values of the upper surface 21 and the supporting surface 23 of the package captured by the depth camera from the front are different, the depth value difference of the area where the two planes are connected is relatively large.
[0068] Based on the principle that the depth values of the upper surface of the package and the supporting surface of the package are different; traverse all pixel points, calculate the average value of the depth value difference between each pixel point and the pixels in its 5*5 neighborhood, and the pixel points whose average value of the difference is greater than the set threshold 15 (this threshold is selected according to the minimum height of the measured carton) constitute the change area points.
[0069] The changed area points are traversed again, and the number of changed area points in the 5*5 neighborhood of each pixel is calculated. The points whose number is greater than the set threshold of 16 (the larger the threshold value, the more obvious the noise removal is, but it is necessary to ensure that the points in the wrapped edge area are sufficient to process the result, generally 0.6 to 0.8 times the total number of pixels in the neighborhood) are wrapped edge points to achieve the purpose of removing noise points. All wrapped edge points constitute the wrapped edge area.
[0070] S3: extracting the minimum circumscribed rectangle of all inner contours in the edge area of the package, and then processing the corresponding minimum circumscribed rectangle of each package in turn in the order from near to far of the upper surface of the package;
[0071] Optionally, use the findContours function in OpenCV to extract all minimum bounding rectangles of the wrapped edge area in the depth map. The findContours function uses the RETR_CCOMP method, and the fourth parameter of the hierarchy selects the contour boundary point with a non-1 value, that is, removes the minimum bounding rectangle of the outer contour of all the wrapped edge areas, and retains the minimum bounding rectangle of the contours in all the wrapped edge areas;
[0072] Each side length of the minimum enclosing rectangle must be greater than 20 pixels (the side length of the package generally needs to be a certain length, and those less than 20 pixels are considered to be rectangles caused by noise and need to be removed), so as to remove the minimum enclosing rectangle that is too small;
[0073] Calculate the median of the depth values of all pixels on the contour corresponding to each minimum bounding rectangle, obtain the median array of the contours corresponding to all minimum bounding rectangles, and then process all the extracted minimum bounding rectangles in order from small to large in the median array based on the principle that the upper surface of the package is closest to the camera.
[0074] S4: searching for boundary pixels of the upper surface of the current minimum circumscribed rectangular area;
[0075] Specifically, Figure 4 As shown, first obtain the pixel coordinates P1, P2, P3, and P4 of the four vertices of the minimum circumscribed rectangle corresponding to the wrapped upper surface, with side lengths P1P2=L1 and P1P4=L2 respectively; for each pixel point p on the line segment P1P2, along the direction of vector P3P2, find the first non-edge area point and stop, record it as point q, and record the traversal length as L, and then along the direction of vector P2P3, search for boundary pixel points within the length range of L; in the process of searching along the direction of vector P2P3, if the depth value difference between the traversed pixel point and the previous pixel point is greater than 15, then this pixel point is the boundary pixel point corresponding to pixel point p, and thus the boundary pixel point set corresponding to edge P1P2 is obtained.
[0076] Similarly, the boundary pixel point sets corresponding to the edges P1P4, P2P3, and P3P4 can also be obtained.
[0077] S5: Perform straight line fitting on the boundary pixel points to obtain four boundary lines and calculate the four intersection points of the four boundary lines, respectively calculate the distances between adjacent intersection points, and obtain the length and width of the package;
[0078] Specifically, a straight line fitting is performed on the set of boundary pixel points corresponding to each edge to obtain the straight line equations of the four boundary lines;
[0079] Take the intersection of two adjacent straight lines as the vertices of the rectangle that wraps the upper surface, so that four vertices are obtained;
[0080] The length and width of the package are obtained based on the three-dimensional distance between adjacent vertices, as follows:
[0081] Calculate the two-dimensional distance value between two adjacent pixels in the four vertices in the plane and convert it into a three-dimensional distance value in space, that is, get the length and width of the package, and then calculate the volume of the package. The two-dimensional distance value is converted into a three-dimensional distance value using the following formula:
[0082]
[0083] Among them, L 3D is the three-dimensional distance value, L 2D is the two-dimensional distance value, fx is the effective focal length of the camera, and H is the actual distance value from the package surface to the depth camera.
[0084] S6: The difference between the depth value of the package support surface outside the minimum circumscribed rectangular area and the depth value of the upper surface of the package inside is the height of the package;
[0085] like Figure 5 As shown, the pixel coordinates of the four vertices P1, P2, P3, and P4 of the minimum circumscribed rectangle corresponding to the contour in the wrapped edge area are obtained, and the side lengths are P1P2=L1 and P1P4=L2 respectively;
[0086] Take the midpoints M1 and M2 of the opposite sides P1P4 and P2P3 respectively, and for each pixel point p on the line segment M1M2, traverse the pixel points in the range of 0.5*L2 along the positive and negative directions of the vector P2P3 respectively. Stop traversal when traversing to the edge area of the package, record the depth values of all traversed pixels, take the median of these depth values, and get the depth value inside the minimum circumscribed rectangular area, which is the distance value from the upper surface of the package to the camera.
[0087] like Figure 6As shown, for each pixel point p on the line segment P1P2, traverse along the direction of vector P3P2, traverse 0.4*L2 length, and stop traversing when encountering the minimum circumscribed rectangle and package edge area corresponding to other packages, and record the pixel depth values of all non-edge areas. Similarly, the pixel depth values of all non-edge areas corresponding to the other three sides can be obtained. The pixel depth values of all non-edge areas form an array Qh;
[0088] Perform k-means clustering on the obtained array Qh, and set the number of clusters to two;
[0089] When the difference between the center values of the two clusters is less than the threshold 20 (the depth difference between the package support surface and the next layer of the plane), the median of the depth values of all pixels in the array Qh is taken as the actual distance value from the package support surface to the depth camera.
[0090] When the difference between the center values of two clusters is greater than the threshold 20 (the depth difference between the package support surface and the next layer of plane), the median of all depth values of the smaller k-means cluster is taken as the actual distance value from the package support surface to the depth camera.
[0091] The difference between the actual distance between the upper surface of the package and the camera and the actual distance between the supporting surface of the package and the camera is the height of the package; thus, the volume of the package is calculated.
[0092] S7, determine whether there is any unprocessed minimum bounding rectangle, if yes, repeat S4 to S6, if no, the volume measurement of all packages in the scene is completed.
[0093] Embodiment 2
[0094] The method for measuring the volumes of multiple packages based on vertical photography by a depth camera in this embodiment has the same specific steps as those in the first embodiment.
[0095] like Figure 7 As shown, it is a schematic diagram of the structure of multiple packages in the visual camera light projection in this embodiment, and their actual sizes are: (1) 90 mm long, 90 mm wide, and 70 mm high; (2) 400 mm long, 275 mm wide, and 200 mm high; (3) 210 mm long, 200 mm wide, and 100 mm high.
[0096] Figure 8 1 is a diagram of the measurement results of multiple packages in this embodiment. The processing and calculation results after steps S1 to S7 of the present invention are respectively (1) length 88 mm, width 88 mm, height 70 mm; (2) length 399 mm, width 273 mm, height 201 mm; (3) length 210 mm, width 202 mm, height 98 mm.
[0097] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.
Claims
1. A method for measuring the volume of multiple packages based on vertical photography by a depth camera, characterized in that: The following steps are involved: S1: Use a vertically-shot depth camera to obtain a depth map of the scene containing the package; S2: According to the depth values of all pixels in the depth map, the depth values of each pixel and its neighboring pixels are compared to obtain the pixels whose depth value changes reach the set threshold, and such pixels are assigned as edge points. All edge points constitute the wrapped edge area; The specific process in step S2 is as follows: S21: According to the principle that the depth values of the upper surface of the package and the supporting surface of the package are different, all pixel points are traversed, and the average value of the depth value difference between each pixel point and the pixel points in its set neighborhood is calculated, and the pixel points whose average value of the difference is greater than the set threshold a are obtained. All such pixel points constitute the change area points; S22: traverse the change area points again, calculate the number of change area points in the set neighborhood of each pixel point, obtain the pixel points whose number is greater than the set threshold b as the wrapping edge points, and all the wrapping edge points constitute the wrapping edge area; S3: extract the minimum circumscribed rectangle of all inner contours in the edge area of the package, and then process the minimum circumscribed rectangle corresponding to each package in order of the distance from the upper surface of the package to the depth camera from small to large; The specific process in step S3 is as follows: S31: extracting the minimum bounding rectangles of all inner contours of the wrapped edge area, removing the minimum bounding rectangles whose side lengths are less than c pixels due to noise points, and the total number of all minimum bounding rectangles is recorded as M; S32: measuring the volumes of multiple packages in order from small to large according to the distances from the upper surface of the packages to the depth camera; S33: Calculate the median of the depth values of all pixels on the contour corresponding to each minimum bounding rectangle, obtain the median array of the contours corresponding to all minimum bounding rectangles, and then process all the extracted minimum bounding rectangles in order from small to large in the median array according to the principle that the upper surface of the package is closest to the depth camera; S4: searching for boundary pixel points corresponding to the upper surface of the package for the current minimum circumscribed rectangular area; The specific process in step S4 is as follows: S41: Obtain pixel coordinates P1, P2, P3, and P4 of four vertices of the minimum circumscribed rectangle corresponding to the contour in the wrapped edge area, with side lengths of P1P2=L1 and P1P4=L2, respectively, where P1 and P3 are two intersection points on a pair of diagonals, and P2 and P4 are two intersection points on another pair of diagonals; S42: for each pixel point p on the line segment P1P2, search along the direction of vector P3P2 until the first non-edge area point is reached and stop, which is recorded as point q. The traversal length is recorded as L. Then search along the direction of vector P2P3 within the length range of L for the boundary pixel point. S43: In the process of searching along the direction of vector P2P3, if the depth difference between the traversed pixel point and the previous pixel point is greater than t, then this pixel point is the boundary pixel point corresponding to pixel point p, that is, the boundary pixel point set corresponding to edge P1P2 is obtained; S44: for the line segment P1P4, the line segment P2P3, and the line segment P3P4, the method of steps S42 and S43 is adopted to obtain the boundary pixel point sets corresponding to the edge P1P4, the edge P2P3, and the edge P3P4; S5: Perform straight line fitting on the boundary pixel points to obtain four boundary lines and calculate the four intersection points of the four boundary lines, respectively calculate the distances between adjacent intersection points, and obtain the length and width of the package; S6: respectively calculating the depth values of the inner and outer points of the current minimum circumscribed rectangular area, performing difference processing to obtain the height value of the package, and thus calculating the volume of the package; The specific process in step S6 is as follows: S61: Obtain pixel coordinates P1, P2, P3, and P4 of four vertices of the minimum circumscribed rectangle corresponding to the contour in the wrapped edge area, with side lengths of P1P2=L1 and P1P4=L2, respectively, where P1 and P3 are two intersection points on a pair of diagonals, and P2 and P4 are two intersection points on another pair of diagonals; S62: Take the midpoints M1 and M2 of the opposite sides P1P4 and P2P3 respectively, and for each pixel point p on the line segment M1M2, traverse the pixel points in the range of f*L2 along the positive and negative directions of the vector P2P3, where f is a scale parameter with a value range of 0.3 to 0.
5. Stop traversal when the package edge area is traversed, record the depth values of all traversed pixels, take the median of the depth values of all the above pixels, and obtain the depth value inside the minimum circumscribed rectangular area, which is the actual distance value from the upper surface of the package to the depth camera; S63: For each pixel point p on the line segment P1P2, traverse along the direction of the vector P3P2 for a length of g*L2, where g is a scale parameter with a value range of 0.2 to 0.
4. Stop traversing when encountering the minimum circumscribed rectangle and package edge area corresponding to other packages, and record the depth values of the pixels in all non-edge areas. Similarly, obtain the depth values of the pixels in all non-edge areas corresponding to the other three sides. The depth values of the pixels in all non-edge areas form an array Qh. S64: Perform k-means clustering processing on the obtained array Qh to obtain the actual distance value from the package support surface to the depth camera; S65: Subtract the actual distance value from the upper surface of the package to the depth camera from the actual distance value from the supporting surface of the package to the depth camera and take the absolute value. The calculated result is the height of the package. The volume of the package is calculated by combining the length and width of the package that have been obtained. In step S64, the number of clusters is set to two. When the difference between the center values of the two clusters is less than the threshold value Tk, it indicates that the entire area of the measured package falls on its support surface. The median of the depth values of all pixels in the array Qh is taken as the actual distance value from the package support surface to the depth camera. When the difference between the center values of the two clusters is greater than the threshold Tk, it indicates that part of the measured package area falls on its support surface, and the other part falls on the lower plane. Since the distance from the package support surface to the depth camera is smaller than the distance from the lower plane to the depth camera, the class with a smaller k-means cluster center value corresponds to the points on the package support surface, and the class with a larger k-means cluster center value corresponds to the points on the lower plane. Therefore, the median of all depth values of the smaller k-means cluster is taken as the actual distance value from the package support surface to the depth camera; S7: Determine whether there is any unprocessed minimum bounding rectangle. If so, repeat S4 to S6. If not, the volume measurement of all packages in the scene is completed.
2. The method for measuring the volumes of multiple packages based on vertical photography by a depth camera according to claim 1, characterized in that: In the step S1, the resolution of the depth map is H*W, where H and W are both positive integers.
3. The method for measuring the volumes of multiple packages based on vertical photography by a depth camera according to claim 1, characterized in that: The specific process in step S5 is as follows: S51: performing straight line fitting on the boundary pixel point set corresponding to each edge to obtain straight line equations of four boundary lines; S52: Taking the intersection of two adjacent straight lines as the vertices of the rectangle wrapping the upper surface, that is, obtaining four vertices; S53: Obtain the length and width of the package according to the three-dimensional distances between adjacent vertices.
4. The method for measuring the volumes of multiple packages based on vertical photography by a depth camera according to claim 3, characterized in that: In step S53, the two-dimensional distance value between two adjacent pixel points in the four vertices in the plane is calculated and converted into a three-dimensional distance value in space, that is, the length and width of the package are obtained, thereby calculating the volume of the package; wherein the two-dimensional distance value is converted into a three-dimensional distance value by the following formula: Among them, L 3D is the three-dimensional distance value, L 2D is the two-dimensional distance value, fx is the effective focal length of the depth camera, and H is the actual distance value from the package surface to the depth camera.
