A machine vision-based cargo box volume measurement and storage location allocation method and system

Abstract

The application discloses a kind of based on machine vision's cargo box volume measurement and warehouse location allocation method and system, it is related to computer three-dimensional point cloud technical field, synchronously collect cargo box image data and three-dimensional original point cloud data;Cargo box image data is handled by target detection network model based on RF-DETR, and detection frame data is obtained;Through internal and external parameter transformation matrix, detection frame data is mapped to three-dimensional space coordinate system, and three-dimensional visual pyramid boundary is obtained, according to three-dimensional visual pyramid boundary in three-dimensional original point cloud data, single target point cloud data is obtained by cutting, target cargo box volume data is obtained by calculating single target point cloud data;According to target cargo box volume data, establish virtual bounding box, based on the virtual bounding box in virtual space, execute stacking array deduction detection, output target warehouse location coordinate data, realize the high-density mixed code layout of flexible cargo box, save warehouse and pallet space.

Description

Technical Field

[0001] This invention relates to the field of computer 3D point cloud technology, and in particular to a method and system for measuring the volume of cargo containers and allocating storage locations based on machine vision. Background Technology

[0002] With the rapid development of smart warehousing and modern logistics, machine vision technology has been widely used in the measurement of cargo box dimensions and the allocation and palletizing of storage locations. Traditional vision measurement systems are mostly designed for regular rigid cargo boxes, and usually use the minimum circumscribed cuboid of the three-dimensional point cloud to calculate the volume and allocate storage locations accordingly.

[0003] However, in real-world logistics scenarios, there are numerous flexible and irregular cargo boxes such as woven bags, soft bags, and film packaging. These flexible cargo boxes often squeeze and stick together tightly on conveyor belts, with extremely distorted edges. Existing 2D object detection algorithms are prone to missing detections when dealing with severely occluded and squeezed flexible cargo boxes. Furthermore, existing pure 3D point cloud clustering algorithms cannot identify the semantic boundaries of object surfaces, often misidentifying multiple tightly packed flexible cargo boxes as a single large object, leading to completely erroneous subsequent processing. The surface of flexible packaging has numerous irregular wrinkles, and the film material easily interferes with 3D sensors. Structured light or lasers can cause specular reflection or absorption, resulting in large areas of missing data on the surface of the cargo box in the collected 3D point cloud, causing topological breaks in the measurement model. Traditional bounding box volume algorithms have extremely large errors. When flexible cargo boxes bulge, they are usually ellipsoidal or teardrop-shaped. If the traditional cuboid bounding box algorithm is continued to be used to calculate the volume, a large amount of extra air space will be included in the calculation, resulting in the measured volume being too large than the actual physical volume of the cargo box. This will cause the warehouse management system to waste a lot of warehouse space when allocating storage locations or performing palletizing simulations, and may even cause mixed-stack interference and failure of anti-tipping warning. Summary of the Invention

[0004] The technical problem solved by this invention is that existing technologies cannot accurately measure the irregular real physical volume of flexible cargo boxes due to optical defects caused by wrinkles and reflections on the surface of flexible materials, and cannot perform rigorous stacking simulations in virtual space.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: Firstly, a machine vision-based method for measuring cargo box volume and allocating storage locations includes the following steps: Step S1: Synchronously collect cargo box image data and 3D raw point cloud data; Step S2: Process the cargo box image data using an RF-DETR-based target detection network model to obtain detection box data; Step S3: Through the transformation matrix of internal and external parameters, the detection box data is mapped to the three-dimensional spatial coordinate system to obtain the three-dimensional view frustum boundary. Based on the three-dimensional view frustum boundary, the individual target point cloud data is obtained by cropping from the three-dimensional original point cloud data. The individual target point cloud data is calculated to obtain the target cargo box volume data. Step S4: Establish a virtual bounding box based on the target cargo box volume data, perform stacking array simulation and detection in virtual space based on the virtual bounding box, and output the target storage location coordinate data.

[0006] Preferably, step S1 includes the following sub-steps: Step S11: Deploy photoelectric sensors. When the cargo box arrives at the detection area corresponding to the photoelectric sensor, send an electrical signal to the PLC. The PLC receives the electrical signal and releases an external pulse trigger signal. Step S12: Send the pulse external trigger signal to the industrial 2D camera and 3D laser scanner through the I / O interface, and simultaneously perform image acquisition processing to output cargo box image data and 3D raw point cloud data; The original 3D point cloud data is a set of point clouds of the cargo box, conveyor belt, and cargo box background.

[0007] Preferably, the processing logic of step S2 is as follows: inputting the cargo box image data into the target detection network model based on RF-DETR, specifically including the following sub-steps: Step S21: Input the cargo box image data into the backbone network of the RF-DETR target detection network model, and perform layer-by-layer downsampling processing to obtain a multi-scale feature map set. , where L is the number of feature levels, and the multi-scale feature map set is used to capture the shallow edge geometric features and deep global semantic features of the current cargo box; Step S22: Based on the receptive field attention encoder, spatial feature aggregation weight allocation is calculated on the multi-scale feature map set to obtain the enhanced feature sequence. The calculation expression for the spatial feature aggregation weight allocation is as follows: in, For querying the matrix, The key matrix, For value matrices, The feature channel dimension constant of the key matrix. For normalized exponential functions, To provide the spatial position offset matrix for the receptive field, To enhance the feature sequence; Step S23: Using a multi-head cross-attention decoder, perform global context mapping and decoding calculation on the pre-initialized object query vector set and the enhanced feature sequence to obtain the decoded query feature sequence; Step S24: Based on the feedforward prediction head of the multilayer perceptron, perform position regression and category classification calculations on each feature vector in the decoded query feature sequence to obtain the initial bounding box coordinate set and the corresponding category confidence distribution vector. The calculation expressions for position regression and category classification are as follows: in, To decode the i-th feature vector in the query feature sequence, It is a feedforward neural network. This contains the coordinates of the i-th predicted bounding box, including the x and y coordinates of the center point and the width and height of the bounding box. It is a feedforward neural network. Let be the category confidence distribution vector corresponding to the i-th predicted bounding box; Step S25: Based on a preset confidence threshold, perform coordinate inverse normalization on the initial bounding box coordinate set and the corresponding category confidence distribution vector, and remove invalid prediction boxes below the confidence threshold to obtain detection box data.

[0008] Preferably, step S3 includes the following sub-steps: Step S31: Using the camera intrinsic parameter matrix and the lidar extrinsic parameter transformation matrix, perform inverse perspective projection spatial transformation calculation on the detection box data to obtain three-dimensional spatial rays. These three-dimensional spatial rays are the four spatial rays corresponding to the four coordinate points in a unit sample of the detection box data. A three-dimensional view frustum boundary is constructed using these three-dimensional spatial rays. The expression for the inverse perspective projection spatial transformation calculation is as follows: in, To transform the direction vector of the i-th ray into the lidar coordinate system, The coordinates of the two-dimensional pixel corresponding to the i-th coordinate point in the detection box data. It is the inverse of the camera intrinsic parameter matrix. This is the inverse of the rotation matrix in the extrinsic transformation matrix, used to convert the direction vector in the camera coordinate system to the world coordinate system. Mark, The depth extension parameter of a 3D spatial ray along its propagation direction. Let i be the equation of the i-th spatial ray of the three-dimensional view frustum boundary; Step S32: Using four three-dimensional spatial rays, two adjacent rays and the camera optical center point are treated as a three-dimensional plane. The inward normal vectors of the four boundary planes of the view frustum (top, bottom, left, and right) are calculated by vector cross product to obtain the spatial geometric inclusion determination operator. The three-dimensional original point cloud data is traversed. When the three-dimensional original point cloud data is within the spatial geometric inclusion determination operator, the current three-dimensional original point cloud data is retained. When the three-dimensional original point cloud data is not within the spatial geometric inclusion determination operator, the current three-dimensional original point cloud data is discarded to obtain the initial point cloud data within the view frustum. Step S33: Based on the Random Sample Consensus (RANSAC) algorithm, perform planar fitting on the point cloud data within the initial view frustum to identify and peel off the conveyor belt support surface and the background point cloud below it. Density-based spatial clustering (DBSCAN) is performed on the remaining point cloud after the support surface is removed. The cluster with the most points is selected as the target cargo box body, resulting in a coarse single-unit point cloud set. Step S34: Based on the inverse distance weighted spatial interpolation algorithm, perform elevation weighted calculation on the coarse single-unit point cloud set to fill the point cloud holes and obtain single-unit target point cloud data; Step S35: The single target point cloud data is sliced ​​vertically using the equidistant slicing method. The polygon area of ​​each slice is calculated using the Gaussian area formula, and the target cargo box volume data is calculated based on the height integral of each slice.

[0009] Preferably, step S34 specifically includes: The coarse point cloud set is projected onto a horizontal reference plane in a grid pattern. The point cloud density within each grid is calculated. When the point cloud density within a grid is lower than a set density threshold, the grid below the density threshold is marked as a void region. Based on the inverse distance weighted spatial interpolation algorithm, a set of known elevation points in the neighborhood surrounding the void region is extracted. The void region is then subjected to elevation weighted calculation using the 3D coordinates of the known elevation points to generate supplementary point cloud data. The elevation weighted calculation expression is as follows: in, The generated supplementary point cloud data represents the elevation coordinates within the cavity area, where W is the total number of points from the extracted known elevation point set. Given the elevation coordinates of the p-th point in a set of known elevation points, Let be the two-dimensional Euclidean distance from the p-th point in the set of known elevation points to the center point of the cavity region; The supplementary point cloud data is filled into the void area to obtain the point cloud data of a single target.

[0010] Preferably, step S35 specifically includes: The point cloud data of a single target is sliced ​​in layers along the vertical direction according to a set fixed height step by the equidistant slicing method, resulting in multiple parallel point cloud slice layers. The data points in each point cloud slice layer are projected onto a two-dimensional plane to obtain a two-dimensional point set. The boundary of the two-dimensional point set is extracted and the direction is sorted based on the convex hull algorithm to obtain an ordered closed outer contour point set. The polygon area of ​​the ordered closed outer contour point set is calculated using the Gaussian area formula to obtain the polygon area value of each slice. The polygon area values ​​of all slice layers are multiplied, accumulated, and integrated with the fixed height step size. The expression for calculating the polygon area is as follows: Where M represents the total number of contour points in the sorted closed outer contour point set corresponding to the k-th layer point cloud slice data. These are the x and y coordinates of the j-th contour point in the sorted closed outer contour point set corresponding to the k-th layer point cloud slice data on the two-dimensional projection plane. The target cargo box volume data is obtained by multiplying and integrating the polygon area values ​​of all slice layers with a fixed height step size. The expression for the multiplication and integration is as follows: in, For target cargo container volume data, The set fixed height step size, N is the total number of point cloud slice layer data.

[0011] Preferably, step S4 establishes a virtual bounding box based on the target cargo box volume data, specifically including: Based on the target cargo box volume data and combined with the minimum circumscribed cuboid parameter of the target cargo box in three-dimensional space, a virtual bounding box adapted to the geometric dimensions of the target cargo box is constructed. The pose information of the virtual bounding box in the three-dimensional spatial coordinate system includes the coordinates of the center point of the bounding box, the length of the long side, the length of the short side, the height, and the rotation azimuth angle around the vertical axis. The virtual bounding box is used to replace the physical entity of the target cargo box in subsequent stacking array simulation and detection to simulate space occupancy.

[0012] Preferably, the processing logic for performing array pattern deduction and detection based on the virtual bounding box in virtual space is as follows: Acquire current warehouse location status data, which includes a set of virtual bounding boxes of occupied locations, a set of coordinates of vacant locations, location load-bearing limit data, and shelf height constraint data; perform stacking simulation on candidate locations one by one in the virtual space based on the virtual bounding boxes, which includes collision detection and stability assessment. The collision detection processing logic is as follows: The virtual bounding box is placed at the candidate storage location coordinates according to a preset stacking posture, and it is determined whether the virtual bounding box has spatial interference with any virtual bounding box in the set of virtual bounding boxes of the occupied storage location; The processing logic for the stability assessment is as follows: Calculate the centroid projection point of the virtual bounding box at the candidate storage location, and determine whether the centroid projection point falls within the support surface range of the candidate storage location: When the collision detection result is no interference and the stability assessment result is that the center of gravity projection point falls within the support surface range, the candidate storage location is marked as a feasible storage location. From all feasible storage locations, the optimal storage location is selected according to the optimization objective, and the coordinate data of the target storage location is output.

[0013] Preferably, the optimization objectives include maximizing space utilization, prioritizing the number of stacking layers, and prioritizing the shortest transport path; The three-dimensional spatial coordinate data corresponding to the selected optimal storage location is output as the target storage location coordinate data to the warehouse management system.

[0014] Secondly, a machine vision-based system for measuring cargo box volume and allocating storage locations includes a data acquisition module, a target network detection module, a cone mapping module, and a storage location allocation module. The data acquisition module is used to simultaneously acquire cargo box image data and three-dimensional raw point cloud data; The target network detection module is used to process the cargo box image data using an RF-DETR-based target detection network model to obtain detection box data; The frustum mapping module is used to map the detection box data to a three-dimensional spatial coordinate system through an intrinsic and extrinsic parameter transformation matrix to obtain the three-dimensional frustum boundary. Based on the three-dimensional frustum boundary, it clips the individual target point cloud data from the three-dimensional original point cloud data and calculates the target cargo box volume data from the individual target point cloud data. The storage location allocation module is used to establish a virtual bounding box based on the target cargo box volume data, perform stacking array simulation and detection in virtual space based on the virtual bounding box, and output the target storage location coordinate data.

[0015] The beneficial effects of this invention are as follows: Based on the RF-DETR architecture, it completely eliminates missed detections and false deletions in densely stacked scenarios. In the two-dimensional target detection stage, this invention introduces a network model based on the RF-DETR receptive field attention transformer. This model utilizes the global context self-attention mechanism of Transformer and bipartite graph matching loss to achieve end-to-end anchorless box detection. It does not rely on the traditional NMS post-processing algorithm, avoiding the industry problem of false deletion of real box frames by the algorithm when flexible boxes are tightly stacked and the detection boxes overlap. This improves the detection rate and edge localization accuracy of irregularly shaped flexible packages. This invention innovatively introduces cross-dimensional frustum mapping to achieve lossless 3D adhesion segmentation. It creatively utilizes the camera's intrinsic and extrinsic transformation matrices to project the 2D detection box inverse perspective into a 3D frustum boundary. This mechanism reduces the globally disordered 3D point cloud segmentation problem to a local background removal problem within the frustum. Leveraging the strong semantic texture recognition capabilities of 2D images to guide geometric clipping in 3D space, it perfectly removes interfering point clouds from adjacent cargo boxes, solving the problem of traditional 3D Euclidean clustering's difficulty in separating edges when flexible cargo boxes are squeezed and adhered. By combining IDW interpolation and convex hull calculus, this invention achieves volume measurement with extreme accuracy. Addressing the issues of missing point clouds and distorted volume calculations on flexible cargo box surfaces, this invention first employs an inverse distance-weighted spatial interpolation algorithm to fill point cloud voids caused by wrinkles and reflections. Subsequently, it completely abandons the traditional cuboid bounding box algorithm and introduces an equidistant slicing calculus mechanism. Specifically, this invention uses a convex hull algorithm to extract closed outer contours in each slice and combines this with the Gaussian area formula for infinitesimal area integration. The convex hull method not only automatically fills in depressions on flexible surfaces and avoids contour self-intersections, but also calculates an area that precisely matches the cargo box's surface area. The physical anti-collision area required for the height layer is output by this mechanism as the absolutely real physical arrangement volume of the flexible cargo box. Combined with digital twin virtual simulation, it maximizes the utilization of warehouse space and operational safety. This invention constructs a compact virtual bounding box based on the obtained real cargo box volume data with the highest accuracy. Before outputting the physical warehouse location coordinates, it performs a stacking array simulation in the digital twin virtual space in advance. This can intercept the risk of overstepping or center of gravity shift caused by the local expansion of the flexible cargo box. Under the premise of ensuring the absolute safety of stacking and anti-tipping, it realizes the high-density mixed stacking of flexible cargo boxes, saving warehouse and pallet space. Attached Figure Description

[0016] Figure 1 A flowchart illustrating the steps of a machine vision-based method for measuring cargo box volume and allocating storage locations, as provided in one embodiment of the present invention. Figure 2 This is a basic flowchart of a machine vision-based cargo box volume measurement and storage location allocation system provided in one embodiment of the present invention. Detailed Implementation

[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0018] Example 1, referring to Figure 1 This paper provides a machine vision-based method for measuring the volume of cargo containers and allocating storage locations, including the following steps: Step S1: Synchronously collect cargo box image data and 3D raw point cloud data; Step S2: Process the cargo box image data using an RF-DETR-based target detection network model to obtain detection box data; Step S3: Through the transformation matrix of internal and external parameters, the detection box data is mapped to the three-dimensional spatial coordinate system to obtain the three-dimensional view frustum boundary. Based on the three-dimensional view frustum boundary, the individual target point cloud data is obtained by cropping from the three-dimensional original point cloud data. The individual target point cloud data is calculated to obtain the target cargo box volume data. Step S4: Establish a virtual bounding box based on the target cargo box volume data, perform stacking array simulation and detection in virtual space based on the virtual bounding box, and output the target storage location coordinate data.

[0019] In this embodiment, the method first acquires high-precision raw data through synchronous acquisition by multiple sensors; secondly, it uses a deep learning model based on RF-DETR to accurately locate the cargo box area from the two-dimensional image; combining the principles of visual geometry, it uses an intrinsic and extrinsic parameter transformation matrix to map the two-dimensional detection box into a view frustum in three-dimensional space, accurately cropping the individual point cloud of the target cargo box from the dense three-dimensional point cloud, and calculating accurate volume data through a point cloud processing algorithm; finally, it constructs a virtual bounding box based on the volume data, performs stacking simulation in the virtual space, and outputs the optimal storage location coordinates. This method solves the problem of inaccurate volume measurement caused by occlusion and background interference in traditional visual measurement, and realizes accurate measurement of cargo box volume and dynamic optimization allocation of storage location.

[0020] Step S1 includes the following sub-steps: Step S11: Deploy photoelectric sensors. When the cargo box arrives at the detection area corresponding to the photoelectric sensor, send an electrical signal to the PLC. The PLC receives the electrical signal and releases an external pulse trigger signal. Step S12: Send the pulse external trigger signal to the industrial 2D camera and 3D laser scanner through the I / O interface, and simultaneously perform image acquisition processing to output cargo box image data and 3D raw point cloud data; The original 3D point cloud data is a collection of point clouds of the cargo box, conveyor belt, and cargo box background.

[0021] In this embodiment, the data acquisition synchronization mechanism is further defined. In traditional methods, if the camera and lidar use soft triggering based on timestamp synchronization, data misalignment is easily caused by network latency, making it impossible for the two-dimensional image and the three-dimensional point cloud to correspond one-to-one. Therefore, this step deploys a photoelectric sensor as a physical trigger. When the cargo box is transported to the detection area, the photoelectric sensor level changes. After receiving the signal, the PLC immediately releases a high-precision pulse external trigger signal. The industrial two-dimensional camera and the three-dimensional laser scanner simultaneously receive the pulse signal through the I / O interface, realizing microsecond-level time synchronization acquisition. The output three-dimensional raw point cloud data includes the target cargo box, the conveyor belt, and the background environment, providing a data foundation for subsequent accurate segmentation.

[0022] The processing logic of step S2 is as follows: input the cargo box image data into the RF-DETR-based object detection network model, specifically including the following sub-steps: Step S21: Input the cargo box image data into the backbone network of the RF-DETR object detection network model, and perform layer-by-layer downsampling processing to obtain a multi-scale feature map set. , where L is the number of feature levels, and the multi-scale feature map set is used to capture the shallow edge geometric features and deep global semantic features of the current cargo box; Step S22: Based on the receptive field attention encoder, spatial feature aggregation weight allocation is calculated on the multi-scale feature map set to obtain the enhanced feature sequence. The calculation expression for spatial feature aggregation weight allocation is as follows: in, For querying the matrix, The key matrix, For value matrices, The feature channel dimension constant of the key matrix. For normalized exponential functions, To provide the spatial position offset matrix for the receptive field, To enhance the feature sequence; Step S23: Using a multi-head cross-attention decoder, perform global context mapping and decoding calculation on the pre-initialized object query vector set and the enhanced feature sequence to obtain the decoded query feature sequence; Step S24: Based on the feedforward prediction head of the multilayer perceptron, perform position regression and category classification calculations on each feature vector in the decoded query feature sequence to obtain the initial bounding box coordinate set and the corresponding category confidence distribution vector. The calculation expressions for position regression and category classification are as follows: in, To decode the i-th feature vector in the query feature sequence, It is a feedforward neural network. This contains the coordinates of the i-th predicted bounding box, including the x and y coordinates of the center point and the width and height of the bounding box. It is a feedforward neural network. Let be the category confidence distribution vector corresponding to the i-th predicted bounding box; Step S25: Based on a preset confidence threshold, perform coordinate inverse normalization on the initial bounding box coordinate set and the corresponding category confidence distribution vector, and remove invalid prediction boxes below the confidence threshold to obtain detection box data.

[0023] This embodiment details the target detection process based on RF-DETR. To address the limitations of traditional convolutional neural networks in their receptive field and difficulty in capturing large-scale global features of cargo boxes, this embodiment introduces a receptive field attention encoder. The core of the receptive field attention encoder lies in the introduction of a receptive field spatial position bias matrix. This matrix is ​​not a fixed constant but is generated through a learnable mechanism: during feature map processing, the network constructs a position-sensitive spatial offset field based on dilated convolution or deformable convolution. This spatial offset field can adaptively redistribute attention weights to discriminative feature regions (such as the edges of the cargo box) according to the geometric shape of the cargo box edges. In attention calculation, the result of traditional scaled dot product attention is added element-wise to the receptive field spatial position bias matrix, so that the attention score depends not only on the semantic similarity of features but also on the relative spatial position relationship between feature points. After Softmax normalization, it is multiplied by the value matrix V to obtain an enhanced feature sequence containing explicit spatial geometric constraints. This mechanism significantly improves the model's detection accuracy for densely stacked and blurred-edge cargo boxes.

[0024] Step S3 includes the following sub-steps: Step S31: Using the camera intrinsic parameter matrix and the lidar extrinsic parameter transformation matrix, perform inverse perspective projection spatial transformation calculation on the detection box data to obtain three-dimensional spatial rays. The three-dimensional spatial rays are the four spatial rays corresponding to the four coordinate points in the unit sample of the detection box data. Construct the three-dimensional view frustum boundary using the three-dimensional spatial rays. The expression for the inverse perspective projection spatial transformation calculation is as follows: in, To transform the direction vector of the i-th ray into the lidar coordinate system, The coordinates of the two-dimensional pixel corresponding to the i-th coordinate point in the detection box data. It is the inverse of the camera intrinsic parameter matrix. It is the inverse of the rotation matrix in the extrinsic transformation matrix. Let the three-dimensional coordinates of the camera's optical center in the lidar coordinate system be given. The depth extension parameter of a 3D spatial ray along its propagation direction. Let be the equation of the i-th spatial ray at the boundary of the three-dimensional view frustum; Step S32: Using four three-dimensional spatial rays, two adjacent rays and the camera optical center point are treated as a three-dimensional plane. The inward normal vectors of the four boundary planes of the view frustum (top, bottom, left, and right) are calculated by vector cross product to obtain the spatial geometric inclusion determination operator. The three-dimensional original point cloud data is traversed. When the three-dimensional original point cloud data is within the spatial geometric inclusion determination operator, the current three-dimensional original point cloud data is retained. When the three-dimensional original point cloud data is not within the spatial geometric inclusion determination operator, the current three-dimensional original point cloud data is discarded to obtain the initial point cloud data within the view frustum. Step S33: Based on the Random Sample Consensus (RANSAC) algorithm, perform planar fitting on the point cloud data within the initial view frustum to identify and peel off the conveyor belt support surface and the background point cloud below it. Density-based spatial clustering (DBSCAN) is performed on the remaining point cloud after the support surface is removed. The cluster with the most points is selected as the target cargo box body, resulting in a coarse single-unit point cloud set. Step S34: Based on the inverse distance weighted spatial interpolation algorithm, perform elevation weighted calculation on the coarse single-unit point cloud set to fill the point cloud holes and obtain the single-unit target point cloud data. Step S35: The point cloud data of a single target is sliced ​​vertically using the equidistant slicing method. The polygon area of ​​each slice is calculated using the Gaussian area formula, and the volume data of the target cargo box is calculated based on the height integral of each slice.

[0025] In this embodiment, a method for extracting individual cargo boxes from a point cloud and calculating their volume is further defined, wherein, The extrinsic transformation matrix is ​​precisely the inverse of the rotation matrix in the extrinsic parameter matrix of the camera-LiDAR joint calibration, and is used to convert the direction vector in the camera coordinate system to the LiDAR coordinate system. The camera intrinsic parameter matrix K and the camera-LiDAR joint calibration matrix are used. This joint calibration matrix is ​​a transformation matrix obtained by aligning the pixel coordinate system of the 2D camera with the measurement coordinate system of the 3D LiDAR in space using the Zhang Zhengyou calibration method combined with a checkerboard calibration object. Through the inverse transformation of this matrix, the four corner points of the 2D detection box are back-projected into 3D space, forming four rays and constructing a view frustum. Then, to remove the interference from the conveyor belt plane, the Random Sample Consensus (RANSAC) algorithm is used. Iteratively, the minimum point set in the point cloud is randomly selected to fit a planar model, finding the one containing the most inliers (i.e., the conveyor belt plane). The planar surface (containing surface points) is peeled off, and the planar surface and the background point cloud below it are used as the support surface to separate a rough single point cloud containing only the main body of the cargo box. In the volume calculation stage, this embodiment adopts a strategy combining the equidistant slicing method and the Gaussian formula. Considering that there may be pits or protrusions on the surface of the cargo box, directly fitting a cuboid will have a large error: the single target point cloud is sliced ​​along the vertical direction (Z-axis) with a set fixed step size; for each slice, the three-dimensional points are projected onto a two-dimensional plane, and the area of ​​the polygon is calculated using the Gaussian formula; finally, the accurate target cargo box volume data is obtained by accumulating through the integral formula.

[0026] Step S34 specifically includes: The coarse point cloud set is projected onto a horizontal reference plane in a grid pattern. The point cloud density within each grid is calculated. When the point cloud density within a grid is lower than a set density threshold, the grid below the threshold is marked as a void region. Based on the inverse distance weighted spatial interpolation algorithm, a set of known elevation points in the neighborhood surrounding the void region is extracted. The 3D coordinates of the known elevation points are used to perform elevation weighted calculation on the void region to generate supplementary point cloud data. The elevation weighted calculation expression is as follows: in, The generated supplementary point cloud data contains the elevation coordinates within the cavity region, where W represents the total number of points from the extracted known elevation point set. Given the elevation coordinates of the p-th point in a set of known elevation points, Let be the two-dimensional Euclidean distance from the p-th point in the set of known elevation points to the center point of the cavity region; The missing point cloud data is filled into the void area to obtain the point cloud data of the individual target.

[0027] This embodiment specifically describes a method for filling voids on the surface of flexible cargo boxes. Because flexible cargo boxes (such as woven bags, soft packaging, and film packaging) have irregular features such as wrinkles and depressions on their surface, 3D laser scanners are prone to creating local scanning blind spots on inclined surfaces when collecting data, resulting in point cloud holes. This step uses a grid-by-grid inverse distance weighted (IDW) spatial interpolation algorithm for accurate repair. The specific process is as follows: The coarse single-unit point cloud set is projected onto the horizontal conveyor belt reference plane in a gridded manner. Preferably, the grid size g is set to 5mm ~ 10mm. The point cloud density in each grid is counted. If the point cloud density of a certain grid is lower than the density threshold (density threshold is 3), it is marked as a candidate void grid. Adjacent (four-neighbor or eight-neighbor) candidate void grids are merged into a connected void region. In subsequent interpolation, the information of surrounding known points is shared to ensure that the filled surface remains continuous and smooth. For each void grid, the geometric center of the grid is taken as the point to be interpolated. The search radius R = k*g (preferably k = 3~5) is set. All known data points within this radius range are extracted to form a known elevation point set. The inverse distance weighted interpolation formula is used to calculate the point to be interpolated. The physical meaning of the elevation value distance weighted interpolation is that the closer the known real point is to the center of the void, the greater its determination weight on the void height, thus perfectly preserving the local undulation characteristics of the flexible surface. Three-dimensional supplementary point cloud data is generated at the two-dimensional projection position of the grid and merged with the original coarse single-unit point cloud to output single-unit target point cloud data with complete surface topology.

[0028] Step S35 specifically includes: The point cloud data of a single target is sliced ​​in layers along the vertical direction according to a set fixed height step by the equidistant slicing method, resulting in multiple parallel point cloud slice layers. The data points in each point cloud slice layer are projected onto a two-dimensional plane to obtain a two-dimensional point set. The boundary of the two-dimensional point set is extracted and the direction is sorted based on the convex hull algorithm to obtain an ordered closed outer contour point set. The Gaussian area formula is used to calculate the polygon area of ​​an ordered closed outer contour point set, thus obtaining the polygon area value for each slice. The polygon area values ​​of all slice layers are multiplied, summed, and integrated with a fixed height step size. The expression for calculating the polygon area is: Where M represents the total number of contour points in the sorted closed outer contour point set corresponding to the k-th layer point cloud slice data. Let x and y be the x and y coordinates of the j-th contour point in the sorted closed outer contour point set corresponding to the k-th layer point cloud slice data, respectively, on the two-dimensional projection plane; when j=M... The same applies to y; The target cargo box volume data is obtained by multiplying, accumulating, and integrating the polygon area values ​​of all slice layers with a fixed height step size. The expression for calculating the multiplication, accumulation, and integration is as follows: in, For target cargo container volume data, The set fixed height step size, N is the total number of point cloud slice layer data.

[0029] This embodiment abandons the traditional minimum circumscribed cuboid (OBB) algorithm and uses slicing calculus combined with convex hull algorithm to accurately calculate the physical unfolded volume of the flexible cargo box. The specific process is as follows: With a fixed height step along the vertical direction (Z-axis) The point cloud of a single target is sliced ​​into layers, and the optimal step size is selected for flexible cargo boxes. =4mm, outer contour extraction based on convex hull method: Project the data points in each slice onto the horizontal plane to obtain a two-dimensional point set. Then, use Graham scan method or Andrew algorithm to calculate the convex hull boundary of this two-dimensional point set. Due to the characteristics of the convex hull algorithm, it will output a closed outer contour point set strictly ordered in the counterclockwise direction. The advantage of using the convex hull method is that it not only completely avoids the self-intersection problem of contour lines caused by the concavity of flexible surfaces, but also the convex hull area exactly represents the physical anti-collision projection area required by the cargo box at this height layer, perfectly matching the warehousing and storage logic. Substitute the outer contour point set ordered in the counterclockwise direction into the Gaussian area formula to calculate the cross-sectional area of ​​the slice. Multiply the area of ​​each slice by the layer height and sum them to obtain the target cargo box volume data. Note that if the actual height of the last slice at the highest point is... If the height step is insufficient, the volume of the last layer will be dynamically compensated and corrected; that is, the volume of the last layer will be calculated as follows: This ensures the absolute accuracy of calculus calculations.

[0030] Step S4 establishes a virtual bounding box based on the target cargo container volume data, specifically including: Based on the target cargo box volume data and combined with the minimum bounding box parameter of the target cargo box in three-dimensional space, a virtual bounding box that matches the geometric dimensions of the target cargo box is constructed. The pose information of the virtual bounding box in the three-dimensional coordinate system includes the coordinates of the bounding box center point, the dimensions of the long side, the dimensions of the short side, the height dimension, and the rotation azimuth angle about the vertical axis; Virtual bounding boxes are used to replace the physical entities of the target cargo boxes in subsequent stacking array simulation and detection to simulate space occupancy.

[0031] In this embodiment, in order to realize the subsequent stacking array simulation, it is necessary to convert the physical cargo box into a geometric proxy model in the virtual space and construct a virtual bounding box that matches the geometric size of the target cargo box: Based on the obtained single target point cloud data, the minimum bounding box parameter of the target cargo box in the three-dimensional spatial coordinate system is extracted. The minimum bounding box parameter is obtained by first calculating the principal direction of the target cargo box point cloud through principal component analysis (PCA) and obtaining the rotation azimuth angle of the cargo box around the vertical axis Z. The point cloud coordinates are rotated to a local coordinate system aligned with the main direction. The extreme coordinate differences of the point cloud in each axis direction are calculated in this local coordinate system to obtain the long side dimension L, the short side dimension W, and the height dimension H. The target cargo box volume data is used for warehouse load-bearing assessment and comparison with the virtual bounding box volume L×W×H to obtain the cargo box filling rate coefficient. The filling rate coefficient is used for stacking pressure judgment in the stacking array simulation.

[0032] Meanwhile, considering that the cargo box may deflect during actual transport, it is necessary to calculate the rotation azimuth angle of the cargo box around the vertical axis (Z-axis). This angle is obtained by principal component analysis (PCA) to construct a virtual bounding box. The three-dimensional spatial pose information of the virtual bounding box includes the coordinates of the center point of the bounding box, the long side dimension L, the short side dimension W, the height dimension H, and the rotation azimuth angle around the vertical axis. Virtual bounding boxes are used to replace the physical entities of target cargo boxes in subsequent stacking array simulation and detection to simulate space occupancy. By abstracting physical cargo boxes into virtual bounding boxes with clear geometric boundaries and pose information, collision detection and layout optimization can be performed efficiently in virtual space, avoiding the computational burden caused by directly manipulating dense point clouds.

[0033] The processing logic for performing array pattern deduction and detection in virtual space based on virtual bounding boxes is as follows: Acquire current warehouse location status data, which includes a set of virtual bounding boxes of occupied locations, a set of coordinates of vacant locations, location load-bearing limit data, and shelf height constraint data; Based on the virtual bounding boxes, perform stacking simulation on candidate locations one by one in the virtual space, including collision detection and stability assessment. The collision detection processing logic is as follows: Place the virtual bounding box at the candidate storage location coordinates according to the preset stacking posture, and determine whether the virtual bounding box has spatial interference with any virtual bounding box in the set of virtual bounding boxes of the already occupied storage location. The processing logic for stability assessment is as follows: Calculate the centroid projection point of the virtual bounding box at the candidate storage location, and determine whether the centroid projection point falls within the support surface range of the candidate storage location: When the collision detection result is no interference and the stability assessment result is that the center of gravity projection point falls within the support surface range, the candidate storage location is marked as a feasible storage location. From all feasible storage locations, the optimal storage location is selected according to the optimization objective, and the coordinate data of the target storage location is output.

[0034] In this embodiment, the current warehouse location status data is obtained. This data is retrieved in real time from the warehouse management system (WMS), specifically including: A set of virtual bounding boxes for occupied storage locations, where each occupied storage location corresponds to a virtual bounding box containing a stored cargo box; The set of coordinates of available storage locations contains the position coordinates of all available storage locations in the warehouse's three-dimensional spatial coordinate system; Storage location weight limit data is used to assess whether the weight of the cargo box exceeds the storage location's carrying capacity; Shelf height constraint data is used to ensure that the height of the goods box does not exceed the shelf height; Based on virtual bounding boxes, candidate storage locations are stacked and simulated one by one in virtual space. For each free storage location coordinate, collision detection and stability assessment are performed: virtual bounding boxes are placed at the candidate storage location coordinates according to preset stacking postures, including flat placement (default posture), vertical placement (swapping the long side and the height side), and rotation (rotating around the vertical axis by 90°, 180°, etc.). After placement, the set of virtual bounding boxes of occupied storage locations is traversed to determine whether the current virtual bounding box has spatial interference with any occupied virtual bounding box. The interference judgment criterion adopts the Separating Axis Theorem (SAT). The separating axes to be checked include the three local coordinate axes of each of the two virtual bounding boxes and the nine axes formed by the cross product of the two sets of local axes, for a total of fifteen separating axes. If the projection intervals on all fifteen separating axes overlap, it is determined that a collision has occurred. If there is no overlap in the projection intervals on any separating axis, it is determined that there is no collision. Calculate the center of gravity projection point of the virtual bounding box at the candidate storage location. For regular boxes, the center of gravity is located at the geometric center of the virtual bounding box. For irregular flexible boxes, the center of gravity position can be calculated from point cloud data (i.e., the weighted average of all point cloud coordinates). Project the center of gravity coordinates onto the horizontal plane (XOY plane) to obtain the center of gravity projection point. Obtain the support surface range of the candidate storage location. The support surface is the rectangular area corresponding to the storage location pallet or shelf. Determine whether the center of gravity projection point falls within the support surface range. If it falls within, it is determined that the box can remain stable after being stacked and will not tip over. If it does not fall within, it is determined to be unstable. Only when the collision detection result is non-interference and the stability assessment result is that the center of gravity projection point falls within the support surface range, is the candidate storage location marked as a feasible storage location and included in the subsequent selection range.

[0035] Optimization objectives include maximizing space utilization, prioritizing the number of stacking layers, and prioritizing the shortest handling path; The three-dimensional spatial coordinate data corresponding to the selected optimal storage location is output as the target storage location coordinate data to the warehouse management system.

[0036] In this embodiment, the setting rule for maximizing space utilization is: prioritize the selection of the storage location that makes the remaining free space most compact after stacking, that is, select the storage location with the smallest gap between the virtual enclosure box and the adjacent cargo box. The stacking layer priority setting rule is: prioritize stacking boxes at lower layer heights to reduce handling energy consumption and facilitate subsequent picking; The shortest handling path priority setting rule is as follows: prioritize selecting the vacant storage location closest to the current inbound port or conveyor line to reduce the travel path length of automated handling equipment (such as AGVs and stacker cranes); output the three-dimensional spatial coordinate data corresponding to the selected optimal storage location as the target storage location coordinate data to the warehouse management system (WMS). The warehouse management system generates handling instructions based on the target storage location coordinate data, and uses automated handling equipment to transport the target box from the inspection station to the designated optimal storage location to complete the inbound operation. This embodiment realizes a complete closed loop from accurate measurement of box volume to intelligent allocation of storage location, improving the space utilization efficiency and automation level of the warehouse.

[0037] Example 2, refer to Figure 2 This paper presents a machine vision-based system for measuring the volume of cargo boxes and allocating storage locations, including a data acquisition module, a target network detection module, a view frustum mapping module, and a storage location allocation module. The data acquisition module is used to simultaneously acquire cargo box image data and 3D raw point cloud data; The target network detection module is used to process cargo box image data through an RF-DETR-based target detection network model to obtain detection box data; The view frustum mapping module is used to map the detection box data to a three-dimensional spatial coordinate system through the transformation matrix of intrinsic and extrinsic parameters to obtain the three-dimensional view frustum boundary. Based on the three-dimensional view frustum boundary, the module clips the individual target point cloud data from the three-dimensional original point cloud data and calculates the target cargo box volume data from the individual target point cloud data. The storage location allocation module is used to create a virtual bounding box based on the target cargo box volume data, perform stacking array simulation and detection in virtual space based on the virtual bounding box, and output the target storage location coordinate data.

[0038] In this embodiment, a network model based on the RF-DETR receptive field attention transformer is introduced in the two-dimensional target detection stage. This model uses the global context self-attention mechanism of Transformer and bipartite graph matching loss to achieve end-to-end anchorless box detection. It does not rely on the traditional NMS post-processing algorithm, avoiding the industry problem that the real box frame is mistakenly deleted by the algorithm when flexible boxes are tightly stacked and the detection boxes overlap. This improves the detection rate and edge localization accuracy of irregular flexible packages. This invention innovatively introduces cross-dimensional frustum mapping to achieve lossless 3D adhesion segmentation. It creatively utilizes the camera's intrinsic and extrinsic transformation matrices to project the 2D detection box inverse perspective into a 3D frustum boundary. This mechanism reduces the globally disordered 3D point cloud segmentation problem to a local background removal problem within the frustum. Leveraging the strong semantic texture recognition capabilities of 2D images to guide geometric clipping in 3D space, it perfectly removes interfering point clouds from adjacent cargo boxes, solving the problem of traditional 3D Euclidean clustering's difficulty in separating edges when flexible cargo boxes are squeezed and adhered. By combining IDW interpolation and convex hull calculus, this invention achieves volume measurement with extreme accuracy. Addressing the issues of missing point clouds and distorted volume calculations on flexible cargo box surfaces, this invention first employs an inverse distance-weighted spatial interpolation algorithm to fill point cloud voids caused by wrinkles and reflections. Subsequently, it completely abandons the traditional cuboid bounding box algorithm and introduces an equidistant slicing calculus mechanism. Specifically, this invention uses a convex hull algorithm to extract closed outer contours in each slice and combines this with the Gaussian area formula for infinitesimal area integration. The convex hull method not only automatically fills in depressions on flexible surfaces and avoids contour self-intersections, but also calculates an area that precisely matches the cargo box's surface area. The physical anti-collision area required for the height layer is output by this mechanism as the absolutely real physical arrangement volume of the flexible cargo box. Combined with digital twin virtual simulation, it maximizes the utilization of warehouse space and operational safety. This invention constructs a compact virtual bounding box based on the obtained real cargo box volume data with the highest accuracy. Before outputting the physical warehouse location coordinates, it performs a stacking array simulation in the digital twin virtual space in advance. This can intercept the risk of overstepping or center of gravity shift caused by the local expansion of the flexible cargo box. Under the premise of ensuring the absolute safety of stacking and anti-tipping, it realizes the high-density mixed stacking of flexible cargo boxes, saving warehouse and pallet space.

[0039] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0040] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the protection scope of the present invention.

Claims

1. A method for measuring cargo box volume and allocating storage locations based on machine vision, characterized in that, Includes the following steps: Step S1: Synchronously collect cargo box image data and 3D raw point cloud data; Step S2: Process the cargo box image data using an RF-DETR-based target detection network model to obtain detection box data; Step S3: Through the transformation matrix of internal and external parameters, the detection box data is mapped to the three-dimensional spatial coordinate system to obtain the three-dimensional view frustum boundary. Based on the three-dimensional view frustum boundary, the individual target point cloud data is obtained by cropping from the three-dimensional original point cloud data. The individual target point cloud data is calculated to obtain the target cargo box volume data. Step S4: Establish a virtual bounding box based on the target cargo box volume data, perform stacking array simulation and detection in virtual space based on the virtual bounding box, and output the target storage location coordinate data.

2. The method for measuring cargo box volume and allocating storage locations based on machine vision as described in claim 1, characterized in that, Step S1 includes the following sub-steps: Step S11: Deploy photoelectric sensors. When the cargo box arrives at the detection area corresponding to the photoelectric sensor, send an electrical signal to the PLC. The PLC receives the electrical signal and releases an external pulse trigger signal. Step S12: Send the pulse external trigger signal to the industrial 2D camera and 3D laser scanner through the I / O interface, and simultaneously perform image acquisition processing to output cargo box image data and 3D raw point cloud data; The original 3D point cloud data is a set of point clouds of the cargo box, conveyor belt, and cargo box background.

3. The method for measuring cargo box volume and allocating storage locations based on machine vision as described in claim 2, characterized in that, The processing logic of step S2 is as follows: inputting the cargo box image data into the RF-DETR-based target detection network model, specifically including the following sub-steps: Step S21: Input the cargo box image data into the backbone network of the RF-DETR target detection network model, and perform layer-by-layer downsampling processing to obtain a multi-scale feature map set. , where L is the number of feature levels, and the multi-scale feature map set is used to capture the shallow edge geometric features and deep global semantic features of the current cargo box; Step S22: Based on the receptive field attention encoder, spatial feature aggregation weight allocation is calculated on the multi-scale feature map set to obtain the enhanced feature sequence. The calculation expression for the spatial feature aggregation weight allocation is as follows: in, For querying the matrix, The key matrix, For value matrices, The feature channel dimension constant of the key matrix. For normalized exponential functions, To provide the spatial position offset matrix for the receptive field, To enhance the feature sequence; Step S23: Using a multi-head cross-attention decoder, perform global context mapping and decoding calculation on the pre-initialized object query vector set and the enhanced feature sequence to obtain the decoded query feature sequence; Step S24: Based on the feedforward prediction head of the multilayer perceptron, perform position regression and category classification calculations on each feature vector in the decoded query feature sequence to obtain the initial bounding box coordinate set and the corresponding category confidence distribution vector. The calculation expressions for position regression and category classification are as follows: in, To decode the i-th feature vector in the query feature sequence, It is a feedforward neural network. This contains the coordinates of the i-th predicted bounding box, including the x and y coordinates of the center point and the width and height of the bounding box. It is a feedforward neural network. Let be the category confidence distribution vector corresponding to the i-th predicted bounding box; Step S25: Based on a preset confidence threshold, perform coordinate inverse normalization on the initial bounding box coordinate set and the corresponding category confidence distribution vector, and remove invalid prediction boxes below the confidence threshold to obtain detection box data.

4. The method for measuring cargo box volume and allocating storage locations based on machine vision as described in claim 3, characterized in that, Step S3 includes the following sub-steps: Step S31: Using the camera intrinsic parameter matrix and the lidar extrinsic parameter transformation matrix, perform inverse perspective projection spatial transformation calculation on the detection box data to obtain three-dimensional spatial rays. These three-dimensional spatial rays are the four spatial rays corresponding to the four coordinate points in a unit sample of the detection box data. A three-dimensional view frustum boundary is constructed using these three-dimensional spatial rays. The expression for the inverse perspective projection spatial transformation calculation is as follows: in, To transform the direction vector of the i-th ray into the lidar coordinate system, The coordinates of the two-dimensional pixel corresponding to the i-th coordinate point in the detection box data. It is the inverse of the camera intrinsic parameter matrix. This is the inverse of the rotation matrix in the extrinsic transformation matrix, used to convert the direction vector in the camera coordinate system to the world coordinate system. Mark, The depth extension parameter of a 3D spatial ray along its propagation direction. Let i be the equation of the i-th spatial ray of the three-dimensional view frustum boundary; Step S32: Using four three-dimensional spatial rays, two adjacent rays and the camera optical center point are treated as a three-dimensional plane. The inward normal vectors of the four boundary planes of the view frustum (top, bottom, left, and right) are calculated by vector cross product to obtain the spatial geometric inclusion determination operator. The three-dimensional original point cloud data is traversed. When the three-dimensional original point cloud data is within the spatial geometric inclusion determination operator, the current three-dimensional original point cloud data is retained. When the three-dimensional original point cloud data is not within the spatial geometric inclusion determination operator, the current three-dimensional original point cloud data is discarded to obtain the initial point cloud data within the view frustum. Step S33: Based on the Random Sample Consensus (RANSAC) algorithm, perform planar fitting on the point cloud data within the initial view frustum to identify and peel off the conveyor belt support surface and the background point cloud below it. Density-based spatial clustering (DBSCAN) is performed on the remaining point cloud after the support surface is removed. The cluster with the most points is selected as the target cargo box body, resulting in a coarse single-unit point cloud set. Step S34: Based on the inverse distance weighted spatial interpolation algorithm, perform elevation weighted calculation on the coarse single-unit point cloud set to fill the point cloud holes and obtain single-unit target point cloud data; Step S35: The single target point cloud data is sliced ​​vertically using the equidistant slicing method. The polygon area of ​​each slice is calculated using the Gaussian area formula, and the target cargo box volume data is calculated based on the height integral of each slice.

5. The method for measuring cargo box volume and allocating storage locations based on machine vision as described in claim 4, characterized in that, Step S34 specifically includes: The coarse point cloud set is projected onto a horizontal reference plane in a grid pattern. The point cloud density within each grid is calculated. When the point cloud density within a grid is lower than a set density threshold, the grid below the density threshold is marked as a void region. Based on the inverse distance weighted spatial interpolation algorithm, a set of known elevation points in the neighborhood surrounding the void region is extracted. The void region is then subjected to elevation weighted calculation using the 3D coordinates of the known elevation points to generate supplementary point cloud data. The elevation weighted calculation expression is as follows: in, The generated supplementary point cloud data represents the elevation coordinates within the cavity area, where W is the total number of points from the extracted known elevation point set. Given the elevation coordinates of the p-th point in a set of known elevation points, Let be the two-dimensional Euclidean distance from the p-th point in the set of known elevation points to the center point of the cavity region; The supplementary point cloud data is filled into the void area to obtain the point cloud data of a single target.

6. The method for measuring cargo box volume and allocating storage locations based on machine vision as described in claim 5, characterized in that, Step S35 specifically includes: The point cloud data of a single target is sliced ​​in layers along the vertical direction according to a set fixed height step by the equidistant slicing method, resulting in multiple parallel point cloud slice layers. The data points in each point cloud slice layer are projected onto a two-dimensional plane to obtain a two-dimensional point set. The boundary of the two-dimensional point set is extracted and the direction is sorted based on the convex hull algorithm to obtain an ordered closed outer contour point set. The polygon area of ​​the ordered closed outer contour point set is calculated using the Gaussian area formula to obtain the polygon area value of each slice. The polygon area values ​​of all slice layers are multiplied, accumulated, and integrated with the fixed height step size. The expression for calculating the polygon area is as follows: Where M represents the total number of contour points in the sorted closed outer contour point set corresponding to the k-th layer point cloud slice data. These are the x and y coordinates of the j-th contour point in the sorted closed outer contour point set corresponding to the k-th layer point cloud slice data on the two-dimensional projection plane. The target cargo box volume data is obtained by multiplying and integrating the polygon area values ​​of all slice layers with a fixed height step size. The expression for the multiplication and integration is as follows: in, For target cargo container volume data, The set fixed height step size, N is the total number of point cloud slice layer data.

7. The method for measuring cargo box volume and allocating storage locations based on machine vision as described in claim 6, characterized in that, Step S4 establishes a virtual bounding box based on the target cargo container volume data, specifically including: Based on the target cargo box volume data and combined with the minimum circumscribed cuboid parameter of the target cargo box in three-dimensional space, a virtual bounding box adapted to the geometric dimensions of the target cargo box is constructed. The pose information of the virtual bounding box in the three-dimensional spatial coordinate system includes the coordinates of the center point of the bounding box, the length of the long side, the length of the short side, the height, and the rotation azimuth angle around the vertical axis. The virtual bounding box is used to replace the physical entity of the target cargo box in subsequent stacking array simulation and detection to simulate space occupancy.

8. The method for measuring cargo box volume and allocating storage locations based on machine vision as described in claim 7, characterized in that, The processing logic for performing array pattern deduction and detection in virtual space based on the virtual bounding box is as follows: Acquire current warehouse location status data, which includes a set of virtual bounding boxes of occupied locations, a set of coordinates of vacant locations, location load-bearing limit data, and shelf height constraint data; perform stacking simulation on candidate locations one by one in the virtual space based on the virtual bounding boxes, which includes collision detection and stability assessment. The collision detection processing logic is as follows: The virtual bounding box is placed at the candidate storage location coordinates according to a preset stacking posture, and it is determined whether the virtual bounding box has spatial interference with any virtual bounding box in the set of virtual bounding boxes of the occupied storage location; The processing logic for the stability assessment is as follows: Calculate the centroid projection point of the virtual bounding box at the candidate storage location, and determine whether the centroid projection point falls within the support surface range of the candidate storage location: When the collision detection result is no interference and the stability assessment result is that the center of gravity projection point falls within the support surface range, the candidate storage location is marked as a feasible storage location. From all feasible storage locations, the optimal storage location is selected according to the optimization objective, and the coordinate data of the target storage location is output.

9. The method for measuring cargo box volume and allocating storage locations based on machine vision as described in claim 8, characterized in that, The optimization objectives include maximizing space utilization, prioritizing the number of stacking layers, and prioritizing the shortest transport path. The three-dimensional spatial coordinate data corresponding to the selected optimal storage location is output as the target storage location coordinate data to the warehouse management system.

10. A machine vision-based system for measuring cargo box volume and allocating storage locations, applied in a machine vision-based method for measuring cargo box volume and allocating storage locations as described in any one of claims 1-9, characterized in that, It includes a data acquisition module, a target network detection module, a view frustum mapping module, and a storage location allocation module; The data acquisition module is used to simultaneously acquire cargo box image data and three-dimensional raw point cloud data; The target network detection module is used to process the cargo box image data through the RF-DETR target detection network model to obtain detection box data; The frustum mapping module is used to map the detection box data to a three-dimensional spatial coordinate system through an intrinsic and extrinsic parameter transformation matrix to obtain the three-dimensional frustum boundary. Based on the three-dimensional frustum boundary, it clips the individual target point cloud data from the three-dimensional original point cloud data and calculates the target cargo box volume data from the individual target point cloud data. The storage location allocation module is used to establish a virtual bounding box based on the target cargo box volume data, perform stacking array simulation and detection in virtual space based on the virtual bounding box, and output the target storage location coordinate data.

Images