A method and apparatus for locating a palletized load

Abstract

This invention relates to a method and apparatus for locating stacked goods. The method includes recognizing acquired image data, constructing a dynamic mesh based on the spatial distribution of an initial mask, performing local fine segmentation on missing meshes in the dynamic mesh to obtain a total mask set, optimizing the mask quality and removing anomalies based on the multidimensional matching degree of the dynamic mesh to obtain a correct mask set, and performing an inverse coordinate transformation on the correct mask set based on the original coordinate system of the image data to obtain a correct binary mask for locating the stacked goods. By recognizing the acquired image data and constructing a dynamic mesh, combined with mask quality optimization and anomaly removal mechanisms, the method effectively overcomes the segmentation noise problem caused by uneven lighting and goods occlusion in the warehouse environment, achieving high-precision positioning on low-computing-power devices and solving the problem of insufficient positioning accuracy of stacked goods on low-computing-power devices such as embedded platforms.

Description

Technical Field

[0001] This invention relates to the fields of smart logistics and image processing technology, and in particular to a method and apparatus for positioning stacked goods. Background Technology

[0002] In the process of logistics automation, precise positioning in stacking operations is a core element for achieving efficient warehouse management. Machine vision technology, with its non-contact measurement advantages, has been deeply applied to goods sorting, stacking operations, and end-to-end logistics management. With the integration of industrial robots and computer vision, deep learning-based target segmentation models, due to their high boundary accuracy and strong environmental adaptability, have become an important direction for upgrading stacking technology. The emergence of lightweight segmentation models such as FastSAM provides new ideas for resource-constrained scenarios, but still faces significant challenges in practical deployment.

[0003] Current industrial stacking systems generally rely on 3D vision positioning solutions, such as binocular vision and structured light measurement. While these methods can acquire depth information, they place stringent demands on hardware computing power, resulting in high equipment costs and complex systems. Especially in small and medium-sized stacking equipment scenarios, the limited computing power of embedded platforms makes it difficult to support the real-time processing requirements of 3D vision. Furthermore, warehouse environments typically present complex factors such as uneven lighting, occlusion from stacked goods, and background interference, making existing segmentation models prone to mask breaks, blurred edges, or missegmentation during recognition. While the FastSAM model is lightweight, it is not optimized for the spatial distribution of goods in stacking scenarios, resulting in initial segmentation results containing a large number of noisy regions, failing to directly meet positioning accuracy requirements. In addition, the lack of dynamic mesh construction and mask quality verification further exacerbates the problem of insufficient recognition reliability on computing-constrained equipment, leading to decreased stacking operation efficiency and increased operational risks.

[0004] Therefore, there is an urgent need to propose a method and device for positioning stacked goods to solve the technical problem that the existing stacking methods have insufficient positioning accuracy on low computing power devices such as embedded platforms and in complex environments. Summary of the Invention

[0005] In view of this, it is necessary to provide a stacked goods positioning method and apparatus to solve the technical problem of insufficient positioning accuracy of stacked goods in existing goods stacking methods on low computing power devices such as embedded platforms and in complex environments.

[0006] To address the aforementioned problems, in a first aspect, the present invention provides a method for positioning stacked goods, comprising:

[0007] The acquired image data is identified to obtain an initial mask, and a dynamic mesh is constructed based on the spatial distribution pattern of the initial mask. The missing meshes in the dynamic mesh are then locally refined to obtain the total mask set.

[0008] Based on the multidimensional matching degree of the dynamic mesh, each mask in the total mask set is optimized for mask quality and anomalies are removed to obtain the correct mask set.

[0009] Based on the original coordinate system of the image data, the correct mask set is subjected to an inverse coordinate transformation to obtain a correct binary mask, and the stacked goods are positioned according to the correct binary mask.

[0010] In one possible implementation, a dynamic mesh is constructed based on the spatial distribution pattern of the initial mask, and the missing meshes in the dynamic mesh are locally refined to obtain a total mask set, including:

[0011] The initial mask is filtered and its angle is corrected to obtain the target mask;

[0012] Based on the spatial distribution pattern of the target mask, a dynamic mesh is generated;

[0013] The missing meshes in the dynamic mesh are locally refined to obtain a supplementary mask;

[0014] Based on the initial mask and the supplementary mask, a total mask set is obtained.

[0015] In one possible implementation, the process of optimizing the mask quality and removing anomalies for each mask in the total mask set based on the multidimensional matching degree of the dynamic mesh to obtain a correct mask set includes:

[0016] The total mask set is filtered based on the calculation results of the multidimensional matching degree between each mask in the total mask set and the dynamic mesh to obtain the first mask set;

[0017] Based on the number of pixels intersecting between each mask in the first mask set and the dynamic grid, the fill rate is obtained, and masks in the first mask set whose fill rate is less than a preset fill rate are removed to obtain a second mask set.

[0018] Based on a preset cross-shaped structural element, each mask in the second mask set is expanded to obtain the expansion result. Then, the expansion result is eroded to obtain the third mask set.

[0019] The third mask set is subjected to a neighborhood consistency check to obtain the correct mask set.

[0020] In one possible implementation, the step of filtering and angle-correcting the initial mask to obtain the target mask includes:

[0021] The first initial mask is obtained by retaining the masks whose geometric features are within the preset range of the standard proportion of the cargo in the initial mask.

[0022] Based on the edge extraction algorithm, the cargo edge features of the first initial mask in the image data are extracted to obtain the vertical edge and the horizontal edge;

[0023] The dominant orientation angles of the vertical and horizontal edges are determined based on the Hough transform.

[0024] The target mask is obtained by rotating the first initial mask using the dominant direction angle.

[0025] In one possible implementation, generating a dynamic mesh based on the spatial distribution pattern of the target mask includes:

[0026] The coordinates of the centroid of each mask in the target mask are calculated to obtain the coordinates of all mask centroids;

[0027] Arrange all the centroid coordinates of the masks in ascending order in both the horizontal and vertical directions to obtain an ordered sequence.

[0028] The distances between adjacent mask centroids in the ordered sequence in the horizontal and vertical directions are calculated to obtain a distance set;

[0029] The mode of the distance set is taken as the most probable nominal horizontal distance and nominal vertical distance;

[0030] Using all the centroids of the target mask as initial seed points, the mesh is expanded along the horizontal and vertical directions with the nominal horizontal distance and the nominal vertical distance as step sizes, respectively, to construct a dynamic mesh covering the area of ​​the stacked goods.

[0031] In one possible implementation, the step of locally refining the missing mesh in the dynamic mesh to obtain a supplementary mask includes:

[0032] A KD tree is constructed based on all the centroids of the masks;

[0033] Based on the KD tree, determine the area-weighted Euclidean distance from the center point of the dynamic mesh to the centroids of multiple nearest neighbors of the mask, and the area of ​​each nearest neighbor of the mask centroid;

[0034] A dynamic threshold is determined based on the area. When the product-weighted Euclidean distance is greater than the dynamic threshold, the network at the corresponding grid center point is determined to be a missing mask region.

[0035] The missing mask region is identified as a local region of interest and subjected to local fine segmentation to obtain a supplementary mask.

[0036] In one possible implementation, the step of filtering the total mask set based on the calculation results of the multidimensional matching degree between each mask in the total mask set and the dynamic mesh to obtain a first mask set includes:

[0037] Masks whose centroids are not within the extended coordinate range of the dynamic mesh are removed from the total mask set to obtain the retained mask set.

[0038] The matching degree between each mask in the reserved mask set and the dynamic mesh is calculated to obtain the overall matching degree;

[0039] The masks in the retained mask set whose multidimensional matching degree is less than the preset matching degree are deleted to obtain the first mask set.

[0040] In one possible implementation, performing a neighborhood consistency check on the third mask set to obtain a correct mask set includes:

[0041] Determine the width and height of the minimum bounding rectangle of each mask in the third mask set, as well as the mean and standard deviation of the width and height of the mask dimensions in adjacent grids;

[0042] The size dynamic threshold is obtained based on the mean and standard deviation.

[0043] The width and height deviation rate is obtained by calculating the width and height deviation of each mask based on the average of the width and height.

[0044] When the width-to-height deviation rate is not greater than the size dynamic threshold, the corresponding mask area is determined as a local region of interest.

[0045] The local region of interest is segmented using the FastSAM model to obtain a new mask, and the cross-union ratio (CUI) between the new mask and the corresponding mask is determined.

[0046] When the cross-union ratio is less than the preset cross-union ratio, the corresponding mask in the third mask set is deleted to obtain the correct mask set.

[0047] In one possible implementation, positioning the stacked goods according to the correct binary mask includes:

[0048] The centroid of the target mask is inversely transformed and calculated to obtain the correct binary mask in the original coordinate system;

[0049] The centroid of the target mask is inversely transformed to obtain the predicted centroid value;

[0050] Determine whether the Euclidean distance between the centroid prediction value and the centroid coordinates of the image data is less than or equal to a preset pixel;

[0051] If so, the coordinates of the stacked goods in the image data are determined according to the correct binary mask, and the stacked goods are located according to the coordinates.

[0052] Secondly, the present invention also provides a stacked goods positioning device, comprising:

[0053] The image acquisition module is used to acquire image data of stacked goods;

[0054] The network construction module is used to identify the acquired image data, obtain an initial mask, construct a dynamic mesh based on the spatial distribution pattern of the initial mask, and perform local fine segmentation on the missing meshes in the dynamic mesh to obtain the total mask set.

[0055] An anomaly removal module is used to optimize the mask quality and remove anomalies for each mask in the total mask set based on the multidimensional matching degree of the dynamic mesh, so as to obtain the correct mask set.

[0056] The cargo positioning module is used to perform an inverse coordinate transformation on the correct mask set based on the original coordinate system of the image data to obtain a correct binary mask, and to position the stacked cargo according to the correct binary mask.

[0057] The beneficial effects of this invention are as follows: The acquired image data is identified to obtain an initial mask; a dynamic mesh is constructed based on the spatial distribution pattern of the initial mask; and the missing meshes in the dynamic mesh are locally refined to obtain a total mask set. Based on the multidimensional matching degree of the dynamic mesh, mask quality optimization and anomaly removal are performed on each mask in the total mask set to obtain a correct mask set. Based on the original coordinate system of the image data, the correct mask set undergoes an inverse coordinate system transformation to obtain a correct binary mask, and the stacked goods are located according to the correct binary mask. By identifying the acquired image data and constructing a dynamic mesh, combined with mask quality optimization and anomaly removal mechanisms, the segmentation noise problem caused by uneven lighting and goods occlusion in the warehouse environment is effectively overcome. This achieves high-precision positioning on low-computing-power devices, solving the problem of insufficient positioning accuracy for stacked goods on low-computing-power devices such as embedded platforms, improving positioning accuracy and environmental adaptability, while reducing system complexity and hardware costs. Attached Figure Description

[0058] Figure 1 A schematic flowchart of an embodiment of the stacked goods positioning method provided by the present invention;

[0059] Figure 2 A schematic diagram of an embodiment of the image data of stacked goods provided by the present invention;

[0060] Figure 3 For the present invention Figure 1 A schematic diagram of an embodiment of step S102;

[0061] Figure 4 For the present invention Figure 1 A schematic flowchart of an embodiment of step S103;

[0062] Figure 5 This is a schematic diagram of an embodiment of the stacked goods positioning device provided by the present invention. Detailed Implementation

[0063] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which constitute a part of the present invention and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0064] like Figure 1 As shown, a specific embodiment of the present invention discloses a method for positioning stacked goods, comprising:

[0065] S101. Recognize the acquired image data to obtain an initial mask, construct a dynamic mesh based on the spatial distribution pattern of the initial mask, and perform local fine segmentation on the missing meshes in the dynamic mesh to obtain the total mask set.

[0066] Image data acquisition refers to the process of extracting visual information from an image source. This can be achieved by using industrial cameras to capture image data in real time, such as by continuously capturing frames above a stacking area using a camera with a fixed focal length, or by manually capturing and transmitting image data using a mobile terminal device. The main purpose is to input basic visual information to support subsequent processing. Figure 2 As shown, Figure 2 This refers to image data of stacked goods, specifically RGB images of warehouse or shelf scenes captured using a camera module. Figure 2The prior knowledge that the stacked goods are regular rectangles arranged in a certain spatial order is used. A lightweight segmentation model can be called to process the acquired image data, generate an initial mask for the stacked goods, and identify the pixel regions corresponding to the goods in the image, providing a basis for subsequent preprocessing operations; local fine segmentation can target potential missing regions in the mesh. This invention combines the output of the lightweight segmentation model (i.e., the FastSAM model) with adaptive mesh processing through a dynamic mesh mechanism, avoiding the dependence of 3D vision solutions on dedicated hardware, thus adapting to the computing power limitations of small and medium-sized stacking equipment, and enhancing the recognition stability in complex environments through mesh framework verification.

[0067] S102. Based on the multidimensional matching degree of the dynamic mesh, perform mask quality optimization and anomaly removal on each mask in the total mask set to obtain the correct mask set.

[0068] S103. Based on the original coordinate system of the image data, perform an inverse coordinate transformation on the set of correct masks to obtain the correct binary mask, and locate the stacked goods according to the correct binary mask.

[0069] The inverse coordinate system transformation can be achieved by using an affine transformation matrix to map the optimized mask coordinates back to the original image coordinate system. For example, position restoration can be achieved by solving the coordinate transformation equation, or a linear transformation can be performed using pre-stored calibration parameters. The main purpose is to ensure consistency between the positioning results and the actual scene to support the execution of stacking operations.

[0070] Compared with existing technologies, this embodiment provides an approach where image data is acquired as the basic input source during the positioning of stacked goods. This method avoids the dependence of 3D vision solutions on dedicated hardware, significantly reducing system complexity and computational requirements, and is particularly suitable for the resource constraints of small and medium-sized stacking equipment. The acquired image data is identified, and an initial mask is generated. This lightweight model efficiently performs segmentation tasks under limited computing power, effectively alleviating real-time pressure in complex scenarios. Based on the spatial distribution pattern of the initial mask, a dynamic mesh is constructed. Its parameters are automatically derived based on the geometric characteristics of the actual goods arrangement, enabling the mesh to adapt to different stacking layouts and avoiding the rigidity problem of preset meshes, thus maintaining robustness in changing environments. When performing local fine-grained segmentation on missing meshes in the dynamic mesh, potential missing areas are identified and targeted supplementary segmentation is performed. This ensures the integrity of mask coverage while focusing only on local areas to minimize computational overhead, solving the problem of segmentation omissions caused by complex lighting or occlusion. The quality of each mask in the total mask set is optimized through the multi-dimensional matching degree of the dynamic mesh, and abnormal segmentation results are effectively eliminated, improving the reliability of the masks. The correct mask set is mapped back to the original image coordinate system through inverse coordinate system transformation, and the correct binary mask is generated. It is used for the accurate positioning of stacked goods, ensuring the consistency between the positioning results and the actual scene.

[0071] In some embodiments of the present invention, such as Figure 3 As shown, step S102 includes:

[0072] S301. Filter and correct the angle of the initial mask to obtain the target mask.

[0073] Among them, filtering refers to selecting the initial mask according to the preset geometric feature range. It can be achieved by using threshold judgment based on the mask aspect ratio, area or shape factor. The purpose is to remove abnormal masks caused by environmental interference and retain masks that conform to the standard shape of the goods. Angle correction refers to correcting the directional deviation of the mask. It can be achieved by using algorithms based on image edge feature extraction and rotation transformation. The purpose is to align the mask with the actual orientation of the goods and eliminate the influence of tilted view or improper placement.

[0074] S302. Generate a dynamic mesh based on the spatial distribution pattern of the target mask.

[0075] Among them, generating dynamic mesh refers to constructing a mesh structure based on the spatial distribution law of the target mask. It can be achieved by using statistical analysis methods based on the distribution of the centroid coordinates of the mask. The purpose is to accurately capture the regular spacing of the goods arrangement and avoid the coverage problem of fixed mesh.

[0076] S303. Perform local fine-grained segmentation on the missing mesh in the dynamic mesh to obtain a supplementary mask.

[0077] Local fine segmentation refers to the fine segmentation of a specific region of a dynamic grid. It can be achieved by using an adaptive segmentation algorithm based on the local region of interest. The purpose is to identify and supplement blind spots in grid coverage and reduce the computational cost of global re-segmentation.

[0078] S304. Based on the initial mask and the supplementary mask, obtain the total mask set.

[0079] The total mask set obtained in this embodiment of the invention refers to the integration of the initial mask and the supplementary mask, which can be achieved by mask merging or fusion operations. The purpose is to completely preserve the cargo outline information and provide high-fidelity input.

[0080] Specifically, the solution of this invention effectively solves the problem of mesh inaccuracy caused by noise interference and orientation deviation in the initial mask by systematically optimizing the initial mask quality and refining the dynamic mesh construction process. First, the filtering process selects geometric features based on a preset range of standard cargo proportions to eliminate abnormal masks and ensure that the retained masks closely resemble the actual cargo shape. Then, the angle correction determines the dominant direction angle based on edge features extracted from the image data and corrects the mask posture through rotation transformation, so that the corrected target mask is strictly aligned with the actual cargo orientation. Next, the dynamic mesh generation determines the nominal horizontal and vertical distances as the mesh step size by calculating the ordered sequence of mask centroid coordinates and the mode of adjacent distances, so that the mesh expansion closely matches the actual spacing of the cargo distribution. Then, the local refined segmentation identifies missing areas based on the spatial correlation between the mesh center point and the mask centroid, and performs local adaptive segmentation for blind spots in mesh coverage. Finally, the total mask set is obtained by integrating the filtered and corrected main mask and the locally refined supplemented detail masks, completely preserving the key information of the cargo outline and providing a high-precision foundation for subsequent mask quality optimization.

[0081] As a preferred embodiment, the present invention is implemented as follows: In the filtering stage, masks whose geometric features conform to the proportion range of common goods can be retained, for example, masks with stable shapes can be selected based on aspect ratio thresholds; in the angle correction stage, the dominant directions of vertical and horizontal edges can be determined based on edge extraction algorithms, and the mask pose can be corrected by rotation transformation; in the dynamic mesh generation stage, an ordered sequence of centroid coordinates of all masks can be calculated, and the mode of adjacent distances can be used as the step size for mesh expansion; in the local fine segmentation stage, regions with missing masks in the dynamic mesh can be identified as local regions of interest, and supplementary masks can be obtained by local resegmentation using a lightweight segmentation model; finally, the initial mask and the supplementary mask are merged to form a total mask set.

[0082] In some embodiments of the present invention, step S301 includes:

[0083] The first initial mask is obtained by retaining the masks whose geometric features are within the preset range of the standard proportion of the goods in the initial mask.

[0084] The setting of geometric features within the preset range of the standard cargo ratio refers to a filtering threshold range set based on the typical aspect ratio of stacked goods. This can be achieved using aspect ratio threshold ranges or area ratio constraints. The purpose is to eliminate abnormal areas caused by cluttered backgrounds or segmentation errors, ensuring that only valid cargo areas are retained. For example, if the aspect ratio of the mask needs to be controlled within the standard cargo ratio... Within the fluctuation range, the mask that conforms to the geometric features is retained to obtain the first initial mask, thereby initially filtering out erroneous segmentation results caused by environmental noise.

[0085] Based on the edge extraction algorithm, the edge features of the cargo are extracted from the first initial mask in the image data to obtain the vertical and horizontal edges.

[0086] Edge extraction algorithms can be understood as technical methods for detecting object boundaries from image data. They can be implemented using Canny, Sobel, or Laplacian operators. The goal is to focus on local edge detection within the filtered mask to avoid redundant calculations in full-image processing.

[0087] The dominant orientation angles of vertical and horizontal edges are determined based on the Hough transform.

[0088] The Hough transform is a mathematical method for detecting lines by mapping edge points in image space to parameter space. It can be implemented using an accumulator array or a variant of the probabilistic Hough transform. Its purpose is to suppress local interference and stably identify the direction of the dominant line through the parameter space accumulation mechanism.

[0089] In this embodiment of the invention, the dominant direction angles of the vertical and horizontal edge lines are obtained using the Hough transform. It is defined as the edge direction with the largest cumulative number of votes, as shown in formula (1):

[0090] (1)

[0091] (2)

[0092] In the formula, For the set of horizontal edge points, δ ( · ) is the Dirac function, only when The value is 1 if the condition is met, and 0 otherwise. This process essentially involves parameterizing edge points. The voting mechanism will ultimately select the candidate with the most votes. Use it as the dominant directional angle for the entire operation.

[0093] The target mask is obtained by rotating the first initial mask using the dominant orientation angle.

[0094] Among them, the guide direction angle can be understood as a key parameter characterizing the overall tilt angle of the cargo. It can be achieved by peak extraction in parameter space or weighted average calculation, with the aim of quantifying the directional characteristics of the cargo boundary. In practical applications, rotation transformation refers to the operation of eliminating directional deviation based on coordinate transformation. It can be achieved by using affine transformation matrix or bilinear interpolation algorithm, with the aim of orthogonally aligning the mask with the image coordinate system to ensure the accuracy of dynamic mesh construction.

[0095] According to the estimated results of the embodiments of the present invention The first initial mask in the image is rotated so that the principal axis of the goods in the image is aligned with the image coordinate system, thereby simplifying the subsequent calculations. The rotation model is shown in formula (3):

[0096] (3)

[0097] This transformation will affect all pixels in the original image. Mapped to the corrected coordinate system This ensures that the orientation of goods is consistent, providing structurally consistent input data for subsequent positioning and segmentation tasks.

[0098] Specifically, the present invention forms a complete technical chain through a dual mechanism of proportional filtering and angle correction: First, the initial mask is screened based on the geometric proportions of the cargo, effectively eliminating noise areas that do not conform to the standard proportions, ensuring that the first initial mask only contains valid cargo areas; on this basis, local edge feature extraction is performed on the filtered mask areas to avoid the computational cost of full-image processing and accurately obtain edge information in the vertical and horizontal directions; further, Hough transform is used to perform robustness analysis on the extracted edge data, ensuring that the dominant direction angle can still be stably determined even under conditions of partial edge loss or noise interference; finally, a rotation transformation is performed on the dominant direction angle to directly eliminate the directional deviation caused by cargo tilt, making the mask coordinate system orthogonally aligned with the image coordinate system. This technical solution, through the strict timing coordination and information flow of each step, ensures that the dynamic mesh construction is based on an accurate spatial distribution, thereby solving the problems of initial mask noise interference and directional deviation.

[0099] In complex stacking scenarios, the FastSAM model may miss segmenting some goods due to occlusion, uneven lighting, or similar textures, resulting in mask gaps. In some embodiments of the present invention, step S302 includes:

[0100] The coordinates of the centroid of each mask in the target mask are calculated to obtain the coordinates of all mask centroids.

[0101] Among them, the centroid coordinates of the mask refer to the coordinates of the geometric center point of the mask area. They can be achieved by image processing algorithms such as centroid calculation or centroid detection, with the aim of accurately obtaining the position reference of each item in the image.

[0102] This invention generates an adaptive dynamic mesh based on the spatial distribution pattern of the extracted masks, specifically by setting an existing set of masks. ,in For the first A mask in pixels The values ​​at the specified locations are used to calculate the centroid coordinates of the mask as shown in formula (4):

[0103] (4)

[0104] Arrange all the centroid coordinates of the masks in ascending order in both the horizontal and vertical directions to obtain an ordered sequence.

[0105] In this embodiment of the invention, all centroids are respectively arranged in the horizontal direction and vertical direction Arrange in ascending order to obtain an ordered sequence, as shown in formula (5):

[0106] (5)

[0107] In the formula, For sorting indexes.

[0108] The distances between the centroids of adjacent masks in the ordered sequence in the horizontal and vertical directions are calculated to obtain a distance set.

[0109] The mode of the distance set is taken as the most probable nominal horizontal distance and nominal vertical distance.

[0110] The mode refers to the value that appears most frequently in the distance set. It can be determined by histogram statistics or frequency analysis. The purpose is to use statistical robustness to filter out abnormal distance values ​​and ensure that the nominal distance reflects the most common cargo spacing.

[0111] Among them, grid expansion can be understood as generating grid lines with a fixed step size starting from the seed point. Specifically, it can be done by iterative expansion or recursive filling. Its purpose is to make the grid construction process closely match the actual cargo distribution and avoid insufficient coverage or overlap.

[0112] In this embodiment of the invention, the distances between the centroids of adjacent masks in the horizontal and vertical directions are calculated to obtain a distance set. The mode of the distance set is then taken as the most probable nominal horizontal and vertical distances. Because when stacking is arranged in a regular pattern, the centroid distances of most adjacent goods are the same, and the number of times the same distance occurs is far greater than the number of abnormal distances caused by obstruction, noise, etc., all modes can reflect the majority consistency, and their calculation is shown in formula (6):

[0113] (6)

[0114] in, The mode of a set, The nominal horizontal distance This is the nominal vertical distance.

[0115] Using all the centroids of the target mask as initial seed points, the mesh is expanded along the horizontal and vertical directions with nominal horizontal and vertical distances as steps, respectively, to construct a dynamic mesh covering the area of ​​the stacked goods.

[0116] In this embodiment of the invention, all the centroids of the target mask are used as initial seed points, and the mesh is expanded along the horizontal and vertical directions with nominal horizontal and vertical distances as steps, respectively, to construct a dynamic mesh covering the area of ​​the stacked goods. .

[0117] Specifically, the present invention provides a real position reference for mesh construction by accurately calculating the centroid coordinates of the target mask and obtaining all coordinate data; after converting the disordered centroid positions into an ordered sequence in the horizontal and vertical directions, the distances between adjacent centroids are systematically calculated to obtain a distance set; the nominal horizontal distance and nominal vertical distance are determined based on the mode statistics, effectively filtering outliers to adapt to local gaps or offsets; finally, with all mask centroids as initial seed points, the mesh is expanded in both directions with the nominal distance as the step size, so that the dynamic mesh construction process closely follows the actual cargo distribution pattern, thereby achieving precise coverage of the stacking area.

[0118] In some embodiments of the present invention, step S303 includes:

[0119] A KD tree is constructed based on all mask centroids.

[0120] Among them, constructing a KD tree based on all mask centroids refers to organizing the spatial coordinate data of the mask centroids into a binary spatial partitioning tree structure, which can be implemented by recursive median partitioning or random projection tree. The purpose is to establish an efficient spatial indexing mechanism, accelerate the subsequent nearest neighbor query process, and avoid the high computational overhead caused by global traversal.

[0121] Based on the KD tree, determine the area-weighted Euclidean distance from the center point of the dynamic mesh to the centroids of multiple nearest neighbors of the mask, as well as the area of ​​each nearest neighbor of the mask centroid.

[0122] Among them, the area-weighted Euclidean distance from the center point of the dynamic grid to the centroids of multiple nearest neighbors based on the KD tree refers to the introduction of the mask area as a weighting factor when calculating the geometric distance after quickly retrieving the nearest neighbors through the KD tree. It can be understood as multiplying the distance value by the inverse of the area or using an area-weighted average function. The purpose is to make the distance measurement more in line with the actual spatial distribution of goods, and to give higher spatial weights to larger areas of goods to reflect their dominance.

[0123] The embodiments of the present invention include the centroids of all masks. Construct a KD tree and calculate the grid center point. The K nearest neighbor mask centroids are found using a KD-tree. The value of K is determined based on the stacking density: 3-5 for typical stacking scenarios (compact cargo arrangement, no severe obstruction), and dynamically adjusted to 2 when there are fewer effective masks at the stacking edge. This is denoted as a set. The area-weighted Euclidean distance from the center point of the grid to its K nearest centroids is calculated as shown in formula (7):

[0124] (7)

[0125] In the formula, For the first m The area of ​​the nearest neighbor of the mask centroid. Based on the local normalized weighting factors of the current K nearest neighbor masks, ensure that the sum of all weight coefficients is 1 to avoid bias caused by global statistics.

[0126] The dynamic threshold is determined based on the area. When the product-weighted Euclidean distance is greater than the dynamic threshold, the network at the corresponding grid center point is determined as the missing mask region.

[0127] In this embodiment of the invention, the dynamic threshold T is calculated as shown in formula (8):

[0128] (8)

[0129] when Greater than Then mark with ( The grid with the center point of the grid is the missing mask area.

[0130] The missing mask area is identified as a local region of interest and subjected to local fine segmentation to obtain a supplementary mask.

[0131] In this embodiment of the invention, for each missing mask region, a local region of interest (ROI) is defined centered on it for fine segmentation. The ROI is shown in formula (9):

[0132] (9)

[0133] In the formula, This is the original image resolution.

[0134] Within this ROI, the FastSAM model is invoked again for refined segmentation, retaining the supplementary mask obtained from the secondary segmentation, and the initial mask and the supplementary mask are combined to form the set of masks to be processed. .

[0135] Specifically, the solution of this invention first constructs a KD-tree based on the centroids of all masks to establish an efficient spatial index structure, enabling nearest neighbor queries of the dynamic grid center point to avoid the high overhead of global traversal. Then, based on the KD-tree, it determines the area-weighted Euclidean distance, associating the geometric distance with the cargo size, assigning higher weights to masks with larger areas to quantify the spatial association strength. Next, it determines a dynamic threshold based on the area, adaptively adjusting the threshold boundary using the mask area to ensure a reasonable tolerance range for cargo of different sizes. Then, when the area-weighted Euclidean distance is greater than the dynamic threshold, it identifies missing mask regions. Through a dual determination mechanism of weighted distance and dynamic threshold, it accurately filters noise interference and identifies the true missing regions. Finally, it only treats the missing mask regions as regions of local interest for fine-grained segmentation, significantly reducing the computational burden. This sequence of steps ensures that, under limited computing power, it can accurately identify missing regions and efficiently generate supplementary masks, thereby effectively solving the problem of decreased positioning accuracy caused by cargo occlusion or deformation in dynamic grids.

[0136] In some embodiments of the present invention, such as Figure 4 As shown, step S103 includes:

[0137] S401. Based on the calculation results of the multidimensional matching degree between each mask in the total mask set and the dynamic mesh, the total mask set is filtered to obtain the first mask set.

[0138] Among them, the multidimensional matching degree (i.e., the comprehensive matching degree) refers to the quantitative index of the degree of matching between the mask and the dynamic mesh in spatial position. It can be achieved by the degree of coincidence between the mask centroid and the mesh coordinate range or the geometric coincidence between the mask boundary and the mesh line. The purpose is to screen out masks with reasonable spatial position and avoid invalid masks caused by centroid offset from entering the subsequent process.

[0139] In some embodiments of the present invention, step S401 includes:

[0140] Masks whose centroids are not within the extended coordinate range of the dynamic mesh are removed from the total mask set to obtain the retained mask set.

[0141] Among them, the centroid of the mask refers to the geometric center point of the mask area, which can be determined by the centroid coordinate calculation algorithm. The purpose is to quickly locate the core position of the mask and avoid redundant calculations for abnormal masks that are significantly deviated from the reasonable area of ​​cargo stacking. The extended coordinate range of the dynamic mesh can be understood as the coordinate boundary area constructed based on the spatial distribution law of the target mask. It can be implemented by the coordinate extension method based on the centroid of the target mask. The purpose is to adaptively define the reasonable geometric constraint range of cargo stacking.

[0142] In this embodiment of the invention, if the centroid does not fall into any dynamic mesh... Extended coordinate range If the mask is found to be an isolated mask generated during supplementary segmentation, it is deleted, resulting in a set of retained masks. These are the width and height of the grid, respectively. Mesh boundary spread factor (usually set to 1) ), used to define the allowable offset tolerance of the centroid.

[0143] The matching degree between each mask in the retained mask set and the dynamic mesh is calculated to obtain the overall matching degree.

[0144] Among them, the comprehensive matching degree refers to the degree of conformity between the quantized mask and the dynamic mesh in terms of position and shape. It can be calculated by weighted combination of position deviation and shape similarity. The purpose is to transform the spatial distribution pattern into a quantifiable screening criterion.

[0145] In this embodiment of the invention, if the centroid falls into a certain grid The mask is used to calculate its overall fit with the mesh. As shown in formula (10):

[0146] (10)

[0147] In the formula, Centroid offset is used to measure the degree of misalignment between the mask center and the mesh center. The smaller the value, the closer the mask center is to the mesh center, as shown in formula (11):

[0148] (11)

[0149] In the formula, Size matching is used to measure the area consistency between the mask and the mesh. The closer the value is to 1, the better the mask size matches the mesh size, as shown in formula (12):

[0150] (12)

[0151] In the formula, , for the area of ​​a single grid cell formed The area formed by the mask for a single cargo. These are the weighting coefficients. The larger the value, the more significant the impact of positional deviation on the results in the overall matching degree calculation.

[0152] Delete masks in the retained mask set whose overall matching degree is less than the preset matching degree to obtain the first mask set.

[0153] The preset matching degree can be understood as a dynamic threshold used to filter masks. It can be adjusted according to the complexity of the stacking scene in order to flexibly deal with complex lighting or occlusion conditions and avoid misjudgment caused by a single standard.

[0154] According to the principle of "position priority", the embodiments of the present invention are set A value of 0.8 makes the weight of the centroid offset higher than that of the size matching degree. If If the mask is determined to be a low-match mask, it is deleted to obtain the first mask set.

[0155] Specifically, the present invention achieves optimized evaluation of mask quality through a phased screening mechanism. First, masks whose centroids are not within the extended coordinate range of the dynamic grid are removed from the total mask set. This step quickly eliminates obviously abnormal masks based on the spatial distribution characteristics of the dynamic grid, reducing subsequent computation. Second, the matching degree between each mask in the retained mask set and the dynamic grid is calculated. This process quantifies the alignment between the mask boundary and the grid lines, transforming spatial distribution patterns into calculable numerical indicators. Finally, masks with a comprehensive matching degree less than a preset matching degree in the retained mask set are deleted. A refined screening is achieved by introducing an adjustable threshold mechanism. These steps form a progressive screening logic, with each stage closely linked: the construction of the extended coordinate range directly depends on the spatial distribution pattern of the target mask, ensuring that the screening benchmark is consistent with the actual arrangement pattern of the goods stack; the matching degree calculation is closely related to the nominal distance parameter of the dynamic grid, improving the targeting of the evaluation; and the threshold screening is dynamically adjusted based on scene complexity, enhancing environmental adaptability. Overall, this mechanism effectively solves the screening error problem caused by mask centroid drift and inaccurate dynamic mesh coverage while reducing computational complexity.

[0156] S402. Based on the number of intersection pixels between each mask in the first mask set and the dynamic grid, the fill rate is obtained, and masks in the first mask set with fill rates less than the preset fill rate are removed to obtain the second mask set.

[0157] The fill rate is a metric for measuring the integrity of the mask's coverage of the dynamic grid area. It can be achieved by using the ratio of the number of intersection pixels to the total number of pixels in the grid area or the normalized ratio of the mask area to the grid cell area. The purpose is to eliminate abnormal masks that are incompletely covered or too small in size, and to ensure the geometric integrity of the mask.

[0158] This step in this embodiment of the invention is morphological consistency verification. The core objective is to accurately remove mis-segmented cargo fragments, background noise, or non-cargo masks with abnormal shapes through fill rate verification and edge alignment verification, ensuring that the retained masks conform to the spatial morphology of real cargo. This is for the first set of retained masks. Statistical analysis of its corresponding grid The fill rate is obtained by dividing the number of intersecting pixels by the total number of pixels in the grid, as shown in formula (13):

[0159] (13)

[0160] The fill rate threshold was calibrated to 0.6 through multiple stacking experiments. If the fill rate is less than 0.6, it is considered that the mask coverage is insufficient, which may be due to missegmented local fragments of goods or background noise. The mask is marked as an incorrect mask and deleted to obtain the second mask set.

[0161] S403. Based on the preset cross-shaped structural element, each mask in the second mask set is expanded to obtain the expansion result, and then the expansion result is eroded to obtain the third mask set.

[0162] Among them, the expansion and erosion operation based on the preset cross structural element refers to the processing mechanism of repairing the continuity of the mask edge through morphological transformation. It can be implemented by using standard-sized cross-shaped structural elements or ring structural elements. The purpose is to fill the small gaps at the mask edge through expansion operation, and remove isolated noise points through erosion operation, so that the mask boundary fits the actual cargo outline better.

[0163] Although the mask in this embodiment of the invention, after filling rate verification, covers a sufficiently large mesh area, there may still be issues with the mask shape not conforming to a regular shape. For example, damaged packaging may cause internal voids during segmentation, or debris within the mesh may be mistakenly segmented as protruding parts of the mask. Therefore, further verification of the mask's edge alignment is required. (The last sentence appears to be incomplete and possibly refers to a different concept: "Let the cross-shaped structural element...") ,use For a single mask Dilation is performed, causing the foreground pixels of the mask to expand into the inward recessed areas, filling small depressions and holes. Then the same process is repeated. The expansion result is eroded to shrink the mask edge to the original body contour, eliminating the excessive edge expansion caused by expansion and preserving the complete internal shape after filling. To address the unevenness or concavity at the mask edges caused by mis-segmentation of debris or damage to cargo corners, an adaptive cross-shaped structural element opening operation is employed, allowing... = use right An erosion-expansion process is performed first. This process results in a more regular mask morphology, leading to a third mask set, which provides a high-quality foundation for subsequent geometric calculations.

[0164] S404. Perform neighborhood consistency verification on the third mask set to obtain the correct mask set.

[0165] Among them, neighborhood consistency verification refers to the verification mechanism that verifies the consistency between the mask size and the mask size of adjacent grids. It can be achieved by comparing the statistical deviation between the aspect ratio and the mean of the neighborhood or by the relative change rate between the mask area and the average area of ​​the local area. The purpose is to ensure that the mask size conforms to the spatial distribution law of stacked goods and to eliminate abnormal masks with sudden size changes.

[0166] Specifically, the present invention improves mask quality through a phased quality optimization mechanism. First, based on the overall matching degree, masks are screened to ensure they are within a reasonable coordinate range of the dynamic mesh, eliminating masks with abnormal centroid positions. Second, masks with incomplete coverage are removed based on the fill rate, ensuring sufficient coverage of the mesh area. Next, morphological dilation and erosion operations are used to restore the continuity of mask edges and eliminate noise interference. Finally, neighborhood consistency verification verifies the consistency of mask dimensions with adjacent masks, eliminating abnormal masks with abrupt size changes. These steps are executed sequentially, forming a complete quality optimization chain, ensuring that the final set of correct masks has high accuracy and high reliability.

[0167] In some embodiments of the present invention, step S404 includes:

[0168] Determine the width and height of the minimum bounding rectangle of each mask in the third mask set, as well as the mean and standard deviation of the width and height of the mask dimensions in adjacent grids.

[0169] Among them, the minimum bounding rectangle refers to the smallest rectangle that tightly surrounds the mask area. It can be implemented using a rotating caliper algorithm or a boundary detection function based on an image processing library. Its purpose is to accurately quantify the geometric size characteristics of the mask. The mean and standard deviation can be understood as the statistical distribution characteristics of the mask size in adjacent grids. They can be obtained by calculating the average value and dispersion of the local area. Their purpose is to provide an environmentally adaptive benchmark for size verification.

[0170] In this embodiment of the invention, the width and height of its minimum bounding rectangle are extracted. The mask dimensions in the neighboring grid are statistically compared. The mean values ​​of the neighborhood width and height are calculated. , and standard deviation , .

[0171] The size dynamic threshold is obtained based on the mean and standard deviation.

[0172] Specifically, the dynamic size threshold is a threshold dynamically determined based on statistical characteristics, such as being expressed as the mean plus or minus. k The standard deviation is set at 1000 times, which aims to adapt to the dimensional variation patterns in different stacking scenarios.

[0173] In this embodiment of the invention, a dynamic threshold is constructed. and As shown in formula (14):

[0174] (14)

[0175] The width and height deviation rate is obtained by calculating the width and height deviation of each mask based on the average width and height.

[0176] Among them, the width and height deviation rate refers to the relative deviation between the size of a single mask and the average size. It can be calculated as (actual width - average width) / average width and (actual height - average height) / average height. Its purpose is to objectively measure the degree of size anomaly.

[0177] In this embodiment of the invention, the width and height deviation rate is calculated as shown in formula (15):

[0178] , (15)

[0179] When the width and height deviation rate is not greater than the size dynamic threshold, the corresponding mask area is determined as the local region of interest.

[0180] Among them, the local region of interest refers to the masked region identified as a potential anomaly. It can be defined as the region where the width and height deviation rate exceeds a threshold. Its purpose is to focus on the key problem area to reduce computational overhead.

[0181] In the embodiments of the present invention, if or If any one of these conditions is not met, then the corresponding mask region is determined as the local region of interest. For example, in formula (16):

[0182] (16)

[0183] In the formula, 1.2 times The expansion coefficient. Formula (16) is used in... The process of internally calling the FastSAM model for re-segmentation.

[0184] Furthermore, if or If the mask size is abnormal, a secondary verification is initiated.

[0185] The FastSAM model is used to segment the local region of interest to obtain a new mask, and the cross-union ratio between the new mask and the corresponding mask is determined.

[0186] FastSAM model segmentation refers to the re-segmentation of local regions using a lightweight segmentation model. It can use a pre-trained FastSAM model for inference, and its purpose is to verify the segmentation consistency of the original mask. The intersection-union ratio is the overlap measure between the new mask and the original mask. It can be calculated as the intersection area divided by the union area, and its purpose is to objectively evaluate the reliability of the segmentation results.

[0187] In the embodiments of the present invention The FastSAM model is called internally for re-segmentation, and the cross-union ratio (CUI) between the new mask and the original mask is calculated. .

[0188] When the cross-union ratio is less than the preset cross-union ratio, the corresponding mask in the third mask set is deleted to obtain the correct mask set.

[0189] The preset crossover ratio refers to a set threshold that can be determined based on experience or experimentation. Its purpose is to strictly determine the effectiveness of the mask.

[0190] In the embodiments of the present invention A smaller value indicates a larger difference between the original mask and the local resegmentation result. With a threshold of 0.5, masks with a threshold less than 0.5 are marked as incorrect masks and deleted, resulting in the final set of correct masks. .

[0191] Specifically, the solution of this invention first analyzes the size characteristics of each mask in the third mask set and the statistical distribution of adjacent meshes, dynamically setting a size verification threshold to identify potential abnormal regions. Then, it performs local resegmentation and consistency verification only on these regions, using the FastSAM model to resegment the local region of interest and calculate the intersection-union ratio (IU). Inconsistent masks are then eliminated based on a strict threshold. This mechanism avoids the rigidity of a globally fixed threshold, combining statistical adaptive verification with local verification to ensure that only critical regions are finely processed, thus forming a closed-loop mask quality optimization process that effectively solves the positioning deviation problem caused by size variations in complex environments.

[0192] To ensure that the coordinate system of the subsequent visual image is consistent with the world coordinate system of the industrial robot, the correct mask set needs to be inversely transformed back to the original image coordinate system to restore the true position of the mask in the original scene. In some embodiments of the present invention, step S104 includes:

[0193] The centroid of the target mask is calculated by performing an inverse transformation to obtain the correct binary mask in the original coordinate system.

[0194] Among them, the centroid of the mask refers to the geometric center point of the coordinates of all foreground pixels within the mask area. It can be achieved by calculating the weighted average of the pixel coordinates within the mask area. Its purpose is to provide a stable and representative position reference point. The inverse transformation calculation can be understood as the mathematical process of mapping the points after coordinate transformation back to the original coordinate system. It can be achieved by using the inverse operation of the affine transformation matrix. Its purpose is to eliminate the systematic errors introduced during the coordinate system transformation process.

[0195] Embodiment of the present invention. The dominant direction angle obtained using the horizontal edge. The resolution of the original image is The center coordinates of the original image are Because the above correction involves rotation around the image center without translation, the center coordinates of the corrected image are consistent with those of the original image, denoted as . ;set up For the corrected coordinate system mask The original coordinates of a foreground pixel. It is obtained through the inverse transformation of "translation-inverse rotation-inverse translation", and the inverse transformation is shown in formula (17):

[0196] (17)

[0197] This transformation maps each foreground pixel back to its theoretical position in the original image. Coordinates after the inverse transformation. The values ​​may be floating-point numbers and may exceed the boundaries of the original image, therefore further processing is required. The calculated floating-point coordinates... Round to the nearest integer coordinate. Then filter out all those that do not meet the requirements. or Invalid coordinates. All valid integer coordinates. The original image is labeled as foreground, and all other coordinates in the original image are labeled as background, thus reconstructing a correct binary mask consistent with the original scene spatial reference in the original coordinate system. .

[0198] The centroid of the target mask is inversely transformed to obtain the predicted centroid value.

[0199] In this embodiment of the invention, the centroid of the target mask is substituted into formula (17) for calculation, and the predicted value of the centroid in the original coordinate system can be obtained.

[0200] Determine whether the Euclidean distance between the predicted centroid value and the centroid coordinates of the image data is less than or equal to a preset pixel.

[0201] Among them, the centroid Euclidean distance specifically refers to the straight-line distance between two centroid points, which can be achieved by calculating the square root of the sum of squares of the coordinate differences. Its purpose is to quantify the degree of positional deviation during the positioning process. The preset pixel refers to the pre-set distance threshold parameter, which can be dynamically adjusted according to the accuracy requirements of the actual application scenario. Its purpose is to establish a quantitative standard for judging the reliability of the positioning result.

[0202] If so, the coordinates of the stacked goods in the image data are determined based on the correct binary mask, and the stacked goods are located based on the coordinates.

[0203] In this embodiment of the invention, if the centroid prediction value is the same as the Euclidean distance of the centroid through the original RGB image... When the preset number of pixels is reached, the inverse transformation is proven to be correct. The final output is the correct mask in the original image coordinate system, i.e., the correct binary mask. The mask is perfectly aligned with the original RGB image coordinates obtained in step S101. The coordinates of the stacked goods in the image data can be determined based on the correct binary mask. These coordinates are then labeled in the visual image coordinate system and transformed with the industrial robot world coordinate system, thereby enabling the positioning of the stacked goods.

[0204] Specifically, the present invention calculates the predicted centroid value in the original coordinate system by performing an inverse transformation on the centroid of the target mask. This predicted value is based on a filtered and angle-corrected target mask, effectively avoiding initial mask noise interference and ensuring that the predicted value closely matches the original image coordinate system, laying the foundation for subsequent verification. The system then determines whether the centroid Euclidean distance between the predicted centroid value and the centroid coordinates of the image data is less than or equal to a preset pixel. This operation uses the centroid coordinates of the image data as a true reference benchmark, quantifies the prediction deviation through Euclidean distance, and the preset pixel threshold design can adapt to different lighting and occlusion environments, dynamically filtering abnormal positioning points caused by coordinate system transformation or external interference. Only when the centroid Euclidean distance is less than or equal to the preset pixel does the system determine the coordinates of the stacked goods in the image data and perform positioning based on the correct binary mask. This conditional execution mechanism ensures that only verified and reliable positioning results are used in actual stacking operations, thereby effectively avoiding invalid operations.

[0205] Furthermore, if the centroid Euclidean distance is greater than a preset pixel (i.e., the centroid Euclidean distance...) If the value is less than a certain pixel, then a significant deviation is considered to exist, and an offset compensation operation needs to be performed.

[0206] The number of uniformly sampled feature points at the mask boundary is , No. The coordinates of the feature points in the original image are: .

[0207] No. The predicted coordinates of the feature points after the inverse transform are: The center coordinates of the original image are .

[0208] To quantify the deviation between the actual and predicted positions, the offset vector is defined as follows: .

[0209] Define the radial vector as .

[0210] This vector is used to establish the spatial relationship between the predicted point and the image center, and its orientation angle , which represents the angle between the radial vector and the positive x-axis of the image coordinate system.

[0211] In the compensation calculation stage, the least squares optimization method is used to estimate the residual rotation angle. The accuracy of the compensation is ensured by fitting the error between the actual offset and the theoretical rotation offset, and its estimation is shown in formula (18):

[0212] (18)

[0213] The numerator of the formula is weighted by the dot product of the offset vector and the theoretical rotation direction vector, combined with the magnitude of the radial vector, to quantify the correlation between the actual offset and the rotation trend. The denominator is normalized by the sum of the squares of the magnitudes of the radial vectors to suppress the influence of extreme values ​​on the estimation results and improve the stability of the rotation angle calculation.

[0214] The compensated coordinates are obtained through the following rotation transformation as shown in formula (19):

[0215] (19)

[0216] The correction process first translates the predicted point to a coordinate system with the image center as the origin, then eliminates the residual rotation deviation through a rotation matrix, and finally translates it back to the original coordinate system to ensure that the compensated coordinates are consistent with the original scene space reference.

[0217] Calculate the Euclidean distance between the compensated coordinates and the original coordinates. If the distance is ≤1.0 pixels, the compensation is considered effective; otherwise, the rotation center needs to be re-evaluated or more feature points need to be sampled for iterative optimization.

[0218] This invention addresses rapid positioning under computational constraints by constructing a stacked goods positioning architecture centered on RGB visual information and a lightweight segmentation model. Through initial mask prediction, dynamic spatial mesh construction, and multi-stage refined post-processing, this method achieves high-efficiency and high-precision target recognition and positioning within the limited computing resources of embedded edge devices. It significantly reduces reliance on high-cost 3D vision technologies and enhances the system's adaptability and generalization capabilities in diverse logistics scenarios.

[0219] To address common issues in complex stacking scenarios such as mask loss, segmentation anomalies, and real-time requirements, this invention proposes a three-level optimization mechanism that integrates dynamic mesh completion, multi-scale morphological operations, and neighborhood consistency verification. This mechanism effectively addresses interference from occlusion, sensor noise, and illumination fluctuations, forming a complete processing chain from coarse segmentation to fine-grained repair, significantly enhancing the system's output stability and anti-interference performance in highly dynamic operating environments.

[0220] To improve system response speed and robustness, the mask generated in this embodiment not only possesses good geometric regularity but also achieves pixel-level matching accuracy with the dynamic mesh. By combining angle correction and inverse coordinate transformation mechanisms based on maximum likelihood estimation and introducing a centroid backtracking verification strategy, the final output is target pose information that is strictly aligned with the original physical space, greatly improving the robustness and decision-making rationality of the robot's grasping operations in complex stacking scenarios.

[0221] The embodiments of this invention demonstrate excellent engineering integration and practical deployment value. The system can operate stably with only a regular RGB camera and a commercial edge computing unit, without relying on additional active optical projection or special hardware configurations. This significantly reduces costs while also reducing sensitivity to environmental conditions. It can adapt to real-world scenarios involving changes in lighting, partial occlusion, and the coexistence of goods made of multiple materials, providing a reliable and efficient visual perception solution for the large-scale application of modern smart logistics systems.

[0222] To better implement the stacked goods positioning method in the embodiments of the present invention, correspondingly, the embodiments of the present invention also provide a stacked goods positioning device, such as... Figure 5 As shown, the stacked goods positioning device 500 includes:

[0223] Image acquisition module 501 is used to acquire image data of stacked goods;

[0224] The network construction module 502 is used to identify the acquired image data, obtain an initial mask, construct a dynamic mesh based on the spatial distribution pattern of the initial mask, and perform local fine segmentation on the missing meshes in the dynamic mesh to obtain the total mask set.

[0225] Anomaly removal module 503 is used to optimize the mask quality and remove anomalies for each mask in the total mask set based on the multidimensional matching degree of the dynamic mesh, so as to obtain the correct mask set.

[0226] The cargo positioning module 504 is used to perform an inverse coordinate transformation on the correct mask set based on the original coordinate system of the image data to obtain the correct binary mask, and to position the stacked cargo according to the correct binary mask.

[0227] The stacked goods positioning device 500 provided in the above embodiments can realize the technical solutions described in the above stacked goods positioning method embodiments. The specific implementation principles of each module or unit can be found in the corresponding content in the above stacked goods positioning method embodiments, which will not be repeated here.

[0228] The stacked goods positioning method and apparatus provided by the present invention have been described in detail above. Specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for positioning stacked goods, characterized in that, include: The acquired image data is identified to obtain an initial mask, and a dynamic mesh is constructed based on the spatial distribution pattern of the initial mask. The missing meshes in the dynamic mesh are then locally refined to obtain the total mask set. Based on the multidimensional matching degree of the dynamic mesh, each mask in the total mask set is optimized for mask quality and anomalies are removed to obtain the correct mask set. Based on the original coordinate system of the image data, the correct mask set is subjected to an inverse coordinate system transformation to obtain a correct binary mask, and the stacked goods are located according to the correct binary mask; The process involves constructing a dynamic mesh based on the spatial distribution pattern of the initial mask, and then performing local fine-tuning on the missing meshes in the dynamic mesh to obtain a total mask set, including: The initial mask is filtered and its angle is corrected to obtain the target mask; Based on the spatial distribution pattern of the target mask, a dynamic mesh is generated; The missing meshes in the dynamic mesh are locally refined to obtain a supplementary mask; Based on the initial mask and the supplementary mask, a total mask set is obtained; The step of generating a dynamic mesh based on the spatial distribution pattern of the target mask includes: The coordinates of the centroid of each mask in the target mask are calculated to obtain the coordinates of all mask centroids; Arrange all the centroid coordinates of the masks in ascending order in both the horizontal and vertical directions to obtain an ordered sequence. The distances between adjacent mask centroids in the ordered sequence in the horizontal and vertical directions are calculated to obtain a distance set; The mode of the distance set is taken as the most probable nominal horizontal distance and nominal vertical distance; Using all the centroids of the target mask as initial seed points, the mesh is expanded along the horizontal and vertical directions with the nominal horizontal distance and the nominal vertical distance as step sizes, respectively, to construct a dynamic mesh covering the area of ​​the stacked goods; The step of performing local fine-grained segmentation of the missing meshes in the dynamic mesh to obtain a supplementary mask includes: A KD tree is constructed based on all the centroids of the masks; Based on the KD tree, determine the area-weighted Euclidean distance from the center point of the dynamic mesh to the centroids of multiple nearest neighbors of the mask, and the area of ​​each nearest neighbor of the mask centroid; A dynamic threshold is determined based on the area. When the product-weighted Euclidean distance is greater than the dynamic threshold, the network at the corresponding grid center point is determined to be a missing mask region. The missing mask region is identified as a local region of interest and subjected to local fine segmentation to obtain a supplementary mask.

2. The method for positioning stacked goods according to claim 1, characterized in that, The multidimensional matching degree based on the dynamic mesh is used to optimize the mask quality and remove anomalies for each mask in the total mask set to obtain a correct mask set, including: The total mask set is filtered based on the calculation results of the multidimensional matching degree between each mask in the total mask set and the dynamic mesh to obtain the first mask set; Based on the number of pixels intersecting between each mask in the first mask set and the dynamic grid, the fill rate is obtained, and masks in the first mask set whose fill rate is less than a preset fill rate are removed to obtain a second mask set. Based on a preset cross-shaped structural element, each mask in the second mask set is expanded to obtain the expansion result. Then, the expansion result is eroded to obtain the third mask set. The third mask set is subjected to a neighborhood consistency check to obtain the correct mask set.

3. The method for positioning stacked goods according to claim 1, characterized in that, The process of filtering and angle correction on the initial mask to obtain the target mask includes: The first initial mask is obtained by retaining the masks whose geometric features are within the preset range of the standard proportion of the cargo in the initial mask. Based on the edge extraction algorithm, the cargo edge features of the first initial mask in the image data are extracted to obtain the vertical edge and the horizontal edge; The dominant orientation angles of the vertical and horizontal edges are determined based on the Hough transform. The target mask is obtained by rotating the first initial mask using the dominant direction angle.

4. The method for positioning stacked goods according to claim 2, characterized in that, The first mask set is obtained by filtering the total mask set based on the calculation results of the multidimensional matching degree between each mask in the total mask set and the dynamic mesh, including: Masks whose centroids are not within the extended coordinate range of the dynamic mesh are removed from the total mask set to obtain the retained mask set. The matching degree between each mask in the reserved mask set and the dynamic mesh is calculated to obtain the overall matching degree; The masks in the retained mask set whose overall matching degree is less than the preset matching degree are deleted to obtain the first mask set.

5. The method for positioning stacked goods according to claim 2, characterized in that, The step of performing neighborhood consistency verification on the third mask set to obtain the correct mask set includes: Determine the width and height of the minimum bounding rectangle of each mask in the third mask set, as well as the mean and standard deviation of the width and height of the mask dimensions in adjacent grids; The size dynamic threshold is obtained based on the mean and standard deviation. The width and height deviation rate is obtained by calculating the width and height deviation of each mask based on the average of the width and height. When the width-to-height deviation rate is not greater than the size dynamic threshold, the corresponding mask area is determined as a local region of interest. The local region of interest is segmented using the FastSAM model to obtain a new mask, and the cross-union ratio (CUI) between the new mask and the corresponding mask is determined. When the cross-union ratio is less than the preset cross-union ratio, the corresponding mask in the third mask set is deleted to obtain the correct mask set.

6. The method for positioning stacked goods according to claim 1, characterized in that, The positioning of stacked goods based on the correct binary mask includes: The centroid of the target mask is inversely transformed and calculated to obtain the correct binary mask in the original coordinate system; The centroid of the target mask is inversely transformed to obtain the predicted centroid value; Determine whether the Euclidean distance between the centroid prediction value and the centroid coordinates of the image data is less than or equal to a preset pixel; If so, the coordinates of the stacked goods in the image data are determined according to the correct binary mask, and the stacked goods are located according to the coordinates.

7. A stacked goods positioning device, characterized in that, include: The network construction module is used to identify the acquired image data, obtain an initial mask, construct a dynamic mesh based on the spatial distribution pattern of the initial mask, and perform local fine segmentation on the missing meshes in the dynamic mesh to obtain the total mask set. An anomaly removal module is used to optimize the mask quality and remove anomalies for each mask in the total mask set based on the multidimensional matching degree of the dynamic mesh, so as to obtain the correct mask set. The cargo positioning module is used to perform an inverse coordinate transformation on the correct mask set based on the original coordinate system of the image data to obtain a correct binary mask, and to position the stacked cargo according to the correct binary mask; The process involves constructing a dynamic mesh based on the spatial distribution pattern of the initial mask, and then performing local fine-tuning on the missing meshes in the dynamic mesh to obtain a total mask set, including: The initial mask is filtered and its angle is corrected to obtain the target mask; Based on the spatial distribution pattern of the target mask, a dynamic mesh is generated; The missing meshes in the dynamic mesh are locally refined to obtain a supplementary mask; Based on the initial mask and the supplementary mask, a total mask set is obtained; The step of generating a dynamic mesh based on the spatial distribution pattern of the target mask includes: The coordinates of the centroid of each mask in the target mask are calculated to obtain the coordinates of all mask centroids; Arrange all the centroid coordinates of the masks in ascending order in both the horizontal and vertical directions to obtain an ordered sequence. The distances between adjacent mask centroids in the ordered sequence in the horizontal and vertical directions are calculated to obtain a distance set; The mode of the distance set is taken as the most probable nominal horizontal distance and nominal vertical distance; Using all the centroids of the target mask as initial seed points, the mesh is expanded along the horizontal and vertical directions with the nominal horizontal distance and the nominal vertical distance as step sizes, respectively, to construct a dynamic mesh covering the area of ​​the stacked goods; The step of performing local fine-grained segmentation of the missing meshes in the dynamic mesh to obtain a supplementary mask includes: A KD tree is constructed based on all the centroids of the masks; Based on the KD tree, determine the area-weighted Euclidean distance from the center point of the dynamic mesh to the centroids of multiple nearest neighbors of the mask, and the area of ​​each nearest neighbor of the mask centroid; A dynamic threshold is determined based on the area. When the product-weighted Euclidean distance is greater than the dynamic threshold, the network at the corresponding grid center point is determined to be a missing mask region. The missing mask region is identified as a local region of interest and subjected to local fine segmentation to obtain a supplementary mask.

Images