Bin-pose real-time detection method for bin-pose real-time detection method for bin-pose real-time detection method for bin-pose real-time detection method for bin-pose real-time detection method for bin-pose real-time detection

Abstract

The application provides a binocular vision constrained embodied intelligent robot cargo pile pose real-time detection method, and relates to the technical field of intelligent robots, which comprises the following steps: synchronously acquiring a pair of continuous image frames of a cargo pile collected by bilateral optical sensors of an embodied intelligent robot, performing optical distortion correction and disparity initial estimation on the pair of image frames to obtain a spatially aligned disparity feature tensor; receiving the spatially aligned disparity feature tensor, analyzing and positioning three real object reference benchmarks of a cargo pile region; fitting an initial virtual space envelope ellipsoid surface with the three real object monitoring benchmark coordinate vectors as constraints, extracting topological parameters to construct a candidate mapping population; based on the candidate mapping population, calculating individual space fitting residuals and mapping them into fitness values, and iteratively updating the pose parameters until converging to a final geometric mapping configuration. The application realizes high-precision and low-delay real-time detection of six-degree-of-freedom poses of a cargo pile under complex dynamic working conditions.

Description

Technical Field

[0001] This invention relates to the field of intelligent robot technology, and in particular to a method for real-time detection of the pose of a stack of goods in an embodied intelligent machine constrained by binocular vision. Background Technology

[0002] In automated warehouses and flexible logistics sorting scenarios, embodied intelligent robots need to frequently perform automatic grasping and palletizing operations of mixed-size stacks of goods. They usually rely on binocular vision systems to calculate the six degrees of freedom pose of the stacks in real time in order to plan the approach trajectory of the end effector.

[0003] In actual warehousing environments, the outer packaging of pallets often exhibits irregular physical characteristics such as hot-melt wrinkles in the stretch film, localized compression deformation of the box body, and obstruction by the pallet fork tooth hole structure. Furthermore, the dynamic movement of shuttle vehicles through aisles and flickering ceiling lighting at the work site cause traditional binocular vision algorithms to be highly susceptible to sudden changes in local deformation and field of view occlusion during disparity matching and 3D spatial mapping. This results in deviations in feature point coordinate mapping and initial pose estimation drift. Existing technical solutions lack adaptive geometric envelope modeling and dynamic pose offset compensation mechanisms for key physical anchor points of the pallet, making it difficult to achieve high-precision, low-latency real-time detection of the pallet reference coordinate system under complex dynamic conditions. This restricts the success rate and operational continuity of embodied intelligent robots in high-density warehousing scenarios. Summary of the Invention

[0004] This invention provides a method for real-time detection of the pose of a stack of goods in an embodied intelligent machine with binocular vision constraints, achieving high-precision and low-latency real-time detection of the six degrees of freedom pose of the stack of goods under complex dynamic conditions.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0006] In a first aspect, a method for real-time detection of the pose of a stack of goods in an embodied intelligent machine constrained by binocular vision is provided, the method comprising:

[0007] Simultaneously acquire continuous image frames of the pallet from the dual optical sensors of the embodied intelligent robot, perform optical distortion correction and preliminary parallax estimation on the image frame pairs, and obtain spatially aligned parallax feature tensors.

[0008] The spatially aligned disparity feature tensor is received, and the physical reference datums of the three pallet areas are analyzed and located. The initial virtual spatial envelope ellipsoid is fitted with the coordinate vectors of the three physical monitoring datums as constraints, and the topological parameters are extracted to construct a candidate mapping population.

[0009] Based on the candidate mapping population, the individual spatial fitting residuals are calculated and mapped to fitness values. The configuration parameters are iteratively updated according to these values ​​until convergence to the final geometric mapping configuration.

[0010] The final configuration is applied to the initial ellipsoid to obtain the target ellipsoid; the target ellipsoid is subjected to manifold discrete sampling to obtain the coordinate sequence and input into the normal curvature gradient field to obtain the curvature gradient distribution matrix; the curvature gradient distribution matrix is ​​orthogonally decomposed to extract the attitude offset vector, and the pose offset compensation parameter is calculated.

[0011] Based on the pose offset compensation parameter, the parallax feature space mapping is guided, and the field of view occlusion interference data is filtered out to converge and extract the surface contour features of the stack, thus obtaining the initial set of discrete spatial representations of the stack.

[0012] Based on the initial set of discrete representations of the stack space, a spatial topological constraint matrix of the stack surface is constructed, and the initial pose parameters of the stack reference coordinate system are derived.

[0013] The initial pose parameters are input into the binocular vision geometric constraint solution process, and the initial pose parameters are dynamically compensated and corrected to obtain the real-time six-degree-of-freedom pose detection results of the pallet.

[0014] Furthermore, the system simultaneously acquires continuous image frames of the pallet from the dual optical sensors of the embodied intelligent robot, performs optical distortion correction and preliminary disparity estimation on the image frame pairs, and obtains spatially aligned disparity feature tensors, including:

[0015] Based on a unified timestamp, left-view image frames and right-view image frames of the stack area are collected to construct a continuous image frame pair sequence;

[0016] Receive a sequence of consecutive image frames, perform radial distortion demapping and tangential distortion compensation operations on each frame in the sequence, and obtain a standard image pair after optical distortion correction.

[0017] Perform a reprojection alignment operation on the standard image pair to make the left viewpoint image row and the right viewpoint image row strictly coplanar, eliminate the vertical offset in the disparity calculation, and obtain the reprojected aligned image pair;

[0018] Pixel-level disparity matching calculation based on cost volume aggregation is performed on the reprojected and aligned image pairs to obtain an initial disparity map. The initial disparity map and the texture channels of the optically distorted standard image pairs are spatially stacked along the feature dimension to obtain a spatially aligned disparity feature tensor.

[0019] Furthermore, the spatially aligned disparity feature tensor is received to resolve and locate the physical reference datums of the three pallet areas; the initial virtual spatial envelope ellipsoid is fitted with the coordinate vectors of the three physical monitoring datums as constraints, and topological parameters are extracted to construct a candidate mapping population, including:

[0020] The system receives spatially aligned parallax feature tensors, separates the depth channel and texture channel through multi-scale feature decoupling operations, locates the three-dimensional coordinate extreme values ​​of the intersection reference of the inner wall of the fork tooth channel at the bottom of the pallet in the depth channel, extracts the gray-scale gradient center of the centroid reference of the hot-melt overlap area of ​​the interlayer wrapping film in the texture channel, and identifies the edge curvature change position of the vertex reference of the corner anti-collision protection of the outer packaging box, thus obtaining a two-dimensional pixel coordinate set of the physical reference reference.

[0021] The two-dimensional pixel coordinate set of the physical reference datum is back-projected onto the robot base coordinate system, and spatial coordinate alignment is performed to obtain the three-dimensional coordinate vectors of the pallet bottom fork tooth hole inner wall intersection datum, the centroid datum of the interlayer wrapping film hot melt overlap area, and the vertex datum of the outer packaging box corner anti-collision corner guard.

[0022] Using the three-dimensional coordinate vectors of three physical reference datums as spatial geometric constraint primitives, the least squares spatial surface iterative fitting operation is performed to construct the initial virtual spatial envelope ellipsoid.

[0023] The axial scaling factor, eccentricity, and spatial orientation angle of the initial virtual spatial envelope ellipsoid are extracted as basic topological parameters. The basic topological parameters are expanded by multidimensional spatial random perturbation to obtain multiple sets of parameter combinations covering the final local neighborhood, and a candidate mapping population is constructed.

[0024] Furthermore, based on the candidate mapping population, the individual spatial fitting residuals are calculated and mapped to fitness values. The configuration parameters are then iteratively updated according to these values ​​until convergence to the final geometric mapping configuration, including:

[0025] Traverse each group of mapping individuals in the candidate mapping population, calculate the Euclidean distance deviation between the virtual envelope obtained by each mapping individual and the three-dimensional coordinate vectors of the three physical reference bases, and sum the Euclidean distance deviations to obtain the individual spatial fitting residual;

[0026] The individual spatial fitting residuals are input into the nonlinear mapping function and subjected to inverse weighting and normalization to obtain the fitness value of the geometric fit of each mapping individual.

[0027] High-quality mapping individuals are selected based on the distribution gradient of fitness values. Crossover and random perturbation operations are performed on the configuration parameters of the high-quality mapping individuals to obtain the next generation of updated population.

[0028] Replace the current candidate mapping population with the next generation of updated population, and iteratively execute the individual spatial fitting residual calculation and fitness value mapping process. When the fitness value change in adjacent iterations converges to the preset threshold range, terminate the iterative evolution, extract the parameters of the mapping individual with the highest fitness in the current population, and output them as the final geometric mapping configuration.

[0029] Furthermore, the final configuration is applied to the initial ellipsoid to obtain the target ellipsoid; manifold discretization sampling is performed on the target ellipsoid to obtain a coordinate sequence, which is then input into the normal curvature gradient field mapping to obtain the curvature gradient distribution matrix, including:

[0030] Receive the final geometric mapping configuration, extract the spatial translation components, axial scaling ratio and rotation attitude matrix within the configuration, superimpose the spatial translation components, axial scaling ratio and rotation attitude matrix onto the basic geometric parameters of the initial virtual space envelope ellipsoid, perform surface morphology affine correction, and obtain the morphology-optimized target virtual space envelope ellipsoid.

[0031] A parametric grid is established on the continuous surface of the target virtual space envelope ellipsoid. A manifold discrete sampling operation is performed along the preset meridional and latitudinal step amounts to capture the spatial position information of each grid node. The coordinate sequence is obtained by arranging the grid nodes according to the sampling traversal path.

[0032] The system receives a coordinate sequence, calculates the first-order tangent vector and second-order normal vector at each sampling node based on a differential geometry algorithm, constructs a local normal curvature calculation neighborhood, and performs gradient difference mapping on the curvature change rate in each neighborhood along the surface normal to obtain the curvature gradient distribution matrix of the local geometric deformation distribution.

[0033] Furthermore, the attitude offset vector is extracted by orthogonal decomposition of the curvature gradient distribution matrix, and the pose offset compensation parameters are calculated, including:

[0034] Receive the curvature gradient distribution matrix, perform singular value orthogonal decomposition on the curvature gradient distribution matrix, and extract the principal singular value components and the corresponding orthogonal characteristic direction basis vectors;

[0035] The spatial angle between the orthogonal characteristic direction basis vector and the axis of the preset standard reference coordinate system is compared, the deflection angle deviation of each axis is calculated, and the deflection angle deviation is tensor-coupled and mapped with the principal singular value component to obtain the spatial attitude offset vector.

[0036] Receive the spatial attitude offset vector, call the robot kinematics forward model to perform coordinate system transformation mapping, and solve the three-dimensional translational and three-dimensional rotational components of the spatial attitude offset vector in the robot end effector reference frame;

[0037] The pose offset compensation parameters are obtained by fusing the three-dimensional translation components and the three-dimensional rotation components and unifying their dimensions.

[0038] Furthermore, based on the pose offset compensation parameter to guide the parallax feature space mapping, the field-of-view occlusion interference data is filtered out to convergently extract the surface contour features of the pallet, resulting in an initial set of discrete spatial representations of the pallet, including:

[0039] The pose offset compensation parameter is parsed into a spatial pose transformation matrix. The spatially aligned disparity feature tensor is then subjected to global coordinate remapping through the spatial pose transformation matrix to obtain the reference aligned disparity feature tensor.

[0040] Based on the benchmark aligned disparity feature tensor, the disparity gradient continuity confidence in the local pixel neighborhood is calculated through iteration, and the disparity data region whose confidence falls below the preset occlusion threshold is marked as field of view occlusion interference data.

[0041] Logical stripping is performed on the field-of-view occlusion interference data from the reference-aligned parallax feature tensor to obtain the stripped effective feature data. Morphological smoothing filtering is then performed on the stripped effective feature data to repair it and converge to extract the surface contour features of the pallet.

[0042] Equal-interval normal spatial sampling is performed along the surface contour features of the pallet to capture the three-dimensional coordinates and surface normal attributes of each sampling node. The data are then restructured according to the spatial topology sequence to obtain the initial set of discrete spatial representations of the pallet.

[0043] Furthermore, based on the initial set of discrete representations of the stack space, a spatial topological constraint matrix for the stack surface is constructed, and the initial pose parameters of the stack reference coordinate system are derived, including:

[0044] The initial set of discrete representations of the stack space is received as the initial set of discrete representations of the stack space. Normal vector consistency clustering analysis is performed on each sampling node in the initial set of discrete representations of the stack space to divide the parallel plane feature clusters of the stack load-bearing layer and the lateral support surface.

[0045] Based on the parallel plane feature cluster, the centroid coordinates and unit normal vectors of each plane feature are solved by the least squares fitting algorithm. The spatial vertical spacing and cosine value of the angle between the normals between adjacent plane features are calculated to obtain the interlayer stacking geometric correlation parameters.

[0046] Based on the interlayer stacking geometric correlation parameters and the coordinates of each centroid, a spatial topological constraint matrix is ​​constructed for the vertical stacking constraint, side alignment constraint, and bottom support constraint of the stack.

[0047] The spatial topological constraint matrix is ​​solved by eigenvalue decomposition and homogeneous coordinate transformation. The spatial translation vector and rotation quaternion that minimize the matrix residual are extracted, and the initial pose parameters of the pallet reference coordinate system are obtained by inversion.

[0048] Furthermore, the initial pose parameters are input into the binocular vision geometric constraint solving process to dynamically compensate and correct the initial pose parameters, obtaining the real-time six-degree-of-freedom pose detection results of the pallet, including:

[0049] Receive the initial pose parameters of the pallet reference coordinate system, map the initial pose parameters to the camera imaging model of the dual-side optical sensors, establish the forward projection relationship between the pixel coordinates of the pallet feature points and the three-dimensional spatial coordinates, and obtain the initial reprojection residual reference.

[0050] The initial reprojection residual benchmark and the binocular epipolar geometric constraints are jointly encapsulated to construct the objective optimization function for multi-source error fusion.

[0051] Based on the objective optimization function, a nonlinear least squares iterative algorithm is used to solve the Jacobian matrix linearization of the translation and rotation components in the initial pose parameters, and the pose estimation value is updated frame by frame to eliminate the cumulative error introduced by dynamic motion.

[0052] When the convergence gradient of the objective optimization function satisfies the preset iteration stopping condition, the pose estimation value of the current frame is locked, and the real-time six-degree-of-freedom pose detection result of the pallet with three-dimensional spatial coordinate position and three-axis attitude angle is obtained.

[0053] Secondly, a binocular vision-constrained embodied intelligent machine pallet pose real-time detection system includes:

[0054] The acquisition module is used to synchronously acquire continuous image frames of the pallet collected by the dual optical sensors of the embodied intelligent robot, perform optical distortion correction and preliminary parallax estimation on the image frame pairs, and obtain spatially aligned parallax feature tensors.

[0055] The fitting module receives the spatially aligned disparity feature tensor, analyzes and locates the physical reference datums of the three pallet areas; it fits the initial virtual spatial envelope ellipsoid with the coordinate vectors of the three physical monitoring datums as constraints, and extracts topological parameters to construct a candidate mapping population.

[0056] The update module is used to calculate the individual spatial fitting residuals based on the candidate mapping population and map them to fitness values, and then iteratively update the configuration parameters until they converge to the final geometric mapping configuration.

[0057] The calculation module is used to apply the final configuration to the initial ellipsoid to obtain the target ellipsoid; to perform manifold discrete sampling on the target ellipsoid to obtain the coordinate sequence and input it into the normal curvature gradient field to obtain the curvature gradient distribution matrix; to orthogonally decompose the curvature gradient distribution matrix to extract the attitude offset vector and solve for the pose offset compensation parameters.

[0058] The mapping module is used to guide the parallax feature space mapping based on the pose offset compensation parameter, filter out the field of view occlusion interference data to converge and extract the surface contour features of the stack, and obtain the initial set of discrete spatial representations of the stack.

[0059] The module is used to construct the spatial topological constraint matrix of the stacked surface of the stack based on the initial set of discrete representations of the stack space, and to inversely derive the initial pose parameters of the stack reference coordinate system.

[0060] The correction module is used to input the initial pose parameters into the binocular vision geometric constraint solution process, dynamically compensate and correct the initial pose parameters, and obtain the real-time six-degree-of-freedom pose detection results of the stack.

[0061] The above-described solution of the present invention has at least the following beneficial effects:

[0062] By using three specific physical reference points—the intersection of the inner walls of the fork holes at the bottom of the pallet, the centroid of the hot-melt overlap area of ​​the interlayer wrapping film, and the apex of the corner guards at the corners of the outer packaging box—as constraints to fit an initial virtual spatial envelope ellipsoid, and constructing a candidate mapping population for iterative optimization, the final geometric mapping configuration is obtained. This is combined with discrete sampling of the target ellipsoid manifold and orthogonal decomposition of the normal curvature gradient field to calculate the pose offset compensation parameters. Then, based on the compensation parameters, the disparity feature space mapping is guided, and visual field occlusion interference data is filtered out. Finally, a topological constraint matrix for the pallet stacking surface is constructed and jointly solved using binocular visual geometric constraints. The positive pose parameter technique overcomes the technical problems of existing technologies that rely on general feature point matching and are easily affected by environmental dynamic interferences such as stack wrapping film wrinkles, local deformation of the box, pallet fork tooth hole obstruction, aisle shuttle passage, and lighting flicker, which lead to feature point mismatch loss, large initial spatial mapping deviation, pose estimation drift and lack of effective dynamic compensation mechanism. Thus, it realizes high-precision, low-latency, and high-robust real-time detection of the six-degree-of-freedom pose of the stack under complex dynamic warehousing conditions, and improves the grasping success rate and operation continuity of the embodied intelligent robot in high-density warehousing scenarios. Attached Figure Description

[0063] Figure 1 This is a flowchart illustrating a binocular vision-constrained method for real-time detection of the pose of a stack of goods in an embodied intelligent machine, according to an embodiment of the present invention.

[0064] Figure 2 This is a schematic diagram of a binocular vision-constrained embodied intelligent machine pallet pose real-time detection system provided by an embodiment of the present invention.

[0065] Figure 3 This is a schematic diagram illustrating the trend of pose detection accuracy as a function of distance.

[0066] Figure 4 This is a simulation diagram of the three-dimensional spatial distribution of the pallet pose detection.

[0067] Figure 5 This is a diagram showing the overall effect comparison of the methods. Detailed Implementation

[0068] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art.

[0069] like Figure 1 As shown, an embodiment of the present invention proposes a real-time detection method for the pose of a stack of goods in an embodied intelligent machine constrained by binocular vision. The method includes the following steps:

[0070] Step 1: Simultaneously acquire continuous image frames of the pallet from the dual optical sensors of the embodied intelligent robot, perform optical distortion correction and preliminary parallax estimation on the image frame pairs, and obtain the spatially aligned parallax feature tensor.

[0071] Step 2: Receive the spatially aligned disparity feature tensor, analyze and locate the physical reference datums of the three pallet areas; fit the initial virtual spatial envelope ellipsoid with the coordinate vectors of the three physical monitoring datums as constraints, and extract topological parameters to construct a candidate mapping population;

[0072] Step 3: Based on the candidate mapping population, calculate the individual spatial fitting residuals and map them to fitness values. Iteratively update the configuration parameters according to this until convergence to the final geometric mapping configuration.

[0073] Step 4: Apply the final configuration to the initial ellipsoid to obtain the target ellipsoid; perform manifold discretization sampling on the target ellipsoid to obtain the coordinate sequence and input it into the normal curvature gradient field to obtain the curvature gradient distribution matrix; orthogonally decompose the curvature gradient distribution matrix to extract the attitude offset vector and solve for the pose offset compensation parameters.

[0074] Step 5: Guide the parallax feature space mapping based on the pose offset compensation parameter, filter out the field of view occlusion interference data to converge and extract the surface contour features of the pallet, and obtain the initial set of discrete representation of the pallet space.

[0075] Step 6: Based on the initial set of discrete representations of the stack space, construct the spatial topological constraint matrix of the stack surface and inversely derive the initial pose parameters of the stack reference coordinate system.

[0076] Step 7: Input the initial pose parameters into the binocular vision geometric constraint solution process, perform dynamic compensation and correction on the initial pose parameters, and obtain the real-time six-degree-of-freedom pose detection results of the pallet.

[0077] In this embodiment of the invention, continuous image frames of the stack from the dual-side optical sensors of the embodied intelligent robot are acquired, and spatially aligned disparity feature tensors are obtained. An initial virtual spatial envelope ellipsoid is fitted using three dedicated physical reference points for the stack as constraints, and a candidate mapping population is constructed for iterative optimization to obtain the final geometric mapping configuration. Pose bias compensation parameters are calculated through discrete sampling of the target ellipsoid manifold and orthogonal decomposition of the normal curvature gradient field. Based on the compensation parameters, disparity feature spatial mapping is guided, and visual field occlusion interference is filtered out to extract discrete spatial representations of the stack. Then, the initial pose is inverted by constructing a spatial topological constraint matrix of the stack stack surface. Finally, binocular visual geometry is combined with... The technical means of constraining and dynamically compensating for the initial pose overcomes the technical problems of existing technologies that rely on general feature point matching and are easily affected by complex dynamic environmental interferences such as stack wrapping film wrinkles, local extrusion deformation of the box, pallet fork tooth hole obstruction, aisle shuttle passage, and ceiling lighting flicker. This leads to feature point mismatch or batch loss, large initial spatial mapping deviation, pose estimation drift, and lack of effective dynamic compensation mechanism. As a result, it realizes high-precision, low-latency, and high-robust real-time detection of the six-degree-of-freedom pose of the stack under complex dynamic warehousing conditions, and improves the grasping success rate and operation continuity of the embodied intelligent robot in high-density warehousing scenarios.

[0078] In a preferred embodiment of the present invention, step 1 above may include:

[0079] Step 1.1: Based on a unified timestamp, collect left-view and right-view image frames of the pallet area to construct a continuous image frame pair sequence. Specifically, this includes: activating the hardware synchronous acquisition mode of the dual optical sensors of the embodied intelligent robot; simultaneously controlling the exposure and acquisition actions of the left and right optical sensors through hardware trigger signals; setting the image acquisition frame rate to 30 frames per second; assigning the same global timestamp with a precision of 1 millisecond to each set of simultaneously triggered left-view and right-view image frames; pairing the left-view image frames with the corresponding right-view image frames according to the order of their timestamps to form a preliminary set of image frame pairs; traversing all image frame pairs in the preliminary set and calculating the timestamp difference between the left-view and right-view image frames in each pair; when the timestamp difference of a pair of image frames is greater than 2 milliseconds, the pair is determined to be asynchronous and removed from the preliminary set; arranging all remaining synchronous image frame pairs in ascending order of their timestamps to finally construct a continuous image frame pair sequence.

[0080] Step 1.2: Receive a sequence of consecutive image frames. Perform radial distortion demapping and tangential distortion compensation operations on each frame in the sequence to obtain a standard image pair after optical distortion correction. Specifically, this includes: completing the single-target calibration of both optical sensors in a laboratory environment beforehand, calibrating the intrinsic parameter matrix of the left optical sensor, the distortion coefficient matrix of the left optical sensor, the intrinsic parameter matrix of the right optical sensor, and the distortion coefficient matrix of the right optical sensor respectively; receiving the sequence of consecutive image frames, and processing each pair of image frames in the sequence according to the order of their timestamps. For the current left viewpoint image to be processed, traverse each pixel in the image and calculate the coordinates of that pixel on the original imaging plane based on the intrinsic parameter matrix of the left optical sensor. Perform radial distortion demapping operation to calculate the pixel coordinates after radial distortion correction. The calculation formula is: ; ;

[0081] in, and These are the pixel coordinates on the original imaging plane. The distance from the pixel to the center of the image. , and denoted as the radial distortion coefficient in the distortion coefficient matrix of the left optical sensor.

[0082] Perform tangential distortion compensation calculation to obtain the final pixel coordinates after tangential distortion compensation. The calculation formula is as follows: ; ;

[0083] in, and denoted as the tangential distortion coefficient in the distortion coefficient matrix of the left optical sensor.

[0084] The grayscale values ​​of the corresponding pixels in the original left viewpoint image are assigned to the corrected left viewpoint image. and At the pixel position, repeat the complete optical distortion correction process described above, process the right viewpoint image of the current group to obtain the corrected right viewpoint image, process all image frame pairs in sequence, and finally obtain the standard image pair after optical distortion correction.

[0085] Step 1.3: Perform a reprojection alignment operation on the standard image pair to make the left viewpoint image row and the right viewpoint image row strictly coplanar, eliminating the longitudinal offset in the disparity calculation, and obtaining the reprojected aligned image pair. Specifically, this includes receiving the calibrated intrinsic parameter matrix of the left optical sensor, the intrinsic parameter matrix of the right optical sensor, the extrinsic parameter matrix of the left optical sensor relative to the robot base coordinate system, the extrinsic parameter matrix of the right optical sensor relative to the robot base coordinate system, and the acquired standard image pair, which includes the original left viewpoint image and the original right viewpoint image.

[0086] Calculate the relative pose parameters of the right optical sensor with respect to the left optical sensor, calculate the inverse matrix of the extrinsic parameter matrix of the left optical sensor, multiply the extrinsic parameter matrix of the right optical sensor with respect to the robot base coordinate system by the inverse matrix of the extrinsic parameter matrix of the left optical sensor to obtain the homogeneous transformation matrix of the right optical sensor with respect to the left optical sensor, extract the first three rows and first three columns of the homogeneous transformation matrix as the relative rotation matrix, and extract the first three rows and fourth column as the relative translation vector.

[0087] Calculate the rotation matrix for binocular stereo correction, construct the correction rotation matrix for the left optical sensor, normalize the relative translation vector to obtain the unit translation vector, take the unit translation vector as the X-axis direction of the corrected left optical sensor, take the cross product of the original optical axis direction of the left optical sensor and the unit translation vector as the Y-axis direction of the corrected left optical sensor, and take the cross product of the X-axis direction and the Y-axis direction as the Z-axis direction of the corrected left optical sensor. Arrange these three orthogonal unit vectors in columns to obtain the correction rotation matrix for the left optical sensor. Construct the correction rotation matrix for the right optical sensor, multiply the correction rotation matrix of the left optical sensor by the relative rotation matrix to obtain the correction rotation matrix of the right optical sensor. Through the above rotation matrix transformation, the optical axis directions of the left and right optical sensors can be made completely parallel, and the imaging planes of the two optical sensors are in the same plane.

[0088] Calculate the corrected camera intrinsic parameter matrix, multiply the original left optical sensor intrinsic parameter matrix by the left optical sensor correction rotation matrix to obtain a temporary intrinsic parameter matrix, adjust the principal point coordinates of the temporary intrinsic parameter matrix to maximize the effective area of ​​the corrected image and minimize the black border area, and use the adjusted temporary intrinsic parameter matrix as the corrected left optical sensor intrinsic parameter matrix, while keeping the corrected right optical sensor intrinsic parameter matrix consistent with the left optical sensor intrinsic parameter matrix.

[0089] Calculate the reprojection mapping tables for the left and right images. For each pixel in the corrected image, convert it to homogeneous coordinates by appending a 1 to the end of the two-dimensional pixel coordinates. Multiply this homogeneous coordinate by the inverse of the corrected intrinsic parameter matrix to obtain the coordinates in the normalized camera coordinate system. Multiply this normalized coordinate by the inverse of the left optical sensor correction rotation matrix to obtain the normalized coordinates in the original left optical sensor camera coordinate system. Multiply this normalized coordinate by the original left optical sensor intrinsic parameter matrix to obtain the corresponding floating-point pixel coordinates in the original left image. Store the corresponding coordinates of all pixels in the left image reprojection mapping table. Repeat the above process to calculate the right image reprojection mapping table.

[0090] Perform bilinear interpolation resampling. For each pixel in the corrected image, find its corresponding floating-point coordinates in the original image according to the reprojection mapping table. Take the gray values ​​of the four adjacent integer pixels at the top left, top right, bottom left, and bottom right corners of these floating-point coordinates. Calculate the interpolated gray values ​​for the upper and lower rows horizontally, using the following formula: ; ;

[0091] in, The decimal part of the floating-point coordinates. , , , Given the grayscale values ​​of four adjacent integer pixels, calculate the final interpolated grayscale value in the vertical direction using the following formula: ;

[0092] in, The fractional part of the floating-point coordinates is used to assign the calculated grayscale value to the corresponding pixel in the corrected image.

[0093] Verify the reprojection alignment effect. Randomly select 100 corner features in the left image, and find the corresponding matching feature points in the right image using a 3x3 pixel template. Calculate the vertical pixel coordinate difference of each pair of matching feature points to obtain the vertical disparity error. Calculate the average and maximum values ​​of all vertical disparity errors. When the average value is less than 0.5 pixels and the maximum value is less than 1 pixel, the reprojection alignment is considered successful. Finally, obtain the reprojected and aligned left and right viewpoint images.

[0094] Step 1.4: Perform pixel-level disparity matching calculation based on cost volume aggregation on the reprojected and aligned image pairs to obtain an initial disparity map. Spatially stack the initial disparity map and the texture channels of the standard image pair after optical distortion correction along the feature dimension to obtain a spatially aligned disparity feature tensor. Specifically, this includes: receiving the reprojected and aligned image pairs; setting the disparity search range to 0 to 128 pixels according to the working distance range of the embodied intelligent robot; for each pixel in the left image, calculating the matching cost corresponding to each candidate disparity in the corresponding row of the right image, from the minimum disparity value to the maximum disparity value. The matching cost is calculated using the sum of absolute differences, and the calculation formula is: ;

[0095] in, The coordinates in the left image are The grayscale value of the pixel, The coordinates in the right image are The grayscale value of the pixel, These are candidate disparity values.

[0096] By traversing all pixels and all candidate disparities, the matching cost corresponding to all candidate disparities at all locations is calculated, constructing a 3D cost volume with dimensions of image height multiplied by image width multiplied by the disparity search range. Multi-scale cost aggregation is performed on the 3D cost volume, downsampling the original cost volume to half and a quarter of its original resolution. Cost aggregation operations with a window size of 9x9 pixels are performed on the cost volumes at three different scales. The aggregation results at different scales are then fused to obtain the final aggregated cost volume. For each pixel, the candidate disparity with the minimum aggregated cost is selected as the final disparity value for that pixel. After traversing all pixels to complete the disparity calculation, an initial disparity map is obtained. The red, green, and blue channels of the optically distortion-corrected standard image pair are extracted as texture channels, with the spatial resolution of the three texture channels being completely consistent with the spatial resolution of the initial disparity map. The initial disparity map is used as the depth channel and spatially stacked with the three texture channels along the feature dimension. During the stacking process, the spatial positions of all channels are maintained in a one-to-one correspondence, ultimately resulting in a spatially aligned disparity feature tensor with dimensions of image height multiplied by image width multiplied by four.

[0097] In this embodiment of the invention, by combining image frame pair acquisition technology with global timestamp verification, optical correction technology with radial distortion demapping and tangential distortion compensation, reprojection line alignment technology with binocular epipolar geometric constraints, and feature construction technology with cost volume aggregation pixel-level disparity matching combined with depth texture channel spatial stacking, the technical problems of disparity calculation deviation caused by asynchronous timing of traditional binocular image acquisition, pixel coordinate mapping error caused by inherent distortion of optical lenses, longitudinal disparity interference caused by non-coplanar left and right images, and feature matching ambiguity caused by lack of texture constraints in single disparity information are overcome.

[0098] In a preferred embodiment of the present invention, step 2 above may include:

[0099] Step 2.1: Receive the spatially aligned disparity feature tensor. Separate the depth channel and texture channel through multi-scale feature decoupling operations. Locate the 3D coordinate extrema of the intersection reference of the inner wall of the fork-tooth channel at the bottom of the pallet in the depth channel. Extract the grayscale gradient center of the centroid reference of the interlayer wrapping film thermal fusion overlap area in the texture channel. Identify the abrupt change in edge curvature of the corner protector vertex reference of the outer packaging box. Obtain the 2D pixel coordinate set of the physical reference reference. Specifically, this includes: downsampling the disparity feature tensor to half and a quarter of the original resolution, and performing channel dimension separation operations on the feature tensor at three different scales. The fourth channel is obtained as the depth channel, and the first three channels are merged as the texture channel. Feature fusion is performed on the three different scale depth channels to obtain the final depth feature map. Feature fusion is performed on the three different scale texture channels to obtain the final texture feature map.

[0100] In the depth feature map, all pixels are traversed to search for local extrema of depth values. A search window of size 15 by 15 pixels is established around each local extrema, and the variance of depth values ​​of all pixels within the window is calculated. When the variance of depth values ​​is greater than 0.8, the region is determined to be the intersection region of the inner wall of the fork tooth channel at the bottom of the tray. The pixel with the largest depth value is further searched within this region, and the coordinates of this pixel are used as the two-dimensional pixel coordinates of the intersection reference of the inner wall of the fork tooth channel at the bottom of the tray.

[0101] In the texture feature map, the grayscale gradient value of each pixel is calculated. The formula for calculating the grayscale gradient value is: ;

[0102] in, This is the square of the grayscale gradient value of the pixel in the horizontal direction. It is the square of the grayscale gradient value of the pixel in the vertical direction.

[0103] Iterate through all pixels and select those with a grayscale gradient value greater than 0.5 to form a candidate pixel set for the thermal fusion overlap area of ​​the interlayer wound film. Calculate the centroid coordinates of the candidate pixel set and use these centroid coordinates as the two-dimensional pixel coordinates of the centroid reference of the thermal fusion overlap area of ​​the interlayer wound film.

[0104] In the texture feature map, the edge curvature value of each pixel is calculated. A calculation window of size 7 by 7 pixels is established around each pixel, and the rate of change of curvature of all edge pixels within the window is calculated. When the rate of change of curvature of a pixel is greater than 1.2, the pixel is determined to be an edge curvature abrupt change point. Among all edge curvature abrupt change points, the pixel with the largest curvature value located in the image edge region is selected, and the coordinates of the pixel are used as the two-dimensional pixel coordinates of the vertex reference of the corner guard of the outer packaging box. The two-dimensional pixel coordinates of the above three references are integrated to obtain the two-dimensional pixel coordinate set of the physical reference reference.

[0105] Step 2.2 involves back-projecting the two-dimensional pixel coordinate set of the physical reference datum onto the robot's base coordinate system and aligning the spatial coordinates to obtain the three-dimensional coordinate vectors of the pallet bottom fork hole inner wall intersection datum, the centroid datum of the interlayer wrapping film hot-melt overlap area, and the vertex datum of the outer packaging box corner anti-collision guard. Specifically, this includes: receiving the two-dimensional pixel coordinate set of the physical reference datum and processing the two-dimensional pixel coordinates of each datum sequentially. For the current datum two-dimensional pixel coordinates to be processed, it is converted into homogeneous coordinate form. This homogeneous coordinate is multiplied by the inverse matrix of the left optical sensor intrinsic parameter matrix to obtain the normalized coordinates of the datum point in the left optical sensor camera coordinate system. This normalized coordinate is then multiplied by the depth value corresponding to the datum point to obtain the three-dimensional coordinates of the datum point in the left optical sensor camera coordinate system.

[0106] The 3D coordinates of the reference point in the left optical sensor camera coordinate system are converted into homogeneous coordinates. These homogeneous coordinates are then multiplied by the extrinsic parameter matrix of the left optical sensor relative to the robot's base coordinate system to obtain the 3D coordinates of the reference point in the robot's base coordinate system.

[0107] Repeat the above complete coordinate back projection and transformation process to process the two-dimensional pixel coordinates of the intersection reference of the inner wall of the pallet bottom fork hole, the centroid reference of the hot melt overlap area of ​​the interlayer wrapping film, and the vertex reference of the corner anti-collision protection of the outer packaging box. Integrate the three three-dimensional coordinates to obtain the three-dimensional coordinate vectors of the three physical reference references.

[0108] Step 2.3: Using the three-dimensional coordinate vectors of the three physical reference bases as spatial geometric constraint primitives, perform least squares spatial surface iterative fitting operation to construct an initial virtual spatial envelope ellipsoid. Specifically, this includes: receiving the three-dimensional coordinate vectors of the three physical reference bases, using them as spatial geometric constraint primitives, performing least squares spatial surface iterative fitting operation, and initializing the parameters of the virtual spatial envelope ellipsoid, including the center position coordinates, the three-axis semi-length parameters, and the spatial attitude Euler angles.

[0109] Calculate the Euclidean distance from each physical reference point to the current virtual ellipsoid. The formula for calculating the Euclidean distance is: ;

[0110] in,( , , ) is the first The three-dimensional coordinates of a physical reference point, , , ) represents the coordinates of the center position of the current virtual ellipsoid.

[0111] Calculate the sum of squared Euclidean distances for all reference points to obtain the fitting residual for the current iteration. Based on the gradient direction of the fitting residual, update the center position coordinates, semi-major parameters of the three axes, and Euler angles of the spatial attitude of the virtual ellipsoid. Repeat the above fitting residual calculation and parameter update process. When the change in fitting residual between two adjacent iterations is less than 0.001, terminate the iteration operation, extract the center position coordinates, semi-major parameters of the three axes, and Euler angles of the spatial attitude obtained in the final iteration, and construct the initial virtual spatial envelope ellipsoid.

[0112] Step 2.4: Extract the axial scale factor, eccentricity, and spatial orientation angle of the initial virtual spatial envelope ellipsoid as basic topological parameters. Perform multidimensional spatial random perturbation expansion on these basic topological parameters to obtain multiple parameter combinations covering the final local neighborhood, constructing a candidate mapping population. Specifically, this includes: receiving the geometric parameters of the initial virtual spatial envelope ellipsoid and calculating the axial scale factor and eccentricity of the ellipsoid. The axial scale factor is the ratio of the semi-length of the longest axis to the semi-length of the shortest axis of the ellipsoid. The formula for calculating the eccentricity is: ;

[0113] in, The semi-length of the major axis of the ellipsoid. It is the semi-length of the minor axis of the ellipsoid.

[0114] The axial scale factor, eccentricity, and spatial orientation angle are extracted as basic topological parameters. The range of the pose parameter solution domain is set: the solution domain for the center position coordinates is the initial value plus or minus 0.5 meters; the solution domain for the three-axis half-length parameters is the initial value plus or minus 0.3 meters; and the solution domain for the spatial attitude Euler angles is the initial value plus or minus 10 degrees. Within the preset pose parameter solution domain, a normally distributed perturbation sequence with a mean of the basic topological parameter value and a standard deviation of one-tenth of the solution domain is obtained. This normally distributed perturbation sequence is introduced into each basic topological parameter to obtain multiple different parameter combinations. The size of the candidate mapping population is set to 50 groups to ensure that the obtained parameter combinations cover the local final neighborhood. All obtained parameter combinations are integrated to construct the candidate mapping population.

[0115] In this embodiment of the invention, a joint positioning technique using three types of differentiated physical reference benchmarks is achieved through multi-scale feature decoupling and separation of depth channels and texture channels, the intersection benchmark of the inner wall of the fork hole at the bottom of the pallet, the centroid benchmark of the hot melt overlap area of ​​the interlayer wrapping film, and the vertex benchmark of the corner protection of the outer packaging box, the spatial alignment technique of back-projecting two-dimensional pixel coordinates to the robot base coordinate system, the least squares spatial surface iterative fitting technique with three benchmark constraints, and the technique of extracting ellipsoidal topological parameters and constructing candidate mapping populations by combining normal distribution perturbation expansion. Therefore, it overcomes the technical problems of traditional algorithms that rely only on a single feature point and are easily affected by local deformation of the pallet and environmental interference, resulting in positioning failure; single channel features cannot simultaneously capture the structural features and texture features of the pallet; lack of overall spatial envelope constraints leading to initial pose estimation drift; and excessively large parameter solution domain search range leading to slow convergence speed. Thus, it achieves fast, robust, and coarse positioning of the overall spatial shape of the pallet.

[0116] In a preferred embodiment of the present invention, step 3 above may include:

[0117] Step 3.1: Traverse each group of mapping individuals within the candidate mapping population, calculate the Euclidean distance deviation between the virtual curved envelope obtained by each mapping individual and the three-dimensional coordinate vectors of the three physical reference bases, and accumulate the Euclidean distance deviations to obtain the individual spatial fitting residual. Specifically, this includes: receiving the constructed candidate mapping population, which contains 50 groups of mapping individuals. Each group of mapping individuals corresponds to a complete set of virtual spatial envelope ellipsoidal parameters, including the center position coordinates, three-axis semi-length parameters, and spatial attitude Euler angles. Traverse each group of mapping individuals within the candidate mapping population, processing each mapping individual sequentially. For the current mapping individual to be processed, obtain the corresponding virtual curved envelope based on its contained ellipsoidal parameters. Calculate the shortest Euclidean distance deviation from the three-dimensional coordinate vectors of the three physical reference bases to the virtual curved envelope sequentially. For each physical reference base point, calculate the shortest distance from that point to the virtual ellipsoidal surface using the following formula: ;

[0118] in, For the first A three-dimensional coordinate vector of a physical reference point. Distance on the virtual ellipsoid The three-dimensional coordinate vector of the nearest point. The Euclidean distance deviations corresponding to the three physical reference points are summed to obtain the individual spatial fitting residual of the mapped individual, calculated using the following formula: ;

[0119] in, For individual spatial fitting residuals, , and These are the Euclidean distance deviations corresponding to the intersection reference of the inner wall of the fork tooth channel at the bottom of the pallet, the centroid reference of the hot-melt overlap area of ​​the interlayer wrapping film, and the vertex reference of the corner guard of the outer packaging box. The complete calculation process described above is repeated, processing all mapping individuals within the candidate mapping population sequentially to obtain the individual space fitting residual for each mapping individual.

[0120] Step 3.2 involves inputting the individual spatial fitting residuals into a nonlinear mapping function for inverse weighting and normalization to obtain the fitness value of geometric fit for each mapped individual. Specifically, this includes: receiving the individual spatial fitting residuals corresponding to all mapped individuals; inputting each individual spatial fitting residual into a preset nonlinear mapping function; and sequentially performing inverse weighting and normalization to obtain the fitness value of geometric fit for each mapped individual. The inverse weighting process converts the fitting residuals into a positive fit index, calculated using the following formula: ;

[0121] in, For the first The weighted fitness value of each mapped individual. For the first Individual spatial fitting residuals of each mapped individual, This is a very small positive number, with a value of 0.0001, used to avoid the abnormal case of a denominator of zero. Then, normalization is performed to map all weighted fitness values ​​to the interval between 0 and 1. Through the above processing, the smaller the individual space fitting residual, the higher the degree of fit between the virtual curved envelope and the actual stack of goods, and the larger the corresponding fitness value.

[0122] Step 3.3: Select high-quality mapping individuals based on the distribution gradient of fitness values, and perform crossover and random perturbation operations on the configuration parameters of the high-quality mapping individuals to obtain the next generation update population. Specifically, this includes: receiving the fitness values ​​of all mapping individuals, sorting all mapping individuals in the candidate mapping population according to the fitness values ​​from largest to smallest, selecting high-quality mapping individuals based on the distribution gradient of fitness values, and selecting the top 20% of mapping individuals as the high-quality mapping individual set. In this embodiment, the number of high-quality mapping individuals is 10. Perform crossover and recombination operations on the configuration parameters of the high-quality mapping individual set. Randomly select two different individuals in the high-quality mapping individual set as parent individuals, and linearly combine the corresponding configuration parameters of the two parent individuals according to a random ratio to obtain the configuration parameters of the offspring individuals. The crossover and recombination ratio is randomly selected between 0.3 and 0.7, and the calculation formula is: ;

[0123] in, For the first generation of individuals Individual shape parameters, For the first parent individual's first Individual shape parameters, For the second parent individual Individual shape parameters, The random crossover ratio coefficient is used. The crossover and recombination operation is repeated to obtain 40 offspring individuals. A random perturbation operation is performed on the configuration parameters of all offspring individuals. For each configuration parameter of each offspring individual, a normal distributed random perturbation with a mean of 0 and a standard deviation equal to one-twentieth of the corresponding parameter's solution domain is introduced. The configuration parameters of all offspring individuals subjected to the random perturbation operation are restricted to a preset pose parameter solution domain, ensuring that all parameters are within a valid value range. The set of high-quality mapped individuals is merged with the offspring individuals obtained through crossover and recombination and the random perturbation operation to obtain a next-generation update population containing 50 mapped individuals.

[0124] Step 3.4: Replace the current candidate mapping population with the next-generation updated population, and iteratively execute the individual space fitting residual calculation and fitness value mapping process. When the change in fitness value in adjacent iterations converges to a preset threshold range, terminate the iterative evolution, and extract the parameters of the mapping individual with the highest fitness in the current population as the final geometric mapping configuration. Specifically, this includes: replacing the current candidate mapping population with the next-generation updated population, iteratively executing the individual space fitting residual calculation and fitness value mapping process, and calculating the average fitness value of the population in the current iteration after each iteration. The calculation formula is as follows: ;

[0125] in, This represents the average fitness value of the population in the current iteration cycle. The population size is used to calculate the change in the average fitness value of the population between two adjacent iterations. The formula is as follows: ;

[0126] in, The change in fitness value This represents the average fitness value for the current iteration period. This is the average fitness value of the previous iteration cycle. When the change in fitness value is less than or equal to the preset convergence threshold of 0.001, or when the number of iterations reaches the preset maximum number of iterations of 50, the iterative evolution process is terminated. After the iteration terminates, all mapping individuals in the current population are sorted in descending order of fitness value, and all configuration parameters of the mapping individual ranked first are extracted and output as the final geometric mapping configuration.

[0127] In this embodiment of the invention, the individual spatial fitting residual calculation technique based on the sum of the Euclidean distance deviations of three physical reference benchmarks, the nonlinear fitness mapping technique of residual reciprocal weighting and normalization, the high-optimal mapping individual selection technique based on fitness distribution gradient, the population update technique of cross-recombination of high-optimal individual configuration parameters and random perturbation, and the iteration termination technique of fitness change threshold control are used to overcome the technical problems of traditional single-parameter optimization methods, such as easy getting trapped in local final solutions, inability to find the global final geometric mapping relationship in multi-dimensional complex parameter space, inability of fitness function to accurately reflect the degree of fit between virtual envelope and actual stack, and slow iteration convergence speed due to low population update efficiency. Thus, it can quickly and accurately converge to the global final geometric mapping configuration and accurately characterize the overall spatial shape of the stack.

[0128] In a preferred embodiment of the present invention, step 4 above may include:

[0129] Step 4.1: Receive the final geometric mapping configuration, extract the spatial translation components, axial scaling ratios, and rotational attitude matrices within the configuration, and superimpose these components onto the basic geometric parameters of the initial virtual spatial envelope ellipsoid. Perform affine surface morphology correction to obtain the morphologically optimized target virtual spatial envelope ellipsoid. Specifically, this includes: receiving the output final geometric mapping configuration, which contains complete virtual spatial envelope ellipsoid parameters optimized through population iteration, including center position coordinates, three-axis semi-length parameters, and spatial attitude Euler angles; extracting the spatial translation components from the final geometric mapping configuration, which are the differences between the center position coordinates of the final ellipsoid and the center position coordinates of the initial virtual spatial envelope ellipsoid; and extracting the axial scaling ratio, which is the ratio of the three-axis semi-length parameters of the final ellipsoid to the corresponding three-axis semi-length parameters of the initial ellipsoid.

[0130] The spatial attitude Euler angles of the final ellipsoid are converted into the corresponding rotation attitude matrix. An affine correction operation is performed on the surface morphology of the initial virtual spatial envelope ellipsoid, in the order of rotation, scaling, and translation. For the 3D coordinates of any point on the initial ellipsoid, the coordinates are multiplied by the rotation attitude matrix to complete the spatial rotation transformation; the rotated coordinates are multiplied by the scaling factor of the corresponding axis to complete the axial scaling transformation; the scaled coordinates are added to the spatial translation component to complete the spatial translation transformation. This process is repeated for all feature points on the initial virtual spatial envelope ellipsoid, obtaining the transformed 3D coordinates of all feature points. Based on these transformed feature points, the morphology-optimized target virtual spatial envelope ellipsoid is reconstructed, achieving the highest degree of fit between this ellipsoid and the overall spatial morphology of the actual stack.

[0131] Step 4.2: Establish a parametric mesh on the continuous surface of the target virtual space envelope ellipsoid. Perform manifold discretization sampling along preset meridional and latitudinal step sizes to capture the spatial position information of each mesh node. Arrange the coordinate sequence according to the sampling traversal path. Specifically, this includes: establishing a spherical parametric mesh on the continuous surface of the target virtual space envelope ellipsoid, describing all points on the ellipsoid using latitude and longitude parametric methods, and defining the parametric equations of the ellipsoid as follows: ;

[0132] in, , , These represent the half-lengths of the X-axis, Y-axis, and Z-axis of the target ellipsoid, respectively. This is a meridional parameter, with a value range from 0 to 360 degrees. The latitudinal parameter has a value range of 0 to 180 degrees. The preset longitudinal step is 10 degrees, and the latitudinal step is 10 degrees. Starting from 0 degrees along the longitudinal direction, the step is increased by 10 degrees each time until 360 degrees are obtained, thus obtaining all longitudinal sampling lines. Starting from 0 degrees along the latitudinal direction, the step is increased by 10 degrees each time until 180 degrees are obtained, thus obtaining all latitudinal sampling lines.

[0133] The intersection of the meridional sampling line and the latitudinal sampling line is the grid node of the manifold discrete sampling. Traversing all grid nodes, the meridional and latitudinal parameters corresponding to each node are substituted into the above ellipsoidal parameter equation to calculate the three-dimensional spatial position information of each grid node. Following the traversal path of first the latitudinal direction and then the meridional direction, the three-dimensional coordinates of all grid nodes are arranged in sequence to obtain an ordered coordinate sequence.

[0134] Step 4.3: Receive the coordinate sequence, calculate the first-order tangent vector and second-order normal vector at each sampling node based on a differential geometry algorithm, construct a local normal curvature calculation neighborhood, and perform gradient difference mapping on the curvature change rate within each neighborhood along the surface normal to obtain the curvature gradient distribution matrix of the local geometric deformation distribution. Specifically, this includes: receiving the coordinate sequence and calculating the first-order tangent vector and second-order normal vector at each sampling node based on a differential geometry algorithm. For any sampling node, calculate the meridional and latitudinal partial derivatives of the ellipsoidal parametric equation to obtain the two first-order tangent vectors at that node. The calculation formula is as follows: ;

[0135] in, It is a first-order tangent vector in the meridional direction. Let be the first-order tangent vector in the latitudinal direction. Perform a cross product of the two first-order tangent vectors to obtain the second-order normal vector at that node. Then, normalize the normal vector to obtain the unit normal vector. The calculation formula is: ;

[0136] in, For each sampling node, the unit normal vector is used. All sampling nodes are traversed to calculate the first-order tangent vector and the unit normal vector. A local normal curvature calculation neighborhood is constructed for each sampling node, with the neighborhood consisting of 3x3 grid nodes surrounding that node. For each local normal curvature calculation neighborhood, the angle between the normal vectors of all nodes within the neighborhood and the normal vector of the central node is calculated to obtain the curvature change rate within that neighborhood. The curvature change rate within each neighborhood is then mapped using gradient difference along the surface normal of the central node to obtain the curvature gradient value corresponding to that node. The curvature gradient values ​​of all sampling nodes are arranged according to their positions in the parameterized grid, constructing a curvature gradient distribution matrix with the number of rows equal to the number of latitudinal sampling points and the number of columns equal to the number of meridional sampling points. This matrix fully characterizes the local geometric deformation distribution of each region on the stack surface.

[0137] In this embodiment of the invention, by extracting the final geometric mapping configuration parameters and using affine correction techniques for the initial virtual space envelope ellipsoid, establishing parameterized meshes for the target ellipsoid and using meridional and latitudinal manifold discretization sampling techniques, calculating first-order tangent vectors and second-order normal vectors based on differential geometry, and constructing a local normal curvature calculation neighborhood and using curvature change rate normal gradient difference mapping techniques, the technical problems of deviation between the initial virtual envelope ellipsoid and the actual overall shape of the stack, the inability of traditional algorithms to perform fine discretization representation of continuous surfaces, the lack of quantitative analysis methods for local curvature changes on the stack surface, and the inability to convert irregular geometric deformations of the stack into structured data are overcome. Thus, a target virtual space envelope ellipsoid that closely matches the actual overall shape of the stack is obtained, realizing fine discretization representation of the continuous surface of the stack and accurately quantifying the local geometric deformation distribution of each region on the stack surface.

[0138] In a preferred embodiment of the present invention, step 4 above may include:

[0139] Step 4.4: Receive the curvature gradient distribution matrix and perform singular value orthogonal decomposition (SVD) on it to extract the principal singular value components and their corresponding orthogonal characteristic direction basis vectors. Specifically, this includes: receiving the generated curvature gradient distribution matrix, where the number of rows equals the number of latitudinal sampling points, the number of columns equals the number of meridional sampling points, and each element corresponds to the curvature gradient value of a grid node on the stack surface. Perform SVD on the curvature gradient distribution matrix, decomposing it into the product of a left orthogonal matrix, a diagonal singular value matrix, and a right orthogonal matrix. The decomposition formula is: ;

[0140] in, The curvature gradient distribution matrix, It is a left orthogonal matrix. It is a diagonal singular value matrix. This is the transpose of the right orthogonal matrix. The diagonal elements of the diagonal singular value matrix are all the singular values ​​of the curvature gradient distribution matrix, and all singular values ​​are non-negative. All singular values ​​are rearranged in descending order, and the column vector order of the left and right orthogonal matrices is adjusted accordingly. The first singular value after rearrangement is extracted as the principal singular value component, which represents the overall intensity of local geometric deformation on the stack surface. The first column vector corresponding to the principal singular value component in the left orthogonal matrix is ​​extracted as the orthogonal characteristic direction basis vector, which represents the spatial direction of the dominant geometric deformation on the stack surface.

[0141] Step 4.5: Compare the spatial angles between the orthogonal characteristic direction basis vectors and the axes of the preset standard reference coordinate system, calculate the deflection angle deviation of each axis, and perform tensor coupling mapping between the deflection angle deviation and the principal singular value components to obtain the spatial attitude offset vector. Specifically, the preset standard reference coordinate system is the standard orthogonal coordinate system of the initial virtual spatial envelope ellipsoid, whose X-axis, Y-axis, and Z-axis correspond to the three principal axis directions of the initial ellipsoid, and the unit vectors of the three axes are respectively... , , The extracted orthogonal feature direction basis vectors are compared with the spatial angles of the three axial unit vectors of the preset standard reference coordinate system. The angle between the orthogonal feature direction basis vectors and each axial unit vector is calculated using the following formula: ;

[0142] in, The orthogonal characteristic direction basis vectors and the first The deflection angle deviation of a unit vector along an axis These are orthogonal characteristic direction basis vectors. For the first A unit vector along the axis, Take 1, 2, and 3 respectively to correspond to the X-axis, Y-axis, and Z-axis.

[0143] The calculated deflection angle deviations along the X, Y, and Z axes are coupled with the principal singular value components using tensor coupling mapping, i.e., multiplying the corresponding components to obtain the attitude offset component for each axis. The attitude offset components along the three axes are then combined in the order of X, Y, and Z to obtain the three-dimensional attitude offset vector.

[0144] Step 4.6: Receive the spatial attitude offset vector, call the robot kinematics forward model to perform coordinate system transformation mapping, and solve the three-dimensional translational and three-dimensional rotational components of the spatial attitude offset vector in the robot end effector reference frame. Specifically, this includes: receiving the spatial attitude offset vector, which is defined in the robot base coordinate system and contains three-dimensional translational and three-dimensional rotational offset components; obtaining the angle values ​​of all joints of the robot at the current moment; and calling the pre-calibrated robot kinematics forward model.

[0145] The robot's forward kinematics model is constructed based on standard DH parameters. The homogeneous transformation matrix of each joint relative to the previous joint is calculated sequentially. Each homogeneous transformation matrix contains the rotational and translational components of that joint. The homogeneous transformation matrices of all joints are multiplied sequentially from the base to the end effector to obtain the total homogeneous transformation matrix from the robot's base coordinate system to the end effector reference system. The calculation formula is as follows: ;

[0146] in, Let be the total homogeneous transformation matrix from the robot's base coordinate system to the end effector's reference system. to For the first to the second The homogeneous transformation matrix of each joint relative to the previous joint. Let be the total number of joints of the robot. The total homogeneous transformation matrix is ​​a fourth-order square matrix. The first three rows and first three columns are rotation submatrices, the first three rows and fourth column are translation subvectors, and the elements of the fourth row are fixed as 0, 0, 0, 1.

[0147] Calculate the 3D translation component of the spatial attitude offset vector in the robot's end effector reference frame. Convert the translation component of the spatial attitude offset vector in the robot's base coordinate system to homogeneous coordinate form, i.e., add an element with a value of 1 to the end of the 3D translation vector. Multiply this homogeneous coordinate by the overall homogeneous transformation matrix to obtain the homogeneous translation coordinate in the end effector reference frame. The calculation formula is as follows: ;

[0148] in, The homogeneous translation coordinates are in the end effector reference frame. The homogeneous translation coordinates are in the robot's base coordinate system. The obtained homogeneous translation coordinates are converted into three-dimensional coordinates by removing the last element with a value of 1, resulting in the three-dimensional translation components of the spatial attitude offset vector in the robot's end effector reference frame.

[0149] The three-dimensional rotation components of the spatial attitude offset vector in the robot's end effector reference frame are calculated. The rotation part of the spatial attitude offset vector in the robot's base coordinate system is converted into a corresponding rotation matrix. The rotation submatrix in the total homogeneous transformation matrix is ​​extracted, and its transpose is calculated. The transpose of the rotation submatrix is ​​multiplied by the rotation matrix in the base coordinate system, and then multiplied by the rotation submatrix again to obtain the rotation matrix in the end effector reference frame. The calculation formula is as follows: ;

[0150] in, The rotation matrix is ​​in the end effector reference frame. It is a rotation submatrix of the total homogeneous transformation matrix. It is the transpose of the rotated submatrix. This is the rotation matrix in the robot's base coordinate system. The rotation matrix is ​​then converted into Euler angles in the order of rotation around the X-axis, Y-axis, and Z-axis to obtain the three-dimensional rotation components of the spatial attitude offset vector in the robot's end effector reference frame.

[0151] Step 4.7 involves fusing and unifying the dimensions of the 3D translation and rotation components to obtain the pose offset compensation parameters. Specifically, this includes receiving the 3D translation and rotation components in the robot's end effector reference frame. The dimension of the 3D translation component is meters, and the dimension of the 3D rotation component is radians, each retaining three decimal places. Dimension unification is then performed to maintain the same dimension for both components, ensuring consistent numerical precision across all parameters. A parameter fusion operation is then performed, arranging the 3D translation components first in the order of X-axis, Y-axis, and Z-axis, and then arranging the 3D rotation components in the order of rotation angle around the X-axis, rotation angle around the Y-axis, and rotation angle around the Z-axis. These components are then concatenated to form a six-dimensional vector, which is the final pose offset compensation parameter. The first three elements correspond to the position compensation amount in the end effector reference frame, and the last three elements correspond to the attitude compensation amount in the end effector reference frame.

[0152] In this embodiment of the invention, the following techniques are employed: singular value orthogonal decomposition of curvature gradient distribution matrix, spatial angle comparison between orthogonal characteristic direction basis vectors and the axes of a preset standard reference coordinate system, tensor coupling mapping of deflection angle deviation and principal singular value components, coordinate system transformation mapping of robot kinematics forward model, and parameter fusion and dimension unification of three-dimensional translation and rotation components. This overcomes the technical problems of traditional algorithms, such as the inability to extract the dominant geometric deformation direction and quantify the deformation degree from complex curvature gradient distributions, the inability to convert attitude offsets in the global coordinate system into reference system parameters recognizable by the robot's end effector, and the inability to unify translation and rotation parameters of different dimensions into directly callable compensation parameters. This allows for accurate calculation of spatial attitude offsets caused by irregular geometric deformation of the pallet, obtaining standardized pose offset compensation parameters, and achieving adaptive compensation for local defects such as pallet wrapping film wrinkles and box extrusion deformation.

[0153] In a preferred embodiment of the present invention, step 5 above may include:

[0154] Step 5.1: The pose offset compensation parameter is parsed into a spatial pose transformation matrix. The spatially aligned disparity feature tensor is then remapped globally using this matrix to obtain a reference-aligned disparity feature tensor. Specifically, this involves: receiving the pose offset compensation parameter, a six-dimensional vector where the first three elements are the three-dimensional translation components in the end-effector reference frame, and the last three elements are the three-dimensional rotation components. The three-dimensional rotation components are converted into corresponding rotational attitude matrices, in the order of rotation around the X-axis, Y-axis, and Z-axis. The three-dimensional translation components are combined with the rotational attitude matrix to construct a fourth-order homogeneous spatial pose transformation matrix. The first three rows and first three columns of this matrix represent the rotational attitude matrix, the fourth column represents the three-dimensional translation components, and the fourth row contains 0, 0, 0, and 1 elements. The spatially aligned disparity feature tensor is received, and each pixel within the tensor is traversed. For the current pixel, its three-dimensional coordinates in the robot's base coordinate system are calculated based on its pixel coordinates and corresponding depth value. The 3D coordinates are converted to homogeneous coordinates by appending a 1 to the end. These homogeneous coordinates are then multiplied by the spatial pose transformation matrix to obtain the pose-corrected homogeneous coordinates. The corrected homogeneous coordinates are then converted back to 3D coordinates and reprojected onto the imaging plane of the left optical sensor to obtain the corrected pixel coordinates and depth values. This coordinate remapping operation is performed on all pixels, updating all values ​​in the depth channel of the disparity feature tensor while keeping the texture channel values ​​unchanged. Finally, a globally pose-corrected reference-aligned disparity feature tensor is obtained.

[0155] Step 5.2: Based on the reference-aligned disparity feature tensor, calculate the disparity gradient continuity confidence within the local pixel neighborhood. Mark disparity data regions where the confidence falls below a preset occlusion threshold as field-of-view occlusion interference data. Specifically, this includes: receiving the reference-aligned disparity feature tensor and extracting the depth channel data as disparity data. Set the size of the local pixel neighborhood to 5x5 pixels, set the preset occlusion threshold to 0.3, traverse each pixel within the disparity data, construct a 5x5 pixel local neighborhood centered on that pixel, and calculate the disparity gradient values ​​for all pixels within that local neighborhood. The formula for calculating the disparity gradient value is: ;

[0156] in, This is the square of the disparity gradient value of the pixel in the horizontal direction. This is the square of the disparity gradient value in the vertical direction for this pixel. The variance of all disparity gradient values ​​within this local neighborhood is calculated and normalized to a value between 0 and 1. The reciprocal of the normalized variance is taken to obtain the disparity gradient continuity confidence score for this pixel, calculated using the following formula: ;

[0157] in, For pixels disparity gradient continuity confidence, The normalized variance of the disparity gradient value in the local neighborhood of the pixel is calculated by traversing all pixels. Pixel regions with confidence values ​​less than the preset occlusion threshold of 0.3 are marked as field-of-view occlusion interference data. An occlusion mask of the same size as the disparity data is generated, where the value of the occluded region is 0 and the value of the effective region is 1.

[0158] Step 5.3 involves logically stripping the field-of-view occlusion interference data from the reference-aligned disparity feature tensor to obtain the stripped effective feature data. Morphological smoothing filtering is then applied to the stripped effective feature data to converge and extract the surface contour features of the pallet. Specifically, this includes: element-wise multiplying the occlusion mask with the reference-aligned disparity feature tensor; setting all channel values ​​corresponding to the field-of-view occlusion interference data to 0; and performing morphological smoothing filtering on the stripped effective feature data. A 3x3 pixel rectangular structuring element is used to perform a closing operation on the depth channel data to fill small holes in the depth data. A median filter operation is then performed on the depth channel data with a 3x3 pixel filter window to remove isolated noise points in the depth data. A threshold segmentation operation is performed on the repaired depth channel data. Pixels with depth values ​​within the stacking operation distance range are marked as stack area pixels, and the remaining pixels are marked as background pixels, resulting in a binary image of the stack area. An edge detection operation is performed on the binary image of the stack area to extract the outer and inner contours of the stack. The extracted contours are then filtered to remove tiny contours with an area of ​​less than 100 pixels, and the contour with the largest area is retained as the surface contour feature of the stack.

[0159] Step 5.4: Perform equidistant normal spatial sampling along the surface contour features of the pallet to capture the three-dimensional coordinates and surface normal attributes of each sampling node. Reorganize the data according to the spatial topology sequence to obtain the initial set of discrete spatial representations of the pallet. Specifically, this includes: directly extracting the surface contour features of the pallet, calculating the total pixel length of the contour, setting the spatial spacing of equidistant sampling to 5 mm, calculating the total number of sampling points based on the total length of the contour and the sampling spacing, selecting sampling points sequentially along the direction of the pallet surface contour according to the calculated sampling spacing, and for each sampling point, calculating the tangent direction on the contour and rotating the tangent direction counterclockwise by 90 degrees to obtain the surface normal direction of the point.

[0160] Moving one pixel inward along the surface normal direction, the actual position of the pallet surface corresponding to the sampling point is obtained. Based on the reference alignment disparity feature tensor, the three-dimensional coordinates and surface normal vector corresponding to the actual position are obtained. The above operation is completed by traversing all sampling points, capturing the three-dimensional coordinates and surface normal attributes of all sampling nodes. According to the traversal order of the pallet surface contour, the three-dimensional coordinates and surface normal attributes of all sampling nodes are structurally reorganized to form an ordered dataset.

[0161] In this embodiment of the invention, based on the global coordinate remapping technique of parsing the pose offset compensation parameter into a spatial pose transformation matrix, the confidence calculation technique of disparity gradient continuity in local pixel neighborhoods, the field-of-view occlusion interference data identification and logical stripping technique based on confidence threshold, the morphological smoothing filtering and repair technique of effective feature data, and the equidistant normal spatial sampling and spatial topology sequence structured recombination technique of the pallet surface contour, the invention overcomes the technical problems of disparity feature spatial reference offset caused by irregular geometric deformation of the pallet, the inability of traditional algorithms to effectively distinguish effective pallet features from field-of-view occlusion interference data such as fork hole occlusion and shuttle vehicle passage occlusion, the incomplete pallet surface contour extraction with noise and burrs caused by residual occlusion data, and the inability to convert continuous contour features into structured three-dimensional discrete representation data. Thus, the invention achieves global reference alignment of disparity feature tensors, accurately filters out various field-of-view occlusion interference data, obtains pure, complete and smooth pallet surface contour features, and constructs an initial set of pallet spatial discrete representations containing three-dimensional coordinates and surface normal attributes.

[0162] In a preferred embodiment of the present invention, step 6 above may include:

[0163] Step 6.1: Receive the initial set of discrete representations of the pallet space as the initial set of discrete representations of the pallet space. Perform normal vector consistency clustering analysis on each sampling node within the initial set of discrete representations of the pallet space to divide the pallet load-bearing layer and the lateral support surface into parallel plane feature clusters. Specifically, this includes: receiving the generated initial set of discrete representations of the pallet space, using it as the initial set of discrete representations of the pallet space. Each sampling node in this set contains three-dimensional coordinate information and corresponding surface normal vector information. Perform normal vector consistency clustering analysis on each sampling node within the initial set of discrete representations of the pallet space. Initialize the cluster label array, setting the initial cluster label of all sampling nodes to -1, indicating unclassified. The initial cluster count is 0. The process iterates through each sampling node in the initial set of discrete representations of the pallet space. If the cluster label of the current sampling node is -1, a new cluster category is created, the cluster count is incremented by 1, the cluster label of the current sampling node is set to the new cluster number, and the normal vector of the sampling node is used as the initial center normal vector of the cluster. For the remaining unclassified sampling nodes, the angle between the normal vector of the sampling node and the center normal vectors of all existing clusters is calculated sequentially. If the angle of any cluster is less than a preset threshold of 5 degrees, the cluster label of the current sampling node is set to the cluster number, and the center normal vector of the cluster is updated to the average of the normal vectors of all sampling nodes in the cluster. If the angle of all clusters is greater than 5 degrees, a new cluster category is created, and the above operation of creating new clusters is repeated. After traversing all sampling nodes to complete the clustering, the clusters are classified according to the direction of the center normal vectors of each cluster. Clusters with an angle of less than 15 degrees between the central normal vector and the vertical direction are classified as planar feature clusters of the load-bearing layer of the pallet, and clusters with an angle of less than 15 degrees between the central normal vector and the horizontal direction are classified as planar feature clusters of the lateral support surface of the pallet.

[0164] Step 6.2: Based on the parallel planar feature clusters, the centroid coordinates and unit normal vectors of each planar feature are solved using a least-squares fitting algorithm. The spatial vertical distance and the cosine of the angle between the normals of adjacent planar features are calculated to obtain the inter-layer stacking geometric correlation parameters. Specifically, this includes: based on the partitioned planar feature clusters of all load-bearing layers and lateral support surfaces, a least-squares planar fitting operation is performed on each planar feature cluster in sequence. For any planar feature cluster, let it contain... There are 1 sampling node, and the three-dimensional coordinates of each sampling node are ( , , ),in From 1 to The general equation of a plane is: ;

[0165] in , , , Let be the coefficients of the plane equation, satisfying a² + b² + c² = 1. Construct a least-squares fitting objective function to minimize the sum of squared distances from all sampling nodes to the plane. The objective function is: ;

[0166] Calculate the partial derivatives of the objective function with respect to a, b, c, and d, and set the partial derivatives equal to 0 to obtain the coefficients a, b, c, and d of the plane equation. Calculate the centroid coordinates of the feature cluster in the plane, which are the average of the three-dimensional coordinates of all sampling nodes within the cluster. The calculation formula is as follows: ;

[0167] in( , , Let (a, b, c) be the coordinates of the centroid of the planar feature cluster. Normalize the coefficients (a, b, c) of the planar equation to obtain the unit normal vector of the planar feature. Calculate the spatial vertical spacing between all adjacent load-bearing layer planar features sequentially. The formula for calculating the spatial vertical spacing is: ;

[0168] in The vertical distance between two adjacent load-bearing floor planes. and These are the constant terms in the equations of the planes of two adjacent load-bearing layers. The cosine of the normal angle between all adjacent planar features is calculated sequentially. The cosine of the normal angle is the dot product of the unit normal vectors of the two planes, calculated using the following formula: ;

[0169] in and These are the unit normal vectors of the two planar features, The normal angle between adjacent planar features is used. All calculated spatial vertical spacings are integrated with the cosine of the normal angle to obtain the inter-layer stacking geometric correlation parameters of the pallet.

[0170] Step 6.3: Based on the interlayer stacking geometric correlation parameters and the coordinates of each central centroid, construct a spatial topological constraint matrix for vertical stacking constraints, side alignment constraints, and bottom support constraints of the stack. Specifically, this includes: constructing a spatial topological constraint matrix containing the three types of constraints based on the obtained interlayer stacking geometric correlation parameters and the coordinates of the central centroids of each planar feature cluster.

[0171] The vertical stacking constraint requires that the unit normal vector of all load-bearing layer planes be parallel to the unit vector of the Z-axis of the stack's reference coordinate system. For each load-bearing layer plane feature, a constraint equation is constructed, which is in the form that the cross product of the unit normal vector of that plane and the unit vector of the Z-axis of the stack's reference coordinate system is equal to zero.

[0172] The stack side alignment constraint requires that the unit normal vector of all lateral support planes of the stack be parallel to the unit vector of the X-axis or Y-axis of the stack's reference coordinate system. For each lateral support plane feature, a constraint equation is constructed, which is in the form that the cross product of the unit normal vector of the plane and the corresponding axial unit vector of the stack's reference coordinate system is equal to zero.

[0173] The bottom support constraint of the stack requires that the center of mass of the bottom load-bearing layer plane lie on the XY plane of the stack's reference coordinate system. A constraint equation is constructed in the form that the Z-coordinate of the center of mass of the bottom load-bearing layer plane is equal to zero.

[0174] All the above constraint equations are transformed into a system of linear equations, and the coefficient matrix of the linear equation system is used as the spatial topological constraint matrix of the stack. The number of rows in this matrix is ​​equal to the number of all constraint equations, and the number of columns is equal to the number of stack pose parameters, i.e., 6, which correspond to the translation parameters and rotation parameters around the X, Y, and Z axes, respectively.

[0175] Step 6.4 involves performing eigenvalue decomposition and homogeneous coordinate transformation on the spatial topological constraint matrix to extract the spatial translation vector and rotation quaternion that minimize the matrix residuals, and inverting to obtain the initial pose parameters of the pallet reference coordinate system. Specifically, this includes performing eigenvalue decomposition on the constructed spatial topological constraint matrix, decomposing the spatial topological constraint matrix into an eigenvalue matrix and a corresponding eigenvector matrix, arranging all eigenvalues ​​in ascending order, extracting the smallest non-zero eigenvalue and the eigenvector corresponding to that eigenvalue, which is the pose parameter vector in homogeneous coordinate form that minimizes the sum of squares of the residuals of all constraint equations.

[0176] The pose parameter vector in homogeneous coordinates is converted to Cartesian coordinates. The first three elements are extracted as the spatial translation vector of the pallet reference coordinate system relative to the robot base coordinate system. The last four elements of the homogeneous coordinate pose parameter vector are extracted and normalized to obtain the rotation quaternions of the pallet reference coordinate system relative to the robot base coordinate system. The rotation quaternions are then converted to the corresponding Euler angles to obtain the rotation angles of the pallet reference coordinate system around the X, Y, and Z axes. The spatial translation vector and the rotation Euler angles are integrated to obtain the initial pose parameters of the pallet reference coordinate system.

[0177] In this embodiment of the invention, the following techniques are employed: dividing the load-bearing layers and lateral support surfaces of the pallet into parallel plane feature clusters based on normal vector consistency clustering analysis; solving the centroid coordinates and unit normal vectors of each plane feature center based on least squares fitting; calculating the inter-layer stacking geometric correlation parameters by the vertical distance between adjacent plane feature spaces and the cosine value of the angle between the normals; constructing a spatial topological constraint matrix by integrating vertical stacking constraints, side alignment constraints, and bottom support constraints; and solving the initial pose parameters by eigenvalue decomposition of the constraint matrix and homogeneous coordinate transformation. These techniques overcome the technical problems of traditional algorithms, such as the susceptibility to noise interference when directly fitting pose from scattered point clouds, low pose estimation accuracy due to the lack of prior knowledge of the inherent stacking structure of the pallet, susceptibility to local point cloud deficiencies due to the lack of multi-dimensional topological constraints, and large computational load and poor real-time performance of direct point cloud registration. Furthermore, these techniques accurately extract the core structural features of the pallet and filter out irrelevant noise, fully utilizing the stacking geometric priors of the pallet to construct a strong constraint system, and rapidly inverting to obtain high-precision initial pose parameters of the pallet reference coordinate system.

[0178] In a preferred embodiment of the present invention, step 7 above may include:

[0179] Step 7.1: Receive the initial pose parameters of the pallet reference coordinate system, map the initial pose parameters to the camera imaging model of the dual-side optical sensors, establish the forward projection relationship between the pixel coordinates of the pallet feature points and the three-dimensional spatial coordinates, and obtain the initial reprojection residual reference. Specifically, this includes: receiving the initial pose parameters of the pallet reference coordinate system, which includes the three-dimensional translation and three-dimensional rotation components of the pallet reference coordinate system relative to the robot base coordinate system; extracting three physical reference points, namely the intersection reference of the inner wall of the pallet bottom fork tooth channel, the centroid reference of the interlayer wrapping film hot melt overlap area, and the vertex reference of the corner anti-collision protection of the outer packaging box; the three-dimensional coordinates of these three reference points in the pallet reference coordinate system are known quantities, and the actual pixel coordinates on the imaging planes of the left and right optical sensors are also known quantities.

[0180] The 3D coordinates of each reference point in the pallet reference coordinate system are converted to homogeneous coordinates by appending a 1 to the end of the 3D coordinates. This homogeneous coordinate is then multiplied by the homogeneous transformation matrix from the pallet reference coordinate system to the robot reference coordinate system to obtain the homogeneous coordinates of the reference point in the robot reference coordinate system. By reading the extrinsic parameter matrix of the left optical sensor relative to the robot reference coordinate system, the homogeneous coordinates of the reference point in the robot reference coordinate system are multiplied by this extrinsic parameter matrix to obtain the homogeneous coordinates of the reference point in the left optical sensor camera coordinate system. This homogeneous coordinate is then converted to 3D coordinates and multiplied by the intrinsic parameter matrix of the left optical sensor to obtain the projected pixel coordinates of the reference point on the imaging plane of the left optical sensor. The above coordinate transformation and projection process is repeated to calculate the projected pixel coordinates of each reference point on the imaging plane of the right optical sensor. The Euclidean distance between the projected pixel coordinates and the actual detected pixel coordinates of each reference point is calculated to obtain the reprojection residual of that reference point. The sum of the squares of the reprojection residuals of all reference points on the left and right images is accumulated to obtain the initial reprojection residual reference. The calculation formula is as follows: ;

[0181] in, As the initial reprojection residual reference, For the first The actual pixel coordinates of each reference point on the left image. For the first The projection pixel coordinates of the reference point on the left image For the first The actual pixel coordinates of each reference point on the right image. For the first The projection pixel coordinates of the reference point on the right image.

[0182] Step 7.2 involves jointly encapsulating the initial reprojection residual benchmark with the binocular epipolar geometric constraints to construct the objective optimization function for multi-source error fusion. Specifically, this includes: reading the fundamental matrix of the calibrated binocular vision system; constructing the binocular epipolar geometric constraint residual term; and calculating the distance from each benchmark point on the left image to the corresponding epipolar line on the right image. ;

[0183] in, For the left image The distance from each reference point to the corresponding epipolar line in the right image The fundamental matrix of a binocular vision system. and The coefficients of the polar equations are respectively Quantity and Components. Similarly, calculate the distance from each reference point on the right image to the corresponding epipolar line in the left image. .

[0184] The epipolar constraint residual term is obtained by summing the squares of the epipolar distances of all reference points. The calculation formula is as follows: ;

[0185] in, This represents the epipolar constraint residual term. The initial reprojection residual benchmark and the epipolar constraint residual term are weighted and summed to construct the objective optimization function for multi-source error fusion. The calculation formula is as follows: ;

[0186] in, Optimize the function for the objective. ω1 is the weighting coefficient for the reprojection residual term, with a value of 1.0, and ω2 is the weighting coefficient for the epipolar constraint residual term, with a value of 0.5.

[0187] Step 7.3: Based on the objective optimization function, a nonlinear least squares iterative algorithm is used to linearize the translation and rotation components in the initial pose parameters using the Jacobian matrix. The pose estimate is updated frame by frame to eliminate the accumulated error introduced by dynamic motion. Specifically, this includes: initializing the current pose parameters to the obtained initial pose parameters of the pallet reference coordinate system; solving the objective optimization function using a nonlinear least squares iterative algorithm; calculating the partial derivatives of the objective optimization function with respect to the six pose parameters at the current pose parameters; constructing a Jacobian matrix, where each row of the Jacobian matrix corresponds to a residual term and each column corresponds to a partial derivative of a pose parameter; performing a first-order Taylor expansion of the nonlinear objective optimization function at the current pose parameters to obtain a linearized approximate expression; solving this system of linear equations to obtain the increment vector of the pose parameters; adding the increment vector of the pose parameters to the current pose parameters to obtain the updated pose parameters; and recalculating the objective optimization function value corresponding to the updated pose parameters. Repeat the Jacobian matrix construction, linearization solution, and parameter update process described above until the change in the objective optimization function value is less than a preset threshold, or the number of iterations reaches the preset maximum number of iterations. For continuously acquired image frame sequences, use the pose parameters finally converged in the previous frame as the initial pose parameters for the current frame, and execute the complete nonlinear least squares iterative optimization process described above. By updating the pose estimation values ​​frame by frame, the cumulative error introduced by the relative motion between the robot and the stack is effectively eliminated.

[0188] Step 7.4: When the convergence gradient of the objective optimization function satisfies the preset iteration stopping condition, lock the pose estimation value of the current frame to obtain the real-time six-degree-of-freedom pose detection result of the pallet in three-dimensional spatial coordinates and three-axis attitude angles. Specifically, this includes setting the preset iteration stopping condition: when the change in the objective optimization function value between two adjacent iterations is less than or equal to 0.0001, or when the number of iterations reaches the preset maximum number of iterations of 20, the iteration evolution process is terminated. When the iteration stopping condition is met, lock the pose parameters obtained in the current iteration.

[0189] The first three elements of the pose parameter are extracted as the three-dimensional spatial coordinate position of the pallet reference coordinate system in the robot base coordinate system. The last three elements of the pose parameter are extracted as the rotation angles of the pallet reference coordinate system around the X-axis, Y-axis, and Z-axis of the robot base coordinate system, i.e., the three-axis attitude angles. The three-dimensional spatial coordinate position and the three-axis attitude angles are integrated to obtain the real-time six-degree-of-freedom pose detection result of the pallet.

[0190] In this embodiment of the invention, the following techniques are employed: mapping initial pose parameters to the camera imaging model to establish the forward projection relationship between the pixel coordinates of the pallet feature points and the three-dimensional spatial coordinates; jointly encapsulating the initial reprojection residual benchmark and the binocular epipolar geometric constraints to construct a multi-source error fusion objective optimization function; combining nonlinear least squares iteration based on the objective optimization function with Jacobian matrix linearization; updating the pose estimation value frame by frame to eliminate dynamic motion accumulation errors; and controlling the convergence gradient threshold to terminate the iteration and output the six-degree-of-freedom pose result. These techniques overcome the technical problems of traditional algorithms, such as unstable solutions due to relying solely on single reprojection residual optimization, insufficient pose estimation accuracy due to not utilizing the inherent constraints of binocular epipolar geometry, pose drift caused by the easy introduction of accumulation errors during dynamic operations, and difficulty in balancing real-time performance and accuracy due to the lack of a reasonable iteration termination mechanism. This achieves joint optimization of multi-source geometric constraints, effectively eliminating pose drift under dynamic conditions, and obtaining high-precision real-time six-degree-of-freedom pose detection results for the pallet, providing accurate pose data for the planning of the end effector approach trajectory of the embodied intelligent robot.

[0191] like Figure 2 As shown, embodiments of the present invention also provide a binocular vision-constrained embodied intelligent machine pallet pose real-time detection system, comprising:

[0192] The acquisition module is used to synchronously acquire continuous image frames of the pallet collected by the dual optical sensors of the embodied intelligent robot, perform optical distortion correction and preliminary parallax estimation on the image frame pairs, and obtain spatially aligned parallax feature tensors.

[0193] The fitting module receives the spatially aligned disparity feature tensor, analyzes and locates the physical reference datums of the three pallet areas; it fits the initial virtual spatial envelope ellipsoid with the coordinate vectors of the three physical monitoring datums as constraints, and extracts topological parameters to construct a candidate mapping population.

[0194] The update module is used to calculate the individual spatial fitting residuals based on the candidate mapping population and map them to fitness values, and then iteratively update the configuration parameters until they converge to the final geometric mapping configuration.

[0195] The calculation module is used to apply the final configuration to the initial ellipsoid to obtain the target ellipsoid; to perform manifold discrete sampling on the target ellipsoid to obtain the coordinate sequence and input it into the normal curvature gradient field to obtain the curvature gradient distribution matrix; to orthogonally decompose the curvature gradient distribution matrix to extract the attitude offset vector and solve for the pose offset compensation parameters.

[0196] The mapping module is used to guide the parallax feature space mapping based on the pose offset compensation parameter, filter out the field of view occlusion interference data to converge and extract the surface contour features of the stack, and obtain the initial set of discrete spatial representations of the stack.

[0197] The module is used to construct the spatial topological constraint matrix of the stacked surface of the stack based on the initial set of discrete representations of the stack space, and to inversely derive the initial pose parameters of the stack reference coordinate system.

[0198] The correction module is used to input the initial pose parameters into the binocular vision geometric constraint solution process, dynamically compensate and correct the initial pose parameters, and obtain the real-time six-degree-of-freedom pose detection results of the stack.

[0199] It should be noted that this system is a system corresponding to the above method. All implementation methods in the above method embodiments are applicable to this embodiment and can achieve the same technical effect.

[0200] Example: Real-time detection experiment of stack pose of embodied intelligent machine under binocular vision constraint;

[0201] This embodiment was completed in a large logistics warehousing center. It uses the binocular vision-constrained embodied intelligent machine pallet pose real-time detection method proposed in this application to systematically detect and analyze standard pallet pallets.

[0202] Experimental Scenario: Automated sorting area of ​​a logistics warehousing center; stacks of cardboard boxes loaded on standard wooden pallets (1200mm×1000mm×150mm); stack height 800-1200mm; single stack weight 500-800kg. Experimental Environment: Indoor lighting conditions 200-500 lux, temperature 15-25℃, relative humidity 40%-70%.

[0203] Experimental equipment: The embodied intelligent robot is equipped with a binocular stereo vision system (baseline distance 120mm, focal length 8mm, resolution 1920×1080, frame rate 30fps), and is equipped with an NVIDIA Jetson AGX Orin edge computing unit (computing power 275TOPS). The robot has an arm span of 1.8m and a load capacity of 10kg.

[0204] Experimental conditions: Binocular camera calibration parameters: focal length f x =f y =1200 pixels, principal point c x =960 pixels, c y =540 pixels, radial distortion coefficients k1=0.12, k2=-0.05, tangential distortion coefficients p1=0.001, p2=-0.002. The disparity calculation adopts the semi-global matching (SGM) algorithm, the cost aggregation window size is 5×5, and the penalty coefficients P1=10, P2=120.

[0205] Geometric mapping configuration convergence, ellipsoidal correction, and pose offset compensation;

[0206] The individual spatial fitting residuals are calculated based on the candidate mapping population and mapped to fitness values. The configuration parameters are iteratively updated until convergence to the final geometric mapping configuration. Then, the ellipsoid is corrected and the pose offset compensation parameters are solved.

[0207] Spatial fitting residual calculation: For each candidate mapping individual, calculate the spatial fitting residual between its predicted pallet surface and the actual detected point cloud;

[0208] Configuration parameter iterative update: Configuration parameters are updated based on fitness values ​​using an elite retention strategy. The top 20% of elite individuals are retained in each generation, with the remaining individuals generated through crossover and mutation. Iteration termination condition: The optimal fitness change is <0.001 for 10 consecutive generations or the maximum number of iterations (100 generations) is reached.

[0209] Ellipsoid Correction: The final geometric mapping configuration is applied to the initial ellipsoid, and the ellipsoid shape is corrected by mapping the normal curvature gradient field. The corrected target ellipsoid achieves a 96.5% fit with the actual stack surface.

[0210] Pose offset compensation: Perform orthogonal decomposition on the curvature gradient distribution matrix to extract the attitude offset vector. The pose offset compensation parameters include: position offset (ΔX, ΔY, ΔZ) and attitude offset angle (ΔRoll, ΔPitch, ΔYaw).

[0211] Field of view occlusion interference filtering: Based on pose offset compensation parameters, parallax feature spatial mapping is guided to filter out field of view occlusion interference data. The accuracy of occlusion region identification is 92.3%, effectively improving the extraction quality of pallet surface contour features.

[0212] Figure 3 The trend of pose detection accuracy with detection distance is shown. Within the detection range of 0.5-4.0m, the position error increases from 1.2mm to 7.2mm, and the attitude error increases from 0.15° to 1.1°. Within the typical working distance range of 1.5-2.5m, the position error is 2.0-3.5mm, and the attitude error is 0.25°-0.48°, meeting the requirements of industrial field applications.

[0213] Figure 4 This demonstrates a 3D spatial distribution simulation of the pallet pose detection. Blue dots represent true pose points, red squares represent detected pose points, and black dashed lines represent detection errors. Within the pallet outline (green dashed box), the detection points are uniformly distributed, the error direction is random, and there is no systematic deviation.

[0214] Binocular vision geometric constraint solution and dynamic compensation correction;

[0215] The initial pose parameters are input into the binocular vision geometric constraint solution process, and the initial pose parameters are dynamically compensated and corrected to obtain the real-time six-degree-of-freedom pose detection results of the pallet.

[0216] Geometric constraint solution: Based on the principle of binocular vision triangulation, the 3D pose of the pallet is calculated using corresponding feature points in the left and right viewpoint images. The triangulation baseline distance is 120mm, the parallax accuracy is 0.1 pixels, and the corresponding depth accuracy is 0.8mm at 1m and 7.2mm at 3m.

[0217] Dynamic compensation correction: Kalman filtering is used for dynamic compensation to address image blurring and pose drift caused by robot motion. The state vector includes the stack pose (6D) and pose change rate (6D), and the observation vector is the visual detection result. The filter update frequency is 30Hz, synchronized with the image frame rate.

[0218] Six-DOF Pose Output: The final output shows the six-DOF pose of the pallet relative to the robot's base coordinate system, including:

[0219] Location (X, Y, Z): The three-dimensional coordinates of the geometric center of the stack in the base coordinate system, in mm.

[0220] Attitude (Roll, Pitch, Yaw): The rotation angle of the stack around the X, Y, and Z axes of the base coordinate system, in degrees.

[0221] Detection performance: Under standard test conditions (distance 2.0m, illumination 300 lux), pose detection accuracy: position error ±2.8mm, attitude error ±0.35°; detection latency: 45ms (image acquisition 30ms + calculation 15ms); detection success rate: 98.5%.

[0222] Figure 5 The performance of the proposed method was compared with that of traditional monocular vision methods and LiDAR methods on five-dimensional evaluation metrics. The proposed method significantly outperforms the comparative methods in terms of position detection accuracy (96.5%), attitude detection accuracy (94.2%), real-time detection (92.8%), occlusion robustness (89.5%), and overall F1 score (93.8%).

[0223] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for real-time detection of the pose of a stack of goods in an embodied intelligent machine under binocular vision constraints, characterized in that, The method includes: Simultaneously acquire continuous image frames of the pallet from the dual optical sensors of the embodied intelligent robot, perform optical distortion correction and preliminary parallax estimation on the image frame pairs, and obtain spatially aligned parallax feature tensors. The system receives spatially aligned disparity feature tensors, resolves and locates physical reference datums for three pallet regions, fits an initial virtual spatial envelope ellipsoid using the three-dimensional coordinate vectors of the three physical reference datums as constraints, extracts topological parameters to construct a candidate mapping population, including: The system receives spatially aligned parallax feature tensors, separates the depth channel and texture channel through multi-scale feature decoupling operations, locates the three-dimensional coordinate extreme values ​​of the intersection reference of the inner wall of the fork tooth channel at the bottom of the pallet in the depth channel, extracts the gray-scale gradient center of the centroid reference of the hot-melt overlap area of ​​the interlayer wrapping film in the texture channel, and identifies the edge curvature change position of the vertex reference of the corner anti-collision protection of the outer packaging box, thus obtaining a two-dimensional pixel coordinate set of the physical reference reference. The two-dimensional pixel coordinate set of the physical reference datum is back-projected onto the robot base coordinate system, and spatial coordinate alignment is performed to obtain the three-dimensional coordinate vectors of the pallet bottom fork tooth hole inner wall intersection datum, the centroid datum of the interlayer wrapping film hot melt overlap area, and the vertex datum of the outer packaging box corner anti-collision corner guard. Using the three-dimensional coordinate vectors of three physical reference datums as spatial geometric constraint primitives, the least squares spatial surface iterative fitting operation is performed to construct the initial virtual spatial envelope ellipsoid. The axial scaling factor, eccentricity, and spatial orientation angle of the initial virtual spatial envelope ellipsoid are extracted as basic topological parameters. The basic topological parameters are expanded by multidimensional spatial random perturbation to obtain multiple sets of parameter combinations covering the local optimal neighborhood, and a candidate mapping population is constructed. Based on the candidate mapping population, the individual spatial fitting residuals are calculated and mapped to fitness values. The configuration parameters are iteratively updated according to these values ​​until convergence to the final geometric mapping configuration. The final configuration is applied to the initial ellipsoid to obtain the target ellipsoid; the target ellipsoid is subjected to manifold discrete sampling to obtain the coordinate sequence and input into the normal curvature gradient field to obtain the curvature gradient distribution matrix; the curvature gradient distribution matrix is ​​orthogonally decomposed to extract the attitude offset vector, and the pose offset compensation parameter is calculated. Based on the pose offset compensation parameter, the parallax feature space mapping is guided, and the field of view occlusion interference data is filtered out to converge and extract the surface contour features of the stack, thus obtaining the initial set of discrete spatial representations of the stack. Based on the initial set of discrete representations of the stack space, a spatial topological constraint matrix of the stack surface is constructed, and the initial pose parameters of the stack reference coordinate system are derived. The initial pose parameters are input into the binocular vision geometric constraint solution process, and the initial pose parameters are dynamically compensated and corrected to obtain the real-time six-degree-of-freedom pose detection results of the pallet.

2. The method for real-time detection of the pose of a stack of goods in an embodied intelligent machine under binocular vision constraints according to claim 1, characterized in that, Simultaneously acquire consecutive image frames of the pallet from the dual optical sensors of the embodied intelligent robot, perform optical distortion correction and preliminary disparity estimation on the image frame pairs, and obtain spatially aligned disparity feature tensors, including: Based on a unified timestamp, left-view image frames and right-view image frames of the stack area are collected to construct a continuous image frame pair sequence; Receive a sequence of consecutive image frames, perform radial distortion demapping and tangential distortion compensation operations on each frame in the sequence, and obtain a standard image pair after optical distortion correction. Perform a reprojection alignment operation on the standard image pair to make the left viewpoint image row and the right viewpoint image row strictly coplanar, eliminate the vertical offset in the disparity calculation, and obtain the reprojected aligned image pair; Pixel-level disparity matching calculation based on cost volume aggregation is performed on the reprojected and aligned image pairs to obtain an initial disparity map. The initial disparity map and the texture channels of the optically distorted standard image pairs are spatially stacked along the feature dimension to obtain a spatially aligned disparity feature tensor.

3. The method for real-time detection of the pose of a stack of goods in an embodied intelligent machine with binocular vision constraints according to claim 2, characterized in that, Based on the candidate mapping population, the spatial fitting residuals of individuals are calculated and mapped to fitness values. Configuration parameters are iteratively updated accordingly until convergence to the final geometric mapping configuration, including: Traverse each group of mapping individuals in the candidate mapping population, calculate the Euclidean distance deviation between the virtual envelope obtained by each mapping individual and the three-dimensional coordinate vectors of the three physical reference bases, and sum the Euclidean distance deviations to obtain the individual spatial fitting residual; The individual spatial fitting residuals are input into the nonlinear mapping function and subjected to inverse weighting and normalization to obtain the fitness value of the geometric fit of each mapping individual. High-quality mapping individuals are selected based on the distribution gradient of fitness values. Crossover and random perturbation operations are performed on the configuration parameters of the high-quality mapping individuals to obtain the next generation of updated population. Replace the current candidate mapping population with the next generation of updated population, and iteratively execute the individual spatial fitting residual calculation and fitness value mapping process. When the fitness value change in adjacent iterations converges to the preset threshold range, terminate the iterative evolution, extract the parameters of the mapping individual with the highest fitness in the current population, and output them as the final geometric mapping configuration.

4. The method for real-time detection of the pose of a stack of goods in an embodied intelligent machine with binocular vision constraints according to claim 3, characterized in that, The final configuration is applied to the initial ellipsoid to correct it, thus obtaining the target ellipsoid. Performing manifold discretization sampling on the target ellipsoid to obtain a coordinate sequence and inputting it into the normal curvature gradient field mapping yields the curvature gradient distribution matrix, including: Receive the final geometric mapping configuration, extract the spatial translation components, axial scaling ratio and rotation attitude matrix within the configuration, superimpose the spatial translation components, axial scaling ratio and rotation attitude matrix onto the basic geometric parameters of the initial virtual space envelope ellipsoid, perform surface morphology affine correction, and obtain the morphology-optimized target virtual space envelope ellipsoid. A parametric grid is established on the continuous surface of the target virtual space envelope ellipsoid. A manifold discrete sampling operation is performed along the preset meridional and latitudinal step amounts to capture the spatial position information of each grid node. The coordinate sequence is obtained by arranging the grid nodes according to the sampling traversal path. The system receives a coordinate sequence, calculates the first-order tangent vector and second-order normal vector at each sampling node based on a differential geometry algorithm, constructs a local normal curvature calculation neighborhood, and performs gradient difference mapping on the curvature change rate in each neighborhood along the surface normal to obtain the curvature gradient distribution matrix of the local geometric deformation distribution.

5. The method for real-time detection of the pose of a stack of goods in an embodied intelligent machine with binocular vision constraints according to claim 4, characterized in that, The attitude offset vector is extracted by orthogonal decomposition of the curvature gradient distribution matrix, and the pose offset compensation parameters are calculated, including: Receive the curvature gradient distribution matrix, perform singular value orthogonal decomposition on the curvature gradient distribution matrix, and extract the principal singular value components and the corresponding orthogonal characteristic direction basis vectors; The spatial angle between the orthogonal characteristic direction basis vector and the axis of the preset standard reference coordinate system is compared, the deflection angle deviation of each axis is calculated, and the deflection angle deviation is tensor-coupled and mapped with the principal singular value component to obtain the spatial attitude offset vector. Receive the spatial attitude offset vector, call the robot kinematics forward model to perform coordinate system transformation mapping, and solve the three-dimensional translational and three-dimensional rotational components of the spatial attitude offset vector in the robot end effector reference frame; The pose offset compensation parameters are obtained by fusing the three-dimensional translation components and the three-dimensional rotation components and unifying their dimensions.

6. The method for real-time detection of the pose of a stack of goods in an embodied intelligent machine with binocular vision constraints according to claim 5, characterized in that, Based on the pose offset compensation parameter to guide the parallax feature space mapping, and filtering out field-of-view occlusion interference data to convergently extract the surface contour features of the pallet, an initial set of discrete representations of the pallet space is obtained, including: The pose offset compensation parameter is parsed into a spatial pose transformation matrix. The spatially aligned disparity feature tensor is then subjected to global coordinate remapping through the spatial pose transformation matrix to obtain the reference aligned disparity feature tensor. Based on the benchmark aligned disparity feature tensor, the disparity gradient continuity confidence in the local pixel neighborhood is calculated through iteration, and the disparity data region whose confidence falls below the preset occlusion threshold is marked as field of view occlusion interference data. Logical stripping is performed on the field-of-view occlusion interference data from the reference-aligned parallax feature tensor to obtain the stripped effective feature data. Morphological smoothing filtering is then performed on the stripped effective feature data to repair it and converge to extract the surface contour features of the pallet. Equal-interval normal spatial sampling is performed along the surface contour features of the pallet to capture the three-dimensional coordinates and surface normal attributes of each sampling node. The data are then restructured according to the spatial topology sequence to obtain the initial set of discrete spatial representations of the pallet.

7. The method for real-time detection of the pose of a stack of goods in an embodied intelligent machine with binocular vision constraints according to claim 6, characterized in that, Based on the initial set of discrete spatial representations of the pallet, a spatial topological constraint matrix for the pallet stacking surface is constructed, and the initial pose parameters of the pallet reference coordinate system are derived, including: The initial set of discrete representations of the stack space is received as the initial set of discrete representations of the stack space. Normal vector consistency clustering analysis is performed on each sampling node in the initial set of discrete representations of the stack space to divide the parallel plane feature clusters of the stack load-bearing layer and the lateral support surface. Based on the parallel plane feature cluster, the centroid coordinates and unit normal vectors of each plane feature are solved by the least squares fitting algorithm. The spatial vertical distance and cosine value of the angle between the normals between adjacent plane features are calculated to obtain the interlayer stacking geometric correlation parameters. Based on the interlayer stacking geometric correlation parameters and the coordinates of each centroid, a spatial topological constraint matrix is ​​constructed for the vertical stacking constraint, side alignment constraint, and bottom support constraint of the stack. The spatial topological constraint matrix is ​​solved by eigenvalue decomposition and homogeneous coordinate transformation. The spatial translation vector and rotation quaternion that minimize the matrix residual are extracted, and the initial pose parameters of the pallet reference coordinate system are obtained by inversion.

8. The method for real-time detection of the pose of a stack of goods in an embodied intelligent machine with binocular vision constraints according to claim 7, characterized in that, The initial pose parameters are input into the binocular vision geometric constraint solving process, and the initial pose parameters are dynamically compensated and corrected to obtain the real-time six-degree-of-freedom pose detection results of the pallet, including: Receive the initial pose parameters of the pallet reference coordinate system, map the initial pose parameters to the camera imaging model of the dual-side optical sensors, establish the forward projection relationship between the pixel coordinates of the pallet feature points and the three-dimensional spatial coordinates, and obtain the initial reprojection residual reference. The initial reprojection residual benchmark and the binocular epipolar geometric constraints are jointly encapsulated to construct the objective optimization function for multi-source error fusion. Based on the objective optimization function, a nonlinear least squares iterative algorithm is used to solve the Jacobian matrix linearization of the translation and rotation components in the initial pose parameters, and the pose estimation value is updated frame by frame to eliminate the cumulative error introduced by dynamic motion. When the convergence gradient of the objective optimization function satisfies the preset iteration stopping condition, the pose estimation value of the current frame is locked, and the real-time six-degree-of-freedom pose detection result of the pallet with three-dimensional spatial coordinate position and three-axis attitude angle is obtained.

9. A binocular vision-constrained embodied intelligent machine pallet pose detection system, the system implementing the method as described in any one of claims 1 to 8, characterized in that, include: The acquisition module is used to synchronously acquire continuous image frames of the pallet collected by the dual optical sensors of the embodied intelligent robot, perform optical distortion correction and preliminary parallax estimation on the image frame pairs, and obtain spatially aligned parallax feature tensors. The fitting module receives the spatially aligned disparity feature tensor, analyzes and locates the physical reference datums of the three pallet areas; it fits the initial virtual spatial envelope ellipsoid with the coordinate vectors of the three physical monitoring datums as constraints, and extracts topological parameters to construct a candidate mapping population. The update module is used to calculate the individual spatial fitting residuals based on the candidate mapping population and map them to fitness values, and then iteratively update the configuration parameters until they converge to the final geometric mapping configuration. The calculation module is used to apply the final configuration to the initial ellipsoid to obtain the target ellipsoid. Perform manifold discrete sampling on the target ellipsoid to obtain a coordinate sequence and input it into the normal curvature gradient field to obtain the curvature gradient distribution matrix; orthogonally decompose the curvature gradient distribution matrix to extract the attitude offset vector and solve for the pose offset compensation parameters. The mapping module is used to guide the parallax feature space mapping based on the pose offset compensation parameter, filter out the field of view occlusion interference data to converge and extract the surface contour features of the stack, and obtain the initial set of discrete spatial representations of the stack. The module is used to construct the spatial topological constraint matrix of the stacked surface of the stack based on the initial set of discrete representations of the stack space, and to inversely derive the initial pose parameters of the stack reference coordinate system. The correction module is used to input the initial pose parameters into the binocular vision geometric constraint solution process, dynamically compensate and correct the initial pose parameters, and obtain the real-time six-degree-of-freedom pose detection results of the stack.

Images