Positioning method, device, equipment and medium based on depth sensing

Abstract

This invention discloses a method, apparatus, device, and medium for positioning a locator based on depth sensing. The method includes: acquiring two sets of local depth images of the target ball-and-socket structure to be positioned using two depth sensors; converting the two sets of local depth images into three-dimensional point cloud data in a camera coordinate system and stitching them together to form a complete point cloud set of the ball-and-socket structure; extracting a subset of the point cloud of the target ball-and-socket center from the point cloud set of the ball-and-socket structure and calculating the first coordinate of the target ball-and-socket center; converting the first coordinate to a second coordinate using preset hand-eye calibration parameters; calculating the positional deviation between the target ball head and the target ball-and-socket center, and adjusting the pose to achieve precise positioning of the target ball head to the target ball-and-socket center. This invention, through the synergy of dual depth sensor deployment and point cloud stitching algorithm, adapts to the high-precision dynamic docking requirements of flexible support devices, demonstrating a deep integration of hardware characteristics and algorithmic logic.

Description

Technical Field

[0001] This invention relates to the field of aircraft component assembly technology, and in particular to a method, device, equipment and medium for positioning a locator based on depth sensing. Background Technology

[0002] In aircraft assembly and docking processes, to ensure the precise assembly of large components, multiple support points are typically arranged at the bottom of the component, and ball-and-socket support structures are installed at these support points. The guided vehicle uses the ball head at the top of its flexible support device to insert into the ball socket, achieving stable support and movement of the aircraft component. To ensure assembly accuracy, the guided vehicle must precisely position the center of the ball socket to ensure that the ball head is accurately aligned and inserted into the socket, thereby avoiding assembly errors or structural damage caused by positioning deviations.

[0003] Currently, the positioning of the ball joint center mainly relies on various sensors, such as vision sensors, depth sensors, millimeter-wave radar, and lidar. These sensors collect three-dimensional information of the ball joint structure, calculate its geometric center, and then guide the flexible support device to adjust the position of the ball joint to achieve precise docking. However, existing sensor technologies still have problems in practical applications: vision sensors are susceptible to ambient light interference and require supplemental lighting; lidar has a near-range blind zone and is bulky; and millimeter-wave radar lacks the accuracy to meet assembly requirements, seriously affecting the accuracy and reliability of docking large aircraft components. Summary of the Invention

[0004] Based on this, the present invention provides a positioning method, device, equipment and medium based on depth sensing to solve the problems of inaccurate positioning of the ball socket center and poor support stability caused by sensor limitations in existing aircraft assembly and docking processes.

[0005] In a first aspect, embodiments of the present invention provide a positioning method for a locator based on depth sensing, applicable to a scenario where a guide vehicle precisely positions the ball heads at the top of each flexible support device on the roof into the centers of the inwardly recessed ball sockets in the ball socket structures at the bottom of the towed object, including:

[0006] Two depth sensors, respectively deployed on both sides of the target ball head on the target flexible support device, acquire two sets of local depth images of the target ball-and-socket structure to be positioned.

[0007] Based on the intrinsic parameter matrix of the depth sensor, the two sets of local depth images are converted into three-dimensional point cloud data in the camera coordinate system, and based on the pre-calibrated extrinsic parameter matrix between the sensors, the two sets of three-dimensional point cloud data are stitched together into a complete point cloud set of the spherical cavity structure.

[0008] Extract a subset of the point cloud containing the target sphere center from the point cloud set of the sphere structure, and calculate the first coordinate of the target sphere center in the camera coordinate system based on the point cloud subset;

[0009] The first coordinate of the target ball socket center is transformed to the second coordinate in the base coordinate system where the target flexible support device is located by using preset hand-eye calibration parameters;

[0010] Based on the real-time coordinates of the target ball head in the base coordinate system and the second coordinate of the target ball socket center, the positional deviation between the target ball head and the target ball socket center is calculated, and the pose of the target flexible support device is adjusted according to the positional deviation to achieve precise positioning of the target ball head into the target ball socket center. The target flexible support device can dynamically adjust its attitude in three vertical directions.

[0011] Secondly, embodiments of the present invention also provide a locator positioning device based on depth sensing, comprising:

[0012] The local depth image acquisition module is used to acquire two sets of local depth images of the target ball-and-socket structure to be positioned by two depth sensors respectively deployed on both sides of the target ball head on the target flexible support device.

[0013] The point cloud assembly construction module is used to convert two sets of local depth images into three-dimensional point cloud data in the camera coordinate system according to the intrinsic parameter matrix of the depth sensor, and to stitch the two sets of three-dimensional point cloud data into a complete point cloud assembly of the spherical-socket structure based on the pre-calibrated extrinsic parameter matrix between the sensors.

[0014] The first coordinate calculation module is used to extract a subset of point clouds containing the center of the target sphere structure from the point cloud set of the sphere structure, and calculate the first coordinate of the center of the target sphere in the camera coordinate system based on the point cloud subset;

[0015] The second coordinate calculation module is used to transform the first coordinate of the target ball hole center to the second coordinate in the base coordinate system where the target flexible support device is located by using preset hand-eye calibration parameters;

[0016] The pose adjustment module is used to calculate the positional deviation between the target ball head and the target ball socket center based on the real-time coordinates of the target ball head in the base coordinate system and the second coordinate of the target ball socket center. Based on the positional deviation, the module adjusts the pose of the target flexible support device to achieve precise positioning of the target ball head into the target ball socket center. The target flexible support device can dynamically adjust its attitude in three vertical directions.

[0017] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising:

[0018] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform a depth-sensing-based locator positioning method according to any embodiment of the present invention.

[0019] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions, which are used to cause a processor to execute and implement a depth-sensing-based locator positioning method as described in any embodiment of the present invention.

[0020] This invention, through the synergy of dual depth sensor deployment and point cloud stitching algorithms, solves the blind spot problem caused by spherical head occlusion from both hardware layout and data processing perspectives. Compared to single-sensor solutions, it can completely capture the three-dimensional information of the spherical head and socket, laying the foundation for accurate positioning. Simultaneously, it deeply integrates the three-dimensional dynamic adjustment capability of the flexible support device with coordinate transformation and deviation-driven logic. Using hand-eye calibration, it achieves deviation-free mapping across coordinate systems and directly responds to quantified deviations through multi-degree-of-freedom adjustment, forming a "perception-decision-execution" closed loop. This addresses the pain points of disconnect between perception and execution and insufficient adjustment freedom in general technologies. It adapts to the high-precision dynamic docking requirements of flexible support devices, demonstrating the deep empowerment of hardware characteristics and algorithmic logic.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of a locator placement method based on depth sensing according to Embodiment 1 of the present invention;

[0024] Figure 2 This is a schematic diagram illustrating the positional relationship between a guide vehicle and a towed object, applicable to an embodiment of the present invention.

[0025] Figure 3 This is a three-dimensional structural diagram of a ball-and-socket structure applicable to an embodiment of the present invention;

[0026] Figure 4 This is a flowchart of another locator placement method based on depth sensing provided in Embodiment 2 of the present invention;

[0027] Figure 5 This is a schematic diagram of a positioning device based on depth sensing according to Embodiment 3 of the present invention.

[0028] Figure 6 This is a schematic diagram of the structure of an electronic device that implements a depth sensing-based locator positioning method according to an embodiment of the present invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] Example 1

[0032] Figure 1 This is a flowchart of a depth-sensing-based locator placement method provided in Embodiment 1 of the present invention. This embodiment is applicable to adaptive positioning support in high-precision assembly scenarios of large aircraft components. The method can be executed by a depth-sensing-based locator placement device, which can be implemented in hardware and / or software. This device can be configured in the positioning unit of the guide vehicle in an automated aircraft assembly docking system. Figure 1 As shown, the method includes:

[0033] S110. Two sets of local depth images of the target ball head structure to be positioned are acquired by two depth sensors deployed on both sides of the target flexible support device.

[0034] A depth image is essentially an image where each pixel contains its pixel coordinates (u, v) and the distance (depth value) from that point to the sensor. It can be understood as a grayscale image with distance information, serving as the raw data for generating a 3D point cloud. The depth sensor, by emitting and receiving reflected signals, calculates the round-trip time or phase difference to obtain the depth value for each pixel, thus measuring the distance to the surface of a target object. The ball head itself may obstruct the sensor's view of the ball-and-socket structure (the ball head is located at the top of the flexible support device, close to the ball-and-socket structure). Deploying a sensor on only one side would create a blind spot due to obstruction, preventing complete acquisition of information about the ball-and-socket structure. Therefore, deploying two sensors on either side of the ball head allows for the acquisition of partial images of the "left half" and "right half" of the ball-and-socket structure, respectively, laying the foundation for subsequent stitching of complete information. For ease of understanding... Figure 2 A schematic diagram of the positional relationship between a guide vehicle and a towed object is shown. AGV1 and AGV2 represent two guide vehicles. Each guide vehicle has two depth sensors and a rectangular flexible support device distributed on its top. The flexible support device has a ball head on its top, which is used to insert into the center of the ball socket of the ball socket structure and support the towed object.

[0035] S120. Based on the intrinsic parameter matrix of the depth sensor, the two sets of local depth images are converted into three-dimensional point cloud data in the camera coordinate system, and based on the pre-calibrated extrinsic parameter matrix between the sensors, the two sets of three-dimensional point cloud data are stitched together into a complete point cloud set of the spherical-socket structure.

[0036] The pixel coordinates (u, v) of a depth image are two-dimensional planar information, which needs to be obtained from the sensor's intrinsic parameter matrix (including focal length). , Principal point coordinates The parameters are converted into 3D coordinates (X, Y, Z) in the camera coordinate system (a 3D coordinate system with the sensor's optical center as the origin). This process realizes the mapping of 2D pixel information to 3D spatial position, and the generated 3D point cloud data is a set of discrete 3D coordinates of the surface of the spherical-socket structure. The point clouds collected by the two sensors are located in their respective camera coordinate systems, and one set of point clouds needs to be transformed to the coordinate system of the other set of point clouds (or unified to the same coordinate system) through an extrinsic parameter matrix. The extrinsic parameter matrix is ​​obtained through pre-calibration (such as a checkerboard calibration method) and describes the relative position and attitude relationship between the two sensors. The complete spherical-socket structure point cloud set obtained after stitching can fully reflect the 3D morphology of the spherical-socket structure, solving the problem of information loss caused by the limited field of view of a single sensor.

[0037] Optionally, based on the intrinsic parameter matrix of the depth sensor, converting the two sets of local depth images into 3D point cloud data in the camera coordinate system may include:

[0038] Based on the principal point coordinates of the first depth sensor and the focal length of the camera, the pixel coordinates and pixel depth values ​​of each pixel in a set of depth images acquired by the first depth sensor in the pixel coordinate system are converted into first three-dimensional point cloud data in the first camera coordinate system that matches the first depth sensor.

[0039] Based on the principal point coordinates of the second depth sensor and the camera focal length, the pixel coordinates and pixel depth values ​​of each pixel in another set of depth images acquired by the second depth sensor in the pixel coordinate system are converted into second three-dimensional point cloud data in the second camera coordinate system that matches the second depth sensor.

[0040] In this embodiment, the intrinsic parameter matrix of the depth sensor is a matrix describing the optical characteristics and imaging relationship of the depth sensor itself, including parameters such as principal point coordinates and camera focal length, used to establish the mapping relationship between the pixel coordinate system and the camera coordinate system. The camera coordinate system is a three-dimensional coordinate system with the optical center of the depth sensor as its origin, used to describe the three-dimensional spatial position of the object in the sensor's field of view. The three-dimensional point cloud data is a set composed of a large number of discrete three-dimensional coordinate points, each point representing a physical position on the object's surface, reflecting the object's three-dimensional shape as a whole. The pixel coordinate system is a two-dimensional coordinate system with the upper left corner of the depth image as its origin, describing the position of the pixel in the image. The principal point coordinates are the coordinates of the intersection of the camera's optical axis and the imaging plane, serving as the reference point for the transformation between the pixel coordinate system and the camera coordinate system. The camera focal length refers to the distance from the optical center of the sensor to the imaging plane, used to convert pixel distance into actual spatial distance.

[0041] Since depth images are essentially combinations of two-dimensional pixel positions and depth values, they cannot directly reflect the three-dimensional spatial structure of an object. The intrinsic parameter matrix (with its core parameters being the principal point coordinates and focal length) serves as a bridge connecting the pixel coordinate system and the camera coordinate system. Through geometric projection, two-dimensional pixel information can be restored to three-dimensional spatial coordinates. This process is based on the inverse projection principle of a pinhole camera model. For example, in a pinhole camera model, a three-dimensional point is projected onto the depth image through the optical center to form a two-dimensional pixel. Inverse projection, then, uses the pixel coordinates, depth value, and intrinsic parameters to infer the position of the corresponding three-dimensional point in the camera coordinate system.

[0042] Furthermore, based on the pre-calibrated inter-sensor extrinsic matrix, the two sets of 3D point cloud data are stitched together to form a complete point cloud set for the spherical-socket structure, which may include:

[0043] Using a pre-calibrated extrinsic rotation matrix and translation vector between two depth sensors, the second 3D point cloud data in the second camera coordinate system is transformed to the first camera coordinate system;

[0044] Based on the transformation results, the three-dimensional point cloud data under the first camera coordinate system are stitched together to obtain a complete set of point cloud data for the spherical cavity structure.

[0045] The extrinsic parameter matrix between sensors describes the relative position and attitude relationship between two depth sensors. It includes a rotation matrix (representing the directional deviation between the two sensor coordinate systems) and a translation vector (representing the positional offset between the two sensor coordinate systems), used to transform a point cloud in one sensor coordinate system to the other. The extrinsic rotation matrix is ​​a 3×3 matrix in 3D space, used to represent the rotational relationship between the two coordinate systems, rotating a point in one coordinate system around its origin to the direction of the other. The translation vector is a 3×1 vector in 3D space, used to represent the positional difference between the origins of the two coordinate systems, translating a point in one coordinate system to the position of the other. Point cloud stitching refers to the process of transforming multiple sets of point cloud data from different coordinate systems to the same coordinate system and fusing them into a complete point cloud. This is used to compensate for the limitations of a single sensor's field of view and restore the complete 3D shape of an object. Since the two sets of 3D point clouds are located in their respective camera coordinate systems, directly merging them due to the different sensor installation positions will result in point cloud misalignment. The extrinsic parameter matrix unifies the coordinate system through mathematical transformation: first, the rotation matrix corrects the directional deviation between the two coordinate systems, and then the translation vector corrects the positional deviation, ultimately achieving spatial alignment of the point cloud and laying the foundation for stitching.

[0046] Point cloud data in the second camera coordinate system can be labeled with the coordinates of each point as follows: The extrinsic rotation matrix R (3×3) and translation vector The conversion formula can be: +T. Rotation operation This refers to the points in the second coordinate system. Rotate the point around its origin to align it with the first coordinate system (eliminating angular discrepancies between the two sensors); the translation operation (+T) moves the rotated point along the translation vector until its origin coincides with the origin of the first coordinate system (eliminating positional discrepancies between the two sensors). The transformed point is obtained in the first camera coordinate system. The first point cloud is in the same coordinate system as the second point cloud. The two sets of point clouds are already in the same coordinate system and correspond to different local areas of the spherical socket structure. The two sets of point clouds are integrated by a point cloud fusion algorithm. For points in overlapping areas, redundancy can be eliminated by filtering with a distance threshold or by weighted averaging. Points in non-overlapping areas are directly retained. After integration, a point cloud set covering the complete surface of the spherical socket structure is formed, which contains all the three-dimensional coordinate information of the center of the spherical socket and the surrounding structure.

[0047] S130. Extract a subset of the point cloud containing the target sphere center from the point cloud set of the sphere structure, and calculate the first coordinate of the target sphere center in the camera coordinate system based on the point cloud subset.

[0048] Sphere-and-socket structures may contain non-sphere-and-socket regions, such as the edges of the structure and other redundant parts. A point cloud segmentation algorithm is needed to extract a subset of the point cloud containing only the sphere-and-socket regions. For example... Figure 3 As shown, since the center of the spherical cavity is an inwardly concave spherical structure, the geometric features (such as curvature distribution and spatial morphology) of its point cloud subset differ significantly from other regions, and can be used as a basis for segmentation. The core feature of the spherical cavity center is the center of the concave sphere. Therefore, the point cloud subset of the spherical cavity is fitted using a spherical fitting algorithm (such as the least squares method). This process achieves accurate positioning of the target spherical cavity center through geometric feature fitting.

[0049] Optionally, a subset of point clouds containing the target spherical socket center is extracted from the point cloud set of the spherical socket structure, and the first coordinates of the target spherical socket center in the camera coordinate system are calculated based on the point cloud subset, including:

[0050] Based on the preset dimensions of the spherical socket structure, a target bounding box containing the center of the target spherical socket is defined, and a subset of the point cloud located within the target bounding box is extracted from the complete point cloud set of the spherical socket structure as the region of interest.

[0051] The point cloud of the region of interest is downsampled and statistically filtered to remove outliers and generate an optimized point cloud of the region of interest.

[0052] The geometric centroid coordinates of the optimized region of interest points are calculated using the geometric centroid calculation formula. Then, using the geometric centroid as the initial value, the first coordinate of the target sphere center in the camera coordinate system is obtained by optimization using the least squares method.

[0053] The point cloud set of the spherical socket structure refers to the complete 3D point cloud data after stitching, containing the 3D coordinate information of all surfaces of the spherical socket structure at the bottom of the towed object; the point cloud subset is a portion of the point cloud data selected from the complete point cloud set, containing only the center of the target spherical socket and its surrounding key areas, used to focus on analyzing the core features of the spherical socket; the target bounding box is a 3D spatial range defined according to the preset dimensions (such as diameter, depth, etc.) of the spherical socket structure, used to quickly locate and extract the area where the spherical socket is located from the complete point cloud; the region of interest (ROI) refers to the area in the point cloud subset that is directly related to the target spherical socket, and is the core object for subsequent processing and calculation; downsampling is a method to reduce the amount of point cloud data. The proposed method reduces computational complexity while preserving key geometric features. Statistical filtering is an algorithm that removes outliers based on the statistical characteristics of local neighborhoods in a point cloud (e.g., by calculating the average distance from a point to its neighbors and removing noise points whose distance exceeds a threshold). Outliers are abnormal points in a point cloud that deviate significantly from the distribution of the surrounding point cloud, usually caused by sensor noise or environmental interference. The geometric centroid is the coordinate of the geometric center of a subset of the point cloud, calculated by averaging the coordinates of all points, and can be used as an initial approximation of the center of the sphere. The least squares method is an optimization algorithm that solves the best-fit model by minimizing the sum of squared errors. In this embodiment, it can be used to fit the spherical equation of the sphere and accurately calculate the coordinates of the sphere center.

[0054] The complete point cloud of the spherical cavity structure contains a large amount of redundant information, and directly using it to calculate the center of the cavity will increase errors and computational burden. Therefore, a process of "extracting the core region → optimizing the point cloud quality → accurately fitting the sphere center" is needed to focus on the geometric features of the cavity and ultimately achieve high-precision positioning. Based on prior knowledge, the analysis scope is narrowed to quickly locate the target area. The cavity is an inwardly concave spherical structure, and its dimensions (such as the spherical diameter and concavity depth) have clearly defined parameters during design. Based on these parameters, a cuboid bounding box slightly larger than the actual size of the cavity is defined in 3D space. The complete point cloud set is traversed, and it is determined whether the 3D coordinates of each point fall within the bounding box. Points that meet the conditions are retained, forming the region of interest. Voxel mesh downsampling is used to divide the point cloud of the region of interest into multiple small cubes, retaining one representative point within each voxel, reducing the number of points while preserving the overall geometric shape. The optimized point cloud retains the spherical geometric features of the cavity while removing noise and redundancy, providing high-quality data for accurately fitting the sphere center.

[0055] S140. Transform the first coordinate of the target ball socket center to the second coordinate in the base coordinate system where the target flexible support device is located by using preset hand-eye calibration parameters.

[0056] The camera coordinate system (sensor viewpoint) and the base coordinate system (motion control coordinate system of the flexible support device) are two independent coordinate systems. Directly calculating the deviation between the ball head and the socket based on the camera coordinates has no practical control significance (the coordinate system of the control actuator is the base coordinate system). Therefore, a coordinate transformation is needed to unify the calculation benchmark. Hand-eye calibration is the process of establishing the transformation relationship between the camera coordinate system and the actuator's base coordinate system. Through this transformation, the center coordinates of the socket are changed from the camera viewpoint to the control viewpoint of the flexible support device, providing a unified coordinate system for subsequent deviation calculations.

[0057] Furthermore, transforming the first coordinate of the target ball's center to the second coordinate in the base coordinate system where the target flexible support device is located, using preset hand-eye calibration parameters, may include:

[0058] The hand-eye calibration parameters pre-stored in the control system are called. Based on the rotation matrix in the hand-eye calibration parameters, the first coordinate of the center of the ball in the camera coordinate system is substituted into the Euler angle rotation formula, and a rotation transformation is performed on the first coordinate.

[0059] Based on the translation vector in the hand-eye calibration parameters, the first coordinate after rotation transformation is superimposed with the translation amount to obtain the second coordinate in the base coordinate system where the target flexible support device is located.

[0060] Hand-eye calibration parameters describe the spatial relationship between the camera coordinate system and the base coordinate system. These parameters include rotation matrices and translation vectors, used to achieve coordinate transformation between the two systems. The base coordinate system is a three-dimensional coordinate system established based on the target flexible support device. It serves as the reference coordinate system for the motion control of the flexible support device, and all control commands are generated based on this coordinate system. The Euler angle rotation formula is a mathematical formula that converts a rotation matrix in three-dimensional space into rotation angles (Euler angles) around the three coordinate axes (usually X, Y, and Z axes), and then achieves coordinate transformation through sequential rotation. It is used to intuitively describe the rotation transformation process. Rotation transformation is a coordinate transformation operation that uses rotation matrices or Euler angles to rotate a point in one coordinate system around its origin, making its direction consistent with another coordinate system. The translation vector is a three-dimensional vector describing the positional difference between the origins of two coordinate systems. It is used to translate the rotated coordinates along the vector direction to align the origins of the two coordinate systems.

[0061] The camera coordinate system and the base coordinate system are two independent coordinate systems, with differences in their origin positions and coordinate axis directions. Hand-eye calibration parameters are transformed using a combination of rotation and translation to establish a mathematical mapping relationship between the two coordinate systems, achieving cross-system coordinate transformation. Through rotation, the direction of the spherical socket center coordinates is aligned with the coordinate axis direction of the base coordinate system. Through translation vectors, the positional deviation of the origins of the two coordinate systems is eliminated, achieving origin alignment.

[0062] S150. Based on the real-time coordinates of the target ball head in the base coordinate system and the second coordinates of the target ball socket center, calculate the positional deviation between the target ball head and the target ball socket center, and adjust the pose of the target flexible support device according to the positional deviation to achieve precise positioning of the target ball head into the target ball socket center. The target flexible support device can dynamically adjust its attitude in three vertical directions.

[0063] Position deviation refers to the difference between the real-time coordinates of the target ball head and the second coordinate of the target ball socket center. It includes lateral and longitudinal deviations in the horizontal plane and vertical deviations in the vertical direction, and is used to quantify the alignment error between the two. Through a closed-loop feedback process of continuously collecting the real-time coordinates of the ball head, calculating the deviation, and adjusting the pose, the deviation gradually converges to within a preset threshold, ultimately achieving mechanical alignment between the ball head and the ball socket center, and completing precise positioning.

[0064] This invention, through the synergy of dual depth sensor deployment and point cloud stitching algorithms, solves the blind spot problem caused by spherical head occlusion from both hardware layout and data processing perspectives. Compared to single-sensor solutions, it can completely capture the three-dimensional information of the spherical head and socket, laying the foundation for accurate positioning. Simultaneously, it deeply integrates the three-dimensional dynamic adjustment capability of the flexible support device with coordinate transformation and deviation-driven logic. Using hand-eye calibration, it achieves deviation-free mapping across coordinate systems and directly responds to quantified deviations through multi-degree-of-freedom adjustment, forming a "perception-decision-execution" closed loop. This addresses the pain points of disconnect between perception and execution and insufficient adjustment freedom in general technologies. It adapts to the high-precision dynamic docking requirements of flexible support devices, demonstrating the deep empowerment of hardware characteristics and algorithmic logic.

[0065] Example 2

[0066] Figure 4 This is a flowchart of another depth-sensing-based locator positioning method provided in Embodiment 2 of the present invention. This embodiment is a refinement based on Embodiment 1, specifically as follows: Figure 4 As shown, the method includes:

[0067] S410: Two sets of local depth images of the target ball head structure to be positioned are acquired by two depth sensors deployed on both sides of the target ball head on the target flexible support device.

[0068] S420. Based on the intrinsic parameter matrix of the depth sensor, the two sets of local depth images are converted into three-dimensional point cloud data in the camera coordinate system, and based on the pre-calibrated extrinsic parameter matrix between sensors, the two sets of three-dimensional point cloud data are stitched together into a complete point cloud set of the spherical-socket structure.

[0069] S430. Extract a subset of the point cloud containing the target sphere center from the point cloud set of the sphere structure, and calculate the first coordinate of the target sphere center in the camera coordinate system based on the point cloud subset.

[0070] S440. Transform the first coordinate of the target ball's center to the second coordinate in the base coordinate system where the target flexible support device is located by using preset hand-eye calibration parameters.

[0071] S450: Read the position feedback data of each motion axis of the target flexible support device to obtain the real-time coordinates of the target ball head in the base coordinate system.

[0072] Each motion axis of the flexible support device is equipped with a position sensor, which can provide real-time feedback on the current position of the axis (such as the X-axis movement and Y-axis movement). The real-time coordinates of the target ball head refer to the current position coordinates of the target ball head in the base coordinate system. These coordinates are acquired in real time through the position feedback data of the motion axes of the flexible support device, reflecting the current position of the ball head. By collecting the position feedback signals of each motion axis of the flexible support device, the current three-dimensional coordinates of the target ball head in the base coordinate system are calculated, serving as one of the benchmarks for deviation calculation.

[0073] S460. Calculate the lateral and longitudinal deviations between the second coordinates of the target ball's center and the real-time coordinates of the target ball's head in the horizontal plane.

[0074] The lateral deviation is the positional difference between the ball head and the center of the socket in the horizontal plane; the longitudinal deviation is the positional difference between the ball head and the center of the socket in the horizontal plane. In the horizontal plane, the differences between the second coordinate of the socket center and the real-time coordinate of the ball head in the lateral and longitudinal directions are calculated to obtain the lateral and longitudinal deviations, reflecting the alignment error in the horizontal direction.

[0075] S470. Adjust the pose of the target flexible support device with multiple degrees of freedom according to the deviation in the horizontal plane, and drive the target ball head to move in the direction of reducing the deviation.

[0076] Multi-degree-of-freedom pose adjustment refers to the attitude adjustment of a flexible support device in multiple directions of motion. Through coordinated adjustment of multiple degrees of freedom, horizontal position and attitude deviations are eliminated, laying the foundation for vertical positioning. The sign of the deviation indicates the orientation of the ball head relative to the center of the socket.

[0077] Optionally, adjusting the pose of the target flexible support device in multiple degrees of freedom based on the deviation in the horizontal plane, and driving the target ball head to move in the direction of reducing the deviation, may include:

[0078] when At that time, control the ball head to move along the positive X-axis of the base coordinate system. distance;

[0079] when At that time, control the ball head to move along the negative X-axis of the base coordinate system. distance;

[0080] when At that time, control the ball head to move along the positive Y-axis of the base coordinate system. distance;

[0081] when At that time, control the ball head to move along the negative Y-axis of the base coordinate system. distance;

[0082] Among them, the For lateral deviation, For longitudinal deviation, the X-axis is parallel to the longitudinal axis of the guide vehicle, with the positive X-axis pointing in the direction of the guide vehicle's forward movement; the Y-axis is parallel to the transverse axis of the guide vehicle, with the positive Y-axis pointing to the left side of the guide vehicle.

[0083] lateral deviation The difference between the second coordinate of the ball socket center and the real-time coordinate of the ball head in the X-axis direction reflects the alignment error along the longitudinal axis of the guide vehicle; longitudinal deviation. The value represents the difference between the second coordinate of the ball socket center and the real-time coordinate of the ball head in the Y-axis direction, reflecting the alignment error along the transverse axis of the guide vehicle.

[0084] If lateral deviation A positive value indicates that the center of the ball socket is located along the positive X-axis of the ball head, meaning the ball head is behind the center of the ball socket. Therefore, the ball head is controlled to move along the direction of travel of the guide vehicle (positive X-axis). Equal absolute distances are used to eliminate directional deviation; if there is lateral deviation... A negative value indicates that the center of the ball socket is in the negative X-axis direction of the ball head, meaning the ball head is in front of the center of the ball socket. Therefore, the ball head is controlled to move in the backward direction (negative X-axis) of the guide vehicle. Equal absolute distances are used to eliminate directional deviation; if there is a longitudinal deviation... A positive value indicates that the center of the ball socket is on the positive Y-axis of the ball head, meaning the ball head is to the right of the center of the ball socket. Therefore, the ball head is controlled to move along the left side of the guide vehicle (positive Y-axis). Equal absolute distances are used to eliminate directional deviation; if there is a longitudinal deviation... A negative value indicates that the center of the ball socket is on the negative Y-axis of the ball head, meaning the ball head is to the left of the center of the ball socket. Therefore, the ball head is controlled to move along the right side of the guide vehicle (negative Y-axis). Equal distances in absolute value are used to eliminate the directional deviation.

[0085] S480. After adjusting the horizontal plane pose, calculate the vertical deviation between the target ball head and the center of the target ball socket, control the target ball head to move in a direction perpendicular to the roof of the guide vehicle, and determine that the positioning is complete when the moving distance reaches the vertical deviation.

[0086] The vertical deviation refers to the positional difference between the ball head and the center of the socket in a direction perpendicular to the horizontal plane. After the horizontal deviation converges, the vertical deviation is calculated. Vertical movement control refers to controlling the Z-axis motor of the flexible support device to drive the ball head to move in a direction perpendicular to the vehicle roof. When the movement distance fed back by the Z-axis reaches the vertical deviation, the real-time coordinates of the ball head coincide with the coordinates of the center of the socket, indicating that the positioning is complete and the movement stops.

[0087] This invention refines the specific control logic and execution rules for ball joint positioning. First, it clarifies that by reading the position feedback data of the flexible support device's motion axis, the coordinates of the ball joint in the base coordinate system are obtained in real time, providing a dynamic benchmark for deviation calculation. Second, the positioning process is broken down into two steps: horizontal adjustment and vertical positioning. First, the ball joint is controlled to move directionally along the X and Y axes of the base coordinate system using quantified values ​​of lateral and longitudinal deviations, followed by vertical movement. Third, the X and Y axes of the base coordinate system are bound to the physical direction of the guide vehicle, clarifying the correspondence between the deviation sign and the direction of movement. This embodiment, through a collaborative design of "real-time feedback - step-by-step adjustment - direction binding," enables the ball joint to quickly converge deviations and stably complete high-precision positioning, effectively improving the efficiency and reliability of the flexible support device docking process.

[0088] Example 3

[0089] Figure 5 This is a schematic diagram of a depth-sensing-based positioning device provided in Embodiment 3 of the present invention. Figure 5 As shown, the device includes:

[0090] The local depth image acquisition module 510 is used to acquire two sets of local depth images of the target ball-and-socket structure to be positioned by two depth sensors respectively deployed on both sides of the target ball head on the target flexible support device.

[0091] The point cloud assembly construction module 520 is used to convert two sets of local depth images into three-dimensional point cloud data in the camera coordinate system according to the intrinsic parameter matrix of the depth sensor, and to stitch the two sets of three-dimensional point cloud data into a complete point cloud assembly of the spherical-socket structure based on the pre-calibrated extrinsic parameter matrix between the sensors.

[0092] The first coordinate calculation module 530 is used to extract a subset of point clouds containing the center of the target sphere structure from the point cloud set of the sphere structure, and calculate the first coordinate of the center of the target sphere in the camera coordinate system based on the point cloud subset;

[0093] The second coordinate calculation module 540 is used to convert the first coordinate of the target ball hole center to the second coordinate in the base coordinate system where the target flexible support device is located by using preset hand-eye calibration parameters;

[0094] The pose adjustment module 550 is used to calculate the positional deviation between the target ball head and the target ball socket center based on the real-time coordinates of the target ball head in the base coordinate system and the second coordinate of the target ball socket center, and to adjust the pose of the target flexible support device according to the positional deviation to achieve precise positioning of the target ball head to the target ball socket center. The target flexible support device can dynamically adjust its attitude in three vertical directions.

[0095] This invention, through the synergy of dual depth sensor deployment and point cloud stitching algorithms, solves the blind spot problem caused by spherical head occlusion from both hardware layout and data processing perspectives. Compared to single-sensor solutions, it can completely capture the three-dimensional information of the spherical head and socket, laying the foundation for accurate positioning. Simultaneously, it deeply integrates the three-dimensional dynamic adjustment capability of the flexible support device with coordinate transformation and deviation-driven logic. Using hand-eye calibration, it achieves deviation-free mapping across coordinate systems and directly responds to quantified deviations through multi-degree-of-freedom adjustment, forming a "perception-decision-execution" closed loop. This addresses the pain points of disconnect between perception and execution and insufficient adjustment freedom in general technologies. It adapts to the high-precision dynamic docking requirements of flexible support devices, demonstrating the deep empowerment of hardware characteristics and algorithmic logic.

[0096] Optionally, based on the above embodiments, the point cloud collection construction module 520 may include:

[0097] The first 3D point cloud data conversion unit is used to convert the pixel coordinates and pixel depth values ​​of each pixel in a set of depth images acquired by the first depth sensor into first 3D point cloud data in the first camera coordinate system that matches the first depth sensor, based on the principal point coordinates of the first depth sensor and the focal length of the camera.

[0098] The second 3D point cloud data conversion unit is used to convert the pixel coordinates and pixel depth values ​​of each pixel in another set of depth images acquired by the second depth sensor into second 3D point cloud data in the second camera coordinate system that matches the second depth sensor, based on the principal point coordinates of the second depth sensor and the camera focal length.

[0099] Optionally, based on the above embodiments, the point cloud collection construction module 520 may further include:

[0100] A camera coordinate system unit is used to transform the second 3D point cloud data in the second camera coordinate system to the first camera coordinate system using the pre-calibrated extrinsic rotation matrix and translation vector between two depth sensors.

[0101] The point cloud data stitching unit is used to stitch together the three-dimensional point cloud data in the first camera coordinate system according to the transformation result to obtain a complete point cloud set of the spherical cavity structure.

[0102] Optionally, based on the above embodiments, the first coordinate calculation module 530 may include:

[0103] The region of interest extraction unit is used to define a target bounding box containing the center of the target spherical socket according to the preset size of the spherical socket structure, and extract a subset of the point cloud located within the target bounding box from the complete point cloud set of the spherical socket structure as the region of interest;

[0104] The region optimization unit is used to perform downsampling and statistical filtering on the point cloud of the region of interest, remove outliers, and generate an optimized point cloud of the region of interest.

[0105] The centroid coordinate solving unit is used to calculate the geometric centroid coordinates of the optimized region of interest points using the geometric centroid calculation formula, and to obtain the first coordinates of the target sphere center in the camera coordinate system using the least squares method with the geometric centroid as the initial value.

[0106] Optionally, based on the above embodiments, the second coordinate calculation module 540 may include:

[0107] The rotation transformation unit is used to call the hand-eye calibration parameters pre-stored in the control system, and based on the rotation matrix in the hand-eye calibration parameters, substitute the first coordinate of the ball's center in the camera coordinate system into the Euler angle rotation formula to perform a rotation transformation on the first coordinate.

[0108] The translation unit is used to obtain the second coordinate in the base coordinate system where the target flexible support device is located by superimposing the translation amount on the first coordinate after rotation transformation based on the translation vector in the hand-eye calibration parameters.

[0109] Optionally, based on the above embodiments, the pose adjustment module 550 may include:

[0110] The ball head position reading unit is used to read the position feedback data of each motion axis of the target flexible support device and obtain the real-time coordinates of the target ball head in the base coordinate system;

[0111] The horizontal deviation calculation unit is used to calculate the lateral and longitudinal deviations between the second coordinates of the target ball's center and the real-time coordinates of the target ball's head in the horizontal plane.

[0112] The horizontal pose adjustment unit is used to adjust the pose of the target flexible support device in multiple degrees of freedom according to the deviation in the horizontal plane, and drive the target ball head to move in the direction of reducing the deviation.

[0113] The vertical pose adjustment unit is used to calculate the vertical deviation between the target ball head and the center of the target ball socket after the pose adjustment in the horizontal plane, control the target ball head to move in a direction perpendicular to the roof of the guide vehicle, and determine that the positioning is completed when the moving distance reaches the vertical deviation.

[0114] Optionally, based on the above embodiments, the horizontal pose adjustment unit can also be used when... At that time, control the ball head to move along the positive X-axis of the base coordinate system. Distance; when At that time, control the ball head to move along the negative X-axis of the base coordinate system. Distance; when At that time, control the ball head to move along the positive Y-axis of the base coordinate system. Distance; when At that time, control the ball head to move along the negative Y-axis of the base coordinate system. distance;

[0115] Among them, the For lateral deviation, For longitudinal deviation, the X-axis is parallel to the longitudinal axis of the guide vehicle, with the positive X-axis pointing in the direction of the guide vehicle's forward movement; the Y-axis is parallel to the transverse axis of the guide vehicle, with the positive Y-axis pointing to the left side of the guide vehicle.

[0116] The depth-sensing-based locator positioning device provided in this embodiment of the invention can execute the depth-sensing-based locator positioning method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0117] Example 4

[0118] Figure 6 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0119] like Figure 6As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0120] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0121] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as a depth-sensing-based locator positioning method.

[0122] That is, by using two depth sensors deployed on both sides of the target ball head on the target flexible support device, two sets of local depth images of the target ball-and-socket structure to be positioned are collected respectively;

[0123] Based on the intrinsic parameter matrix of the depth sensor, the two sets of local depth images are converted into three-dimensional point cloud data in the camera coordinate system, and based on the pre-calibrated extrinsic parameter matrix between the sensors, the two sets of three-dimensional point cloud data are stitched together into a complete point cloud set of the spherical cavity structure.

[0124] Extract a subset of the point cloud containing the target sphere center from the point cloud set of the sphere structure, and calculate the first coordinate of the target sphere center in the camera coordinate system based on the point cloud subset;

[0125] The first coordinate of the target ball socket center is transformed to the second coordinate in the base coordinate system where the target flexible support device is located by using preset hand-eye calibration parameters;

[0126] Based on the real-time coordinates of the target ball head in the base coordinate system and the second coordinate of the target ball socket center, the positional deviation between the target ball head and the target ball socket center is calculated, and the pose of the target flexible support device is adjusted according to the positional deviation to achieve precise positioning of the target ball head into the target ball socket center. The target flexible support device can dynamically adjust its attitude in three vertical directions.

[0127] In some embodiments, a depth-sensing-based locator positioning method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the depth-sensing-based locator positioning method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform a depth-sensing-based locator positioning method by any other suitable means (e.g., by means of firmware).

[0128] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0129] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0130] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0131] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0132] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0133] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0134] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0135] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A positioning method for a locator based on depth sensing, applied in a scenario where a guide vehicle precisely positions the ball heads at the top of each flexible support device on the roof into the centers of the inwardly recessed ball sockets in the ball socket structures at the bottom of the towed object, characterized in that... The method includes: Two depth sensors, respectively deployed on both sides of the target ball head on the target flexible support device, acquire two sets of local depth images of the target ball-and-socket structure to be positioned; Based on the intrinsic parameter matrix of the depth sensor, the two sets of local depth images are converted into three-dimensional point cloud data in the camera coordinate system, and based on the pre-calibrated extrinsic parameter matrix between the sensors, the two sets of three-dimensional point cloud data are stitched together into a complete point cloud set of the spherical cavity structure. Extract a subset of the point cloud containing the target sphere center from the point cloud set of the sphere structure, and calculate the first coordinate of the target sphere center in the camera coordinate system based on the point cloud subset; The first coordinate of the target ball socket center is transformed to the second coordinate in the base coordinate system where the target flexible support device is located by using preset hand-eye calibration parameters; Based on the real-time coordinates of the target ball head in the base coordinate system and the second coordinate of the target ball socket center, the positional deviation between the target ball head and the target ball socket center is calculated, and the pose of the target flexible support device is adjusted according to the positional deviation to achieve precise positioning of the target ball head into the target ball socket center. The target flexible support device can dynamically adjust its attitude in three vertical directions.

2. The method according to claim 1, characterized in that, Based on the intrinsic parameter matrix of the depth sensor, the two sets of local depth images are converted into 3D point cloud data in the camera coordinate system, including: Based on the principal point coordinates of the first depth sensor and the focal length of the camera, the pixel coordinates and pixel depth values ​​of each pixel in a set of depth images acquired by the first depth sensor in the pixel coordinate system are converted into first three-dimensional point cloud data in the first camera coordinate system that matches the first depth sensor. Based on the principal point coordinates of the second depth sensor and the camera focal length, the pixel coordinates and pixel depth values ​​of each pixel in another set of depth images acquired by the second depth sensor in the pixel coordinate system are converted into second three-dimensional point cloud data in the second camera coordinate system that matches the second depth sensor.

3. The method according to claim 2, characterized in that, Based on the pre-calibrated inter-sensor extrinsic matrix, the two sets of 3D point cloud data are stitched together to form a complete point cloud set of the spherical-socket structure, including: Using a pre-calibrated extrinsic rotation matrix and translation vector between two depth sensors, the second 3D point cloud data in the second camera coordinate system is transformed to the first camera coordinate system; Based on the transformation results, the three-dimensional point cloud data under the first camera coordinate system are stitched together to obtain a complete set of point cloud data for the spherical cavity structure.

4. The method according to claim 1, characterized in that, Extract a subset of the point cloud containing the target spherical socket structure from the point cloud set of the spherical socket structure, and calculate the first coordinate of the target spherical socket center in the camera coordinate system based on the point cloud subset, including: Based on the preset dimensions of the spherical socket structure, a target bounding box containing the center of the target spherical socket is defined, and a subset of the point cloud located within the target bounding box is extracted from the complete point cloud set of the spherical socket structure as the region of interest. The point cloud of the region of interest is downsampled and statistically filtered to remove outliers, generating an optimized point cloud of the region of interest. The geometric centroid coordinates of the optimized region of interest points are calculated using the geometric centroid calculation formula. Then, using the geometric centroid as the initial value, the first coordinate of the target sphere center in the camera coordinate system is obtained by optimization using the least squares method.

5. The method according to claim 1, characterized in that, The first coordinate of the target ball's center is transformed to the second coordinate in the base coordinate system of the target flexible support device by using preset hand-eye calibration parameters, including: The hand-eye calibration parameters pre-stored in the control system are called. Based on the rotation matrix in the hand-eye calibration parameters, the first coordinate of the center of the ball in the camera coordinate system is substituted into the Euler angle rotation formula, and a rotation transformation is performed on the first coordinate. Based on the translation vector in the hand-eye calibration parameters, the first coordinate after rotation transformation is superimposed with the translation amount to obtain the second coordinate in the base coordinate system where the target flexible support device is located.

6. The method according to claim 1, characterized in that, Based on the real-time coordinates of the target ball head in the base coordinate system and the second coordinates of the target ball socket center, the positional deviation between the target ball head and the target ball socket center is calculated. The pose of the flexible support device is then adjusted according to this positional deviation to achieve precise positioning of the target ball head relative to the target ball socket center. This includes: Read the position feedback data of each motion axis of the target flexible support device to obtain the real-time coordinates of the target ball head in the base coordinate system; Calculate the lateral and longitudinal deviations in the horizontal plane between the second coordinate of the target ball's center and the real-time coordinates of the target ball's head; The target flexible support device is adjusted in multiple degrees of freedom according to the deviation in the horizontal plane, and the target ball head is driven to move in the direction of reducing the deviation. After adjusting the horizontal plane pose, the vertical deviation between the target ball head and the center of the target ball socket is calculated. The target ball head is then controlled to move in a direction perpendicular to the roof of the guide vehicle. When the moving distance reaches the vertical deviation, the positioning is considered complete.

7. The method according to claim 6, characterized in that, Based on the deviation within the horizontal plane, the target flexible support device is adjusted in multiple degrees of freedom to drive the target ball head to move in the direction that reduces the deviation, including: when At that time, control the ball head to move along the positive X-axis of the base coordinate system. distance; when At that time, control the ball head to move along the negative X-axis of the base coordinate system. distance; when At that time, control the ball head to move along the positive Y-axis of the base coordinate system. distance; when At that time, control the ball head to move along the negative Y-axis of the base coordinate system. distance; Among them, the This is the lateral deviation. For longitudinal deviation, the X-axis is parallel to the longitudinal axis of the guide vehicle, with the positive X-axis pointing in the direction of the guide vehicle's forward movement; the Y-axis is parallel to the transverse axis of the guide vehicle, with the positive Y-axis pointing to the left side of the guide vehicle.

8. A positioning device based on depth sensing, applied in a scenario where a guide vehicle precisely positions the ball heads at the top of each flexible support device on the roof into the center of each inwardly recessed ball socket in the ball socket structure at the bottom of the towed object, characterized in that... The device includes: The local depth image acquisition module is used to acquire two sets of local depth images of the target ball-and-socket structure to be positioned by two depth sensors respectively deployed on both sides of the target ball head on the target flexible support device. The point cloud assembly construction module is used to convert two sets of local depth images into three-dimensional point cloud data in the camera coordinate system according to the intrinsic parameter matrix of the depth sensor, and to stitch the two sets of three-dimensional point cloud data into a complete point cloud assembly of the spherical-socket structure based on the pre-calibrated extrinsic parameter matrix between the sensors. The first coordinate calculation module is used to extract a subset of point clouds containing the center of the target sphere structure from the point cloud set of the sphere structure, and calculate the first coordinate of the center of the target sphere in the camera coordinate system based on the point cloud subset; The second coordinate calculation module is used to transform the first coordinate of the target ball hole center to the second coordinate in the base coordinate system where the target flexible support device is located by using preset hand-eye calibration parameters; The pose adjustment module is used to calculate the positional deviation between the target ball head and the target ball socket center based on the real-time coordinates of the target ball head in the base coordinate system and the second coordinate of the target ball socket center. Based on the positional deviation, the module adjusts the pose of the target flexible support device to achieve precise positioning of the target ball head into the target ball socket center. The target flexible support device can dynamically adjust its attitude in three vertical directions.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform a depth-sensing-based locator positioning method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute and implement the depth-sensing-based locator positioning method according to any one of claims 1-7.

Images