Method and system for automatic hanging of a suspension by means of three-dimensional vision measurement

Abstract

The application provides a method and system for automatic hanging of a suspension by three-dimensional vision measurement, the method comprising the following steps: collecting images of targets in an aircraft cabin, and obtaining pose information related to an actual suspension and an aircraft rack; based on the pose information, calculating pose deviation of a bomb loading vehicle relative to a design axis of the aircraft cabin in horizontal, vertical, pitch and rotation directions, and controlling a lifting device to move cooperatively with a six-degree-of-freedom platform to adjust the spatial position and attitude of the actual suspension, so that the suspension lug and the hook of the aircraft rack are precisely connected, and automatic hanging is completed. The method accurately measures targets in the aircraft cabin, can obtain the pose information of the suspension and the rack in real time and with high precision, and calculates accurate pose deviation. Based on the deviation, the lifting device and the six-degree-of-freedom platform move cooperatively to automatically and accurately adjust the spatial position and attitude of the suspension, and finally realize precise connection of the lug and the hook.

Description

Technical Field

[0001] This application relates to the field of cargo mounting technology, specifically to a method and system for automatically mounting suspended objects using trinocular vision measurement. Background Technology

[0002] In the field of aviation support, the mounting of suspended objects (such as missiles, auxiliary fuel tanks, etc.) is a critical operation. Currently, traditional mounting methods mainly rely on the manual judgment and operation of operators, or supplemented by simple mechanical positioning devices. Specifically, operators need to drive the loading vehicle into the cabin and, through visual inspection and experience, repeatedly adjust the position and attitude of the loading vehicle and lifting device to try to align the lifting lugs of the suspended object with the hooks of the aircraft pylon and complete the mounting.

[0003] This method has several inherent drawbacks: First, it suffers from low operational precision and poor consistency, heavily relying on the operator's individual skills and experience. Results vary significantly depending on the operator or even the same operator under different conditions, making it difficult to meet the increasingly stringent precision requirements of modern fighter jets. Second, it is inefficient, with repeated manual adjustments being time-consuming and severely impacting the aircraft's readiness time for re-deployment. Third, it carries high safety risks. In the confined and dimly lit cockpit, manual operation is prone to visual fatigue or misjudgment, leading to collisions between suspended equipment and aircraft pylons, causing damage to valuable aviation equipment and even personal injury accidents.

[0004] While some existing technologies attempt to automate the process, most suffer from poor adaptability, insufficient stability in complex cabin environments, or high costs. Therefore, there is an urgent need in this field for a method that can achieve high-precision, high-efficiency, and high-reliability automated loading and unloading. Summary of the Invention

[0005] In view of the above-mentioned defects or deficiencies in the prior art, this application aims to provide a method and system for automatically mounting suspended objects through trinocular vision measurement.

[0006] In a first aspect, this application proposes a method for automatically mounting suspended objects using trinocular vision measurement, comprising the following steps: Images of targets inside the aircraft cabin are captured by a three-eye camera mounted on the ammunition loading vehicle to obtain positional information related to the actual suspended objects and aircraft pylons. Based on the pose information, the pose deviation of the loading vehicle relative to the aircraft cabin design axis is calculated in the horizontal, vertical, pitch and rotation directions. Based on the positional deviation, the lifting device and the six-degree-of-freedom platform are controlled to move in coordination to adjust the spatial position and attitude of the actual suspended object, so that the lugs of the actual suspended object can be precisely aligned with the hooks of the aircraft pylon, thus completing the automatic mounting.

[0007] According to the technical solution provided in the embodiments of this application, before obtaining the pose information related to the actual suspended object and aircraft pylon, system calibration is further included, which includes the following steps: With the six-degree-of-freedom platform in a zero-state state, a simulated suspended object is installed on the six-degree-of-freedom platform, and an in-situ measuring device is installed on the simulated suspended object; The target on the in-situ measuring device is measured by the three-eye camera, and the pose of the simulated suspended object is adjusted according to the measurement data so that its longitudinal position offset is less than or equal to a first preset threshold, its roll angle is less than or equal to a second preset threshold, and the first relative position data between the simulated suspended object and the platform is recorded. The six-degree-of-freedom platform is manipulated to connect the lugs of the simulated suspended object with the aircraft hook. The connection accuracy is controlled so that the longitudinal and lateral relative positions are less than or equal to the third preset threshold, and the height is less than or equal to the fourth preset threshold. The second relative position data of the lugs and the hook are recorded. Based on the first relative position data and the second relative position data, the first position compensation value is calculated; Based on the first position compensation value, the actual suspended object is attached.

[0008] According to the technical solution provided in the embodiments of this application, after the system calibration and before acquiring images of the target inside the aircraft cabin through a three-eye camera mounted on the ammunition loading vehicle and obtaining pose information related to the suspended object and pylon, the system further includes self-homing and walking, which includes the following steps: The three-lens camera identifies preset navigation markings within the cabin. The loading vehicle is controlled to travel automatically or manually along the preset navigation line until it moves directly below the attachment point, and the three-eye camera is able to observe the target inside the aircraft cabin.

[0009] According to the technical solution provided in the embodiments of this application, when the control lifting device and the six-degree-of-freedom platform move in coordination to adjust the spatial position and attitude of the actual suspended object, the method further includes attachment verification; the attachment verification includes the following steps: After the lifting lug and the hook are connected, the six-degree-of-freedom platform is controlled to slowly fall. By monitoring changes in the force state of the platform, it can be determined whether the actual suspended object has been reliably attached to the hook; When an abnormal force condition is detected, the descent of the six-degree-of-freedom platform is paused and an alarm is issued.

[0010] According to the technical solution provided in the embodiments of this application, after calculating the first position compensation value based on the first relative position data and the second relative position data, and before hanging the actual suspended object based on the first position compensation value, the following steps are further included: Determine whether the deviation between the initial pose of the actual suspended object and the zero pose of the six-degree-of-freedom platform exceeds the compensable range of the six-degree-of-freedom platform during the mounting process. The process of attaching the actual suspended object based on the first position compensation value includes the following steps: If not, then the actual suspended object is mounted based on the first position compensation value.

[0011] According to the technical solution provided in the embodiments of this application, after determining whether the deviation between the initial pose of the actual suspended object and the zero pose of the six-degree-of-freedom platform exceeds the compensable range of the six-degree-of-freedom platform during the mounting process, the method further includes the following steps: If so, the in-situ measuring device is installed on the actual suspended object; The target on the in-situ measuring device is measured by the three-lens camera, and the pose of the actual suspended object is adjusted according to the measurement data; The actual suspended object is fixed on the six-degree-of-freedom platform, and the third relative position data between the actual suspended object and the six-degree-of-freedom platform is recorded; Based on the first relative position data, the second relative position data, and the third relative position data, a second position compensation value is calculated for mounting the actual suspended object.

[0012] According to the technical solution provided in the embodiments of this application, the control lifting device and the six-degree-of-freedom platform move in coordination to adjust the spatial position and attitude of the suspended object, including the following steps: Based on the posture deviation and / or the second position compensation value, the kinematic model is used to calculate and generate a predetermined upward trajectory for the lug of the actual suspended object to move from its current position to precisely dock with the hook. During the lifting process, the target inside the aircraft cabin is identified in real time by a three-eye camera, and the positional deviation is dynamically updated. The pose deviation calculated in real time is compared with the expected pose of the predetermined ascent trajectory to generate a tracking error; Based on the tracking error, the combined motion of the lifting device and the six-degree-of-freedom platform is controlled in real time, so that the lugs of the actual suspended object move along the predetermined upward trajectory until they dock with the hook.

[0013] According to the technical solution provided in the embodiments of this application, the method further includes the following steps: The operating status of the three-lens camera is monitored in real time, including image clarity, target recognition confidence, or data transmission continuity. The lifting lugs of the actual suspended object move along the predetermined upward trajectory, including the following steps: If the operating state is abnormal, it will automatically switch to the pose prediction and tracking mode based on the inertial measurement unit. The pose prediction and tracking modes include: Acquire acceleration and angular velocity data output by an inertial measurement unit installed on the actual suspended object or six-degree-of-freedom platform; Based on the acceleration and angular velocity data and the effective visual pose of the previous moment, predict the first current pose of the actual suspended object; Based on the first current pose and the desired pose of the predetermined upward trajectory, control commands are generated to cause the lifting device and the six-degree-of-freedom platform to move together, so as to maintain the lug moving along the predetermined upward trajectory. After the three-lens camera returns to normal operation, it automatically switches back to the tracking mode dominated by three-lens vision measurement.

[0014] According to the technical solution provided in the embodiments of this application, the process of controlling the lifting device and the six-degree-of-freedom platform to move in coordination is divided into at least two control stages and different control strategies are adopted based on the real-time distance between the lifting lug and the hook: In the first stage, when the distance between the lug and the hook is greater than a first distance threshold, a first control strategy based on position tracking is adopted to control the actual suspended object to quickly approach the target area. In the second stage, when the distance between the lug and the hook is less than or equal to the first distance threshold, the system switches to a second control strategy dominated by force / position hybrid control. While continuing to track the pose, the system introduces sensitivity to contact force to prepare for compliant docking. The control gain of the second control strategy is lower than that of the first control strategy, in order to meet the compliance requirements required for fine docking.

[0015] Secondly, this application proposes a system for automatically mounting suspended objects using trinocular vision measurement, for implementing the aforementioned method, comprising: Ammunition loading vehicle, serving as a mobile transport platform; A lifting device, installed on the loading vehicle, is used to achieve vertical lifting and lowering movements; A six-degree-of-freedom platform, mounted on the lifting device, is used to support and adjust the spatial position and attitude of the suspended object; A three-lens camera is installed on the ammunition loading vehicle or a six-degree-of-freedom platform to capture images of targets inside the aircraft cabin. The in-situ measuring device can be detachably mounted on a simulated or actual suspended object, and is equipped with a high-precision target for tri-lens recognition. Control and computing systems, including: The lower-level machine integrates a communication module, an image processing module, a zero-position calibration module, a measurement module, and an anomaly handling module, which are used to calculate the pose deviation in real time and generate control commands; The host computer communicates with the slave computer and provides a visual operation interface for displaying system status information and receiving operator instructions; The control and calculation system controls the lifting device and the six-degree-of-freedom platform to move in coordination based on the tri-vision measurement results, thereby achieving fully automatic and precise mounting of suspended objects.

[0016] Compared with the prior art, the beneficial effects of this application are as follows: I. Achieved high-precision fully automated mounting, significantly improving operational quality: Through a tri-vision vision system, the system precisely measures the target inside the aircraft cabin, enabling real-time, high-precision acquisition of the position and orientation information of the suspended object and the mounting bracket, and calculating the accurate orientation deviation. Based on this deviation, the lifting device and the six-degree-of-freedom platform coordinate their movements, automatically and precisely adjusting the spatial position and attitude of the suspended object, ultimately achieving precise docking of the lifting lug and hook. This process completely eliminates the uncertainty and random errors of manual operation, improving mounting accuracy to the millimeter level, fundamentally guaranteeing consistency and reliability.

[0017] Second, significantly improve operational efficiency and shorten the preparation time for fighter jets to redeploy: The entire loading process is automated and intelligent, avoiding the time-consuming manual adjustments and trials of traditional methods. The loading vehicle can automatically track and locate, and the suspended object can automatically rise and dock along a predetermined trajectory, significantly shortening the loading operation time that originally required tens of minutes, greatly improving aviation support efficiency and meeting the requirements of rapid response.

[0018] Third, it effectively ensures the safety of personnel and equipment and reduces operational risks: This method avoids situations where operators need to perform delicate operations for extended periods under suspended objects or in close proximity to high-risk areas, fundamentally eliminating major personal safety hazards. At the same time, the automated and precise control effectively prevents collisions and scratches between suspended objects and aircraft pylons caused by improper operation, protecting expensive aviation equipment.

[0019] IV. High System Adaptability and Intelligence: This method uses visual feedback for closed-loop control, enabling it to adapt to subtle differences in different aircraft models and attachment points, demonstrating excellent versatility. The entire system integrates measurement, calculation, control, and verification, forming a complete intelligent attachment solution that represents the advanced development direction of aviation support equipment. Attached Figure Description

[0020] Figure 1 A flowchart illustrating the steps of a method for automatically mounting suspended objects using trinocular vision measurement, as provided in this application embodiment. Detailed Implementation

[0021] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0022] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0023] Example 1 As mentioned in the background section, in view of the problems in the prior art, this application proposes a method for automatically mounting suspended objects using trinocular vision measurement, such as... Figure 1 As shown, it includes the following steps: S1. Acquire images of the target inside the aircraft cabin using a three-eye camera mounted on the ammunition loading vehicle, and obtain positional information related to the actual suspended object and aircraft pylon. S2. Based on the pose information, calculate the pose deviation of the loading vehicle relative to the aircraft cabin design axis in the horizontal, vertical, pitch and rotation directions. S3. Based on the positional deviation, control the lifting device and the six-degree-of-freedom platform to move in coordination to adjust the spatial position and attitude of the actual suspended object, so that the lugs of the actual suspended object can be precisely connected with the hooks of the aircraft rack, and the automatic mounting can be completed.

[0024] Specifically, a trinocular camera refers to three rigorously calibrated cameras mounted on the front of the loading vehicle or a six-DOF platform. These cameras simulate the human eye imaging the same target (an aircraft cabin target) from two slightly different perspectives. The target is typically a high-contrast marker or patterned board with a specific geometry (such as a circle, cross, or coded pattern), fixed in an easily observable location near the aircraft pylon. Pose information is a technical term in robotics and computer vision, short for position and attitude information. Here, it specifically refers to the three-dimensional spatial data related to the actual suspended lugs and aircraft pylon hooks, calculated by processing images acquired by the trinocular cameras and using algorithms such as stereo vision matching and 3D reconstruction. This includes the position coordinates (X, Y, Z) and attitude angles (pitch, roll, yaw) in a three-dimensional coordinate system (e.g., a coordinate system based on the aircraft cabin's design axes).

[0025] Calculating attitude deviations: This step is completed in the lower-level computer (such as an industrial control computer or a high-performance embedded computer). The lower-level computer runs specialized measurement module software, which compares the extracted three-dimensional coordinates of the target, as well as the coordinates of the suspended objects and pylon feature points obtained through coordinate transformation, with the pre-stored theoretical ideal position and attitude based on the aircraft cabin design axis. Through spatial geometric calculation, the final output is the horizontal deviation (left-right offset), vertical deviation (height offset), pitch deviation (forward-backward tilt), and rotation deviation (yaw in the horizontal plane) of the loading vehicle's current attitude relative to the ideal axis. Controlling the lifting device and the six-degree-of-freedom platform to coordinate motion: This is a closed-loop control process. After receiving the real-time attitude deviations calculated by the lower-level computer, the upper-level computer (such as a Windows-based control computer) generates coordinated control commands according to a preset control algorithm (such as PID control). The lifting device (usually a hydraulic or electric cylinder) is responsible for achieving long-stroke vertical lifting and lowering. A six-degree-of-freedom platform (consisting of six electric or servo cylinders) is mounted on a lifting device, enabling translation along the X, Y, and Z axes and rotation around these three axes, providing fine-tuning in six degrees of freedom. The control system drives these two systems in conjunction; for example, the lifting device first raises the suspended object to a near-height position, and then the six-degree-of-freedom platform performs precise position and angle adjustments to jointly correct pose deviations.

[0026] In a preferred embodiment, prior to acquiring the pose information related to the actual suspended object and aircraft pylon, a system calibration is further included, which comprises the following steps: With the six-degree-of-freedom platform in a zero-state state, a simulated suspended object is installed on the six-degree-of-freedom platform, and an in-situ measuring device is installed on the simulated suspended object; The target on the in-situ measuring device is measured by the three-eye camera, and the pose of the simulated suspended object is adjusted according to the measurement data so that its longitudinal position offset is less than or equal to a first preset threshold, its roll angle is less than or equal to a second preset threshold, and the first relative position data between the simulated suspended object and the platform is recorded. The six-degree-of-freedom platform is manipulated to connect the lugs of the simulated suspended object with the aircraft hook. The connection accuracy is controlled so that the longitudinal and lateral relative positions are less than or equal to the third preset threshold, and the height is less than or equal to the fourth preset threshold. The second relative position data of the lugs and the hook are recorded. Based on the first relative position data and the second relative position data, the first position compensation value is calculated; Based on the first position compensation value, the actual suspended object is attached.

[0027] Specifically, system calibration refers to a series of operations performed before the formal mounting operation to establish a precise transformation relationship between the "six-degree-of-freedom platform coordinate system," the "aircraft hook coordinate system," and the "visual measurement coordinate system." It is a preliminary preparatory step. The simulated suspension is a training projectile or counterweight that is completely identical to the actual suspension (such as a missile) in physical dimensions, interfaces, and lug shape, but does not contain a warhead or internal systems. It is used for high-precision calibration operations without damaging the actual ammunition. The in-situ measuring device is a detachable measuring fixture with a high-precision target integrated on it. The target pattern (such as concentric circles or a specific dot matrix) is pre-entered into the vision system database. When the in-situ measuring device is installed on the simulated or actual suspension, the vision system can accurately deduce the spatial pose of a specific point on the suspension (such as the center of the lug) by recognizing the target. The first position compensation value is essentially a spatial transformation matrix. This matrix encapsulates the fixed deviation relationship between the "ideal position of the aircraft hook" and the "position of a reference point on the platform when the six-degree-of-freedom platform is in mechanical zero position." The calculation process is as follows: First, the first relative position data between the simulated suspended object and the platform is recorded step by step to obtain the position P1 of the lifting lug when the platform is at zero position; then, the second relative position data between the lifting lug and the hook is recorded step by step to obtain the position P2 of the hook. Finally, the first position compensation value = P2 - P1. When the actual suspended object is subsequently installed, the control system will add this compensation value to the pose adjustment command calculated in real time, thereby automatically compensating for the inherent installation error of the system.

[0028] In a preferred embodiment, after the system calibration and before acquiring images of the target inside the aircraft cabin using a tri-lens camera mounted on the loading vehicle to obtain pose information related to the suspended object and pylon, the system further includes self-homing and positioning walking, which includes the following steps: The three-lens camera identifies preset navigation markings within the cabin. The loading vehicle is controlled to travel automatically or manually along the preset navigation line until it moves directly below the attachment point, and the three-eye camera is able to observe the target inside the aircraft cabin.

[0029] Specifically, the self-locating and walking armament loading vehicle no longer relies entirely on visual driving by the driver when entering the narrow and complex aircraft cabin. Instead, it uses sensors to automatically identify the path and drive to the designated work point with assistance or autonomously. Pre-defined navigation lines are guide lines with specific colors (such as bright yellow or white) and widths pre-marked on the aircraft cabin floor. Alternatively, they can be arrays of visual tags such as QR codes or Apriltags, forming a visual navigation network.

[0030] Implementation Process: After the ammunition loading vehicle starts, its three-lens camera continuously captures images of the ground ahead. The image processing module (running on the lower-level computer) processes the images in real time, identifying navigation lines through algorithms such as color segmentation, edge detection, and feature extraction. Upon successful identification, the control algorithm calculates the lateral and angular deviations between the vehicle's centerline and the navigation lines. In automatic mode, the control system directly drives the vehicle's steering and travel motors using PID control laws based on these deviations, enabling the vehicle to automatically travel along the center of the lines, achieving unmanned driving. In manual guidance mode, the driver observes the real-time camera feed (similar to a reversing camera) on a handheld remote control to help determine the vehicle's relative position to the lines and manually operates the remote control. In this mode, the system may only provide deviation alerts via audio or visual indicators. The three-lens camera must be able to observe the target inside the aircraft cabin: this is the termination condition for self-homing. When the ammunition loading vehicle reaches the vicinity of the attachment point, it stops through a preset program or by recognizing a specific termination marker. At this point, by adjusting the gimbal or directly utilizing a wide-angle lens, the tri-lens camera mounted on the six-degrees-of-freedom platform is ensured to clearly see the target located on the top or side wall of the aircraft cabin, preparing for subsequent precise attitude measurement and automatic loading. The system treats the loading vehicle as a mobile robot, using navigation markings as a predefined global path. The vision system, acting as an attitude sensor, provides real-time local positioning information of the vehicle relative to the path. The control system, as the decision-making and execution unit, continuously reduces the lateral and directional errors between the vehicle and the path through feedback control, thereby achieving precise path tracking.

[0031] In a preferred embodiment, the controlled lifting device and the six-degree-of-freedom platform move in coordination to adjust the spatial position and attitude of the actual suspended object, and the attachment verification includes the following steps: After the lifting lug and the hook are connected, the six-degree-of-freedom platform is controlled to slowly fall. By monitoring changes in the force state of the platform, it can be determined whether the actual suspended object has been reliably attached to the hook; When an abnormal force condition is detected, the descent of the six-degree-of-freedom platform is paused and an alarm is issued.

[0032] Specifically, docking verification refers to confirming, through a non-visual, physical sensing method, whether the suspended object has been safely and reliably mounted on the aircraft pylon after the system deems docking complete. Monitoring changes in the platform's stress state is the core of the verification. In practice, force sensors on each support leg of the six-degree-of-freedom platform or load cells at the connection between the platform and the lifting device monitor changes in the platform's load in real time. After the control system issues a docking completion command, it will then issue a command for the platform to slowly descend.

[0033] Verification logic: Normal scenario (successful attachment): The platform begins to descend and detaches from the suspended object. At this moment, the entire weight of the suspended object (originally borne by the platform) is instantly transferred to the aircraft's hook. The force sensor detects a steep, large pressure drop signal. This specific pressure change curve is recognized by the system as a characteristic signal of "successful attachment".

[0034] Abnormal Situation (Hanging Failure): If, for some reason (such as the lifting lug not being fully inserted into the slot or interference), the suspended object fails to be successfully hung, the platform will still support the suspended object when it falls, or only a portion of the weight will be transferred. In this case, the pressure drop detected by the force sensor will be much smaller than expected, or the descent curve will be abnormal. The system will immediately determine this as an "abnormal force state".

[0035] "Pause descent and issue an alarm": Once an anomaly is detected, the control system will immediately halt the descent process and may instruct the platform to rise slightly to eliminate the risk. Simultaneously, a red warning box will pop up on the host computer interface, a rapid beeping sound will be emitted, and an alarm may even be sent to the ground command station via wireless network, prompting operators that "attachment is suspected to have failed and requires immediate manual verification." Operators must approach the site to observe, confirm the problem, and troubleshoot the fault before instructing the system to continue or restart the attachment process.

[0036] The technical principle of this implementation is based on binary state detection using mechanical feedback. It converts complex mechanical docking states into a quantifiable physical signal (pressure value) through a force sensor. The system monitors the dynamic response pattern of this signal under a specific action (platform drop) to non-contactly and reliably determine a binary state: whether the suspended object is "securely attached" or "not securely attached".

[0037] In a preferred embodiment, after calculating the first position compensation value based on the first relative position data and the second relative position data, and before mounting the actual suspended object based on the first position compensation value, the following steps are further included: Determine whether the deviation between the initial pose of the actual suspended object and the zero pose of the six-degree-of-freedom platform exceeds the compensable range of the six-degree-of-freedom platform during the mounting process. The process of attaching the actual suspended object based on the first position compensation value includes the following steps: If not, then the actual suspended object is mounted based on the first position compensation value.

[0038] Specifically, the initial pose of the actual suspended object refers to its natural state in space when it is first mounted on a six-degree-of-freedom platform (such as a real missile) but before any pose adjustment has been made. This state may differ from the ideal state due to manufacturing tolerances of the suspended object itself, clamping stress, and minor deviations in the platform's clamping mechanism. The compensable range of the six-degree-of-freedom platform during mounting is a pre-set software threshold. It defines the remaining range of motion and capability of the six-degree-of-freedom platform when it is already carrying the suspended object. This range takes into account the platform's physical limits, motion speed, stability, and control accuracy. For example, the platform might be specified to have a real-time vertical adjustment range of only ±50 mm and an attitude angle adjustment range of ±3 degrees when fully loaded. This range is much smaller than the platform's unloaded full travel.

[0039] Judgment and Decision Implementation: After the actual suspended object is installed on the platform, the system does not immediately begin the attachment process. Instead, it first executes a rapid measurement procedure: the operator installs the in-situ measuring device onto the actual suspended object, and the system measures the target using a tri-lens camera, quickly calculating the deviation of the actual suspended object from the zero-position attitude of the six-DOF platform. Next, the control system compares this measured initial pose deviation with the compensable range stored in the system.

[0040] If not (deviation not exceeded): This indicates that the initial state of the actual suspended object is good and within the platform's capabilities. The system then determines that the first position compensation value obtained through previous simulation of the suspended object can be directly applied. The system will prompt the operator to remove the in-situ measuring device and begin the automatic mounting process based on the first position compensation value. This is a high-efficiency mode suitable for suspended objects of the same model with good batch consistency and standardized installation.

[0041] This implementation achieves the following: Achieving an intelligent balance between efficiency and accuracy: For suspended objects in good condition, lengthy secondary calibration is skipped, greatly improving operational efficiency; for those in poor condition, enhancement measures are activated to ensure ultimate accuracy, realizing intelligent resource allocation. Enhancing system robustness: It can cope with individual differences in suspended objects and random errors in initial installation, preventing the entire installation task from failing or exceeding accuracy limits due to abnormal conditions of individual suspended objects. Preventing equipment from operating beyond its limits: By judging and avoiding forcibly driving the six-degree-of-freedom platform to compensate for a deviation beyond its capacity, the platform mechanism is protected, preventing potential mechanical damage or control instability.

[0042] Furthermore, after determining whether the deviation between the initial pose of the actual suspended object and the zero-position pose of the six-degree-of-freedom platform exceeds the compensable range of the six-degree-of-freedom platform during the mounting process, the method further includes the following steps: If so, the in-situ measuring device is installed on the actual suspended object; The target on the in-situ measuring device is measured by the three-lens camera, and the pose of the actual suspended object is adjusted according to the measurement data; The actual suspended object is fixed on the six-degree-of-freedom platform, and the third relative position data between the actual suspended object and the six-degree-of-freedom platform is recorded; Based on the first relative position data, the second relative position data, and the third relative position data, a second position compensation value is calculated for mounting the actual suspended object.

[0043] Specifically, actual suspended object installation calibration: This is a proprietary calibration performed on a specific individual suspended object to obtain a more accurate position compensation value specific to that suspended object. Third relative position data: This refers to the relative pose relationship between the actual suspended object and the six-DOF platform after the actual suspended object has been adjusted to a good initial state. This good initial state means that, through adjustments, the pose of the actual suspended object has entered the aforementioned compensable range.

[0044] Implementation process: Installation and Measurement: The in-situ measuring device is installed on the actual suspended object and measured by a three-lens camera. At this point, the system recognizes that the initial pose deviation is too large.

[0045] Adjusting the orientation: Based on the measurement results, the system will prompt the operator to manually adjust the bracket or clamping mechanism of the actual suspended object (or directly control the platform in some more automated versions) to ensure its orientation meets the requirements (e.g., to keep the suspended object within 1 / 5 of the adjustment range of the six-degree-of-freedom platform). The purpose of this step is to establish a stable and controllable "working reference" for the suspended object.

[0046] Record the third relative position data: After adjustment and fixation, the system again uses a tri-lens camera to accurately measure the target of the in-situ measuring device and records the third relative position data between the actual suspended object and the platform. After recording, remove the in-situ measuring device.

[0047] Calculating the second position compensation value: The system fuses the previously recorded first relative position data (relationship between the simulated projectile and the platform), second relative position data (relationship between the hook and the simulated projectile), and new third relative position data (relationship between the current actual projectile and the platform) with the coordinate system. Finally, a second position compensation value is calculated. This compensation value integrates the system's inherent errors and the individual installation errors of the current actual suspended object, providing the ultimate solution for this specific suspended object.

[0048] Furthermore, after completing the installation and calibration of the actual suspended object and obtaining the third relative position data, the following calculation steps are performed to obtain the second position compensation value: First, the system retrieves pre-stored calibration data, namely, first relative position data and second relative position data. The first relative position data defines the spatial pose of the simulated suspended object relative to the platform when the six-degree-of-freedom platform is in a zero-state state. The second relative position data defines the spatial pose of the hook relative to the simulated suspended object when the simulated suspended object successfully docks with the aircraft hook.

[0049] Next, the system fuses the first relative position data with the second relative position data. This fusion process essentially involves coordinate transformation, the purpose of which is to calculate an intermediate reference quantity—the absolute spatial pose relationship of the aircraft hook relative to the six-DOF platform in its zero-position state. This intermediate reference quantity comprehensively reflects the fixed correlation between the aircraft hook's position and the platform's mechanical zero position; it includes the system's inherent installation errors and serves as the spatial reference for all subsequent attachment actions.

[0050] The system then incorporates the latest measured third relative position data. This data defines the spatial pose relationship of the actual suspended object, now with its pose adjusted, relative to the six-DOF platform in the current state.

[0051] Finally, the system synthesizes the third relative position data with the intermediate reference value. The core of this synthesis calculation lies in using the actual suspended object as the viewpoint reference to calculate the direct relative pose relationship between it and the aircraft hook. Through this calculation, the second position compensation value is finally output.

[0052] The second position compensation value is a spatial pose transformation command that includes three-dimensional translation and three-dimensional rotation. This command precisely specifies the final pose adjustment that the six-DOF platform needs to drive the actual suspended object to complete the mounting. This calculation method effectively eliminates the combined effects of inherent system errors and individual installation errors of the suspended object by integrating the calibration data from the system's earlier stages and the measurement data for the specific suspended object in the current stage. This provides a direct and reliable control basis for achieving extremely high-precision automatic mounting of individual suspended objects.

[0053] In a preferred embodiment, the controlled lifting device moves in coordination with a six-degree-of-freedom platform to adjust the spatial position and attitude of the suspended object, including the following steps: Based on the posture deviation and / or the second position compensation value, the kinematic model is used to calculate and generate a predetermined upward trajectory for the lug of the actual suspended object to move from its current position to precisely dock with the hook. During the lifting process, the target inside the aircraft cabin is identified in real time by a three-eye camera, and the positional deviation is dynamically updated. The pose deviation calculated in real time is compared with the expected pose of the predetermined ascent trajectory to generate a tracking error; Based on the tracking error, the combined motion of the lifting device and the six-degree-of-freedom platform is controlled in real time, so that the lugs of the actual suspended object move along the predetermined upward trajectory until they dock with the hook.

[0054] Specifically, the planned ascent trajectory is not a simple straight path. It is a spatial curve planned in three-dimensional space by the control and calculation system based on a kinematic model, moving from the current position (at the start of lifting) to the target position (precise docking with the hook). This trajectory is continuous and smoothly changing in position and attitude, comprehensively considering factors such as obstacle avoidance, motion stability, and optimal time. Kinematic model calculation: The lower-level machine integrates the kinematic model of the loading vehicle lifting device and the six-degree-of-freedom platform. This model describes the mathematical relationship between the motion of each joint of the platform (lifting cylinder, six electric cylinders) and the position and attitude of the end effector (i.e., the suspended object's lifting lug) in space. When planning the trajectory, the system uses an inverse kinematics algorithm to decompose the spatial trajectory of the lifting lug into the desired motion sequence of each joint. Dynamic update of pose deviation and tracking error: During the lifting process, the three-eye camera continuously identifies targets inside the aircraft cabin, and the measurement module calculates a new pose deviation in real time, reflecting the current instantaneous state. This real-time pose deviation is used to calculate the actual pose of the lifting lug. Subsequently, the system compares the actual pose with the desired pose corresponding to the predetermined ascent trajectory at the current moment, and the difference is the tracking error. Based on the generated tracking error, the control system adjusts the control commands (such as speed and position commands) sent to the lifting device and the various drives of the six-degree-of-freedom platform in real time, forming a high-frequency closed-loop control. The purpose is to eliminate this tracking error, so that the actual pose of the lifting lug closely follows the desired pose on the predetermined ascent trajectory, and moves steadily along the predetermined path like a car with cruise control until docking.

[0055] In a preferred embodiment, the method further includes the following steps: The operating status of the three-lens camera is monitored in real time, including image clarity, target recognition confidence, or data transmission continuity. The lifting lugs of the actual suspended object move along the predetermined upward trajectory, including the following steps: If the operating state is abnormal, it will automatically switch to the pose prediction and tracking mode based on the inertial measurement unit. The pose prediction and tracking modes include: Acquire acceleration and angular velocity data output by an inertial measurement unit installed on the actual suspended object or six-degree-of-freedom platform; Based on the acceleration and angular velocity data and the effective visual pose of the previous moment, predict the first current pose of the actual suspended object; Based on the first current pose and the desired pose of the predetermined upward trajectory, control commands are generated to cause the lifting device and the six-degree-of-freedom platform to move together, so as to maintain the lug moving along the predetermined upward trajectory. After the three-lens camera returns to normal operation, it automatically switches back to the tracking mode dominated by three-lens vision measurement.

[0056] Specifically, the operating status of the tri-lens camera: The system continuously performs self-diagnosis of the working health of the tri-lens camera, and the monitoring indicators include: Image sharpness: Determines whether a lens is contaminated or out of focus by calculating the gradient or contrast of the image.

[0057] Target recognition confidence score: When recognizing a target, the image processing algorithm will output a confidence score (between 0 and 1). A score that is too low indicates that it may be affected by occlusion, strong light, or severe motion blur.

[0058] Data transmission continuity: Check whether the image data stream from the camera is continuous and without frame drops.

[0059] An inertial measurement unit (IMU) is a microelectromechanical system (MEMS) sensor that typically includes a three-axis accelerometer and a three-axis gyroscope, and sometimes a magnetometer. It is rigidly mounted on a six-degree-of-freedom platform or an actual suspended object to measure its own acceleration and angular velocity.

[0060] Pose Prediction Tracking Mode: Trigger Condition: When the system detects an abnormal operating status of the three-lens camera (such as a confidence level consistently below the threshold, image loss, etc.), it immediately and automatically switches to this mode. Data Acquisition and Prediction: The system begins high-speed reading of the acceleration and angular velocity data from the inertial measurement unit. Combining this with the previously provided pose data from the vision system, considered valid, as the initial state, the system predicts the first current pose using an inertial navigation algorithm (integrating acceleration once to obtain velocity, integrating twice to obtain position; integrating angular velocity to obtain angle). Maintaining Tracking: Based on this predicted pose, the control system continues to compare and perform closed-loop control with the predetermined ascent trajectory, ensuring the suspended object maintains its original trajectory. Automatic Recovery: The system continuously monitors the vision status. Once the camera returns to normal (e.g., the obstruction is removed), after a brief data verification, it automatically switches back to the tracking mode primarily based on high-precision three-lens vision measurement. The inertial measurement unit data continues to serve as an auxiliary, undergoing Kalman filtering fusion with the vision data to smooth the trajectory and suppress noise.

[0061] In a preferred embodiment, the process of controlling the lifting device to move in coordination with the six-degree-of-freedom platform is divided into at least two control stages and employs different control strategies based on the real-time distance between the lifting lug and the hook: In the first stage, when the distance between the lug and the hook is greater than a first distance threshold, a first control strategy based on position tracking is adopted to control the actual suspended object to quickly approach the target area. In the second stage, when the distance between the lug and the hook is less than or equal to the first distance threshold, the system switches to a second control strategy dominated by force / position hybrid control. While continuing to track the pose, the system introduces sensitivity to contact force to prepare for compliant docking. The control gain of the second control strategy is lower than that of the first control strategy, in order to meet the compliance requirements required for fine docking.

[0062] Specifically, the real-time distance refers to the straight-line spatial distance between the actual suspended object's lug and the aircraft hook, measured and calculated in real time by the tri-vision system during the lifting process. This distance is dynamically updated and serves as the basis for determining the control phase switching. The "first distance threshold" is a preset empirical value or a calculated critical distance parameter, for example, set to 50 mm. It marks the boundary where a fundamental shift in the control strategy is required. When the distance is greater than this value, the goal is rapid approach; when it is less than or equal to this value, the goal is precise docking. The first control strategy (position tracking-dominated): In this phase, the control system primarily focuses on pose deviation. It drives the lifting device and the six-degree-of-freedom platform with higher control gain (i.e., a faster and more rigid system response), ensuring the actual suspended object moves rapidly along a predetermined ascent trajectory, aiming to deliver the suspended object to the vicinity of the target area, i.e., the boundary of the first distance threshold, in the shortest possible time. The second control strategy (force / position hybrid control-dominated): When the distance between the lug and the hook enters the range of the first distance threshold, the system immediately switches to this strategy. At this point, the control objective changes from simple position tracking to coordinated control of position and contact force. Introducing sensitivity to contact force: In this mode, in addition to receiving visual pose feedback, the control system begins to closely monitor the readings of the force sensors mounted on the six-degree-of-freedom platform. The control algorithm is designed to be highly sensitive to minute changes in contact force. Control gain is lower than the first control strategy: To achieve "compliant docking," the system actively reduces the control gain at this stage, making the system response softer. This means that when unavoidable minor alignment errors or contact occur, the platform will not produce rigid, resistant movement, but will instead make slight, compliant adjustments based on force feedback, guiding the lifting lug into the hook slot, thereby avoiding large impacts or jamming.

[0063] This implementation method is based on variable gain control and force / position hybrid control. It recognizes that the dynamic performance requirements of the control system are contradictory at different stages of the mounting process: high gain is needed at long distances to ensure speed and trajectory accuracy, while low gain is needed at short distances to ensure compliance and safety. By switching the control strategy and gain according to a specific physical quantity (real-time distance), the conflicting requirements of different stages are balanced.

[0064] Example 2 Based on Example 1, this example proposes a system for automatically mounting suspended objects using trinocular vision measurement, to implement the method described above, including: Ammunition loading vehicle, serving as a mobile transport platform; A lifting device, installed on the loading vehicle, is used to achieve vertical lifting and lowering movements; A six-degree-of-freedom platform, mounted on the lifting device, is used to support and adjust the spatial position and attitude of the suspended object; A three-lens camera is installed on the ammunition loading vehicle or a six-degree-of-freedom platform to capture images of targets inside the aircraft cabin. The in-situ measuring device can be detachably mounted on a simulated or actual suspended object, and is equipped with a high-precision target for tri-lens recognition. Control and computing systems, including: The lower-level machine integrates a communication module, an image processing module, a zero-position calibration module, a measurement module, and an anomaly handling module, which are used to calculate the pose deviation in real time and generate control commands; The host computer communicates with the slave computer and provides a visual operation interface for displaying system status information and receiving operator instructions; The control and calculation system controls the lifting device and the six-degree-of-freedom platform to move in coordination based on the tri-vision measurement results, thereby achieving fully automatic and precise mounting of suspended objects.

[0065] Specifically, the loading vehicle, serving as the system's mobile base, possesses a walking drive mechanism responsible for carrying all the mounted equipment into the aircraft cabin. The lifting device, integrated on the loading vehicle, is typically a telescopic mechanism composed of hydraulic cylinders or electric screws, providing a large-stroke vertical lifting motion for the suspended object. The six-DOF platform, mounted on top of the lifting device, is a parallel robotic mechanism capable of precise pose adjustment of the load in three directions (translation in three directions and rotation in three directions) within three-dimensional space. The tri-lens camera, the system's core sensor, consists of two precisely calibrated cameras, fixedly mounted at appropriate locations on the loading vehicle or the six-DOF platform to ensure its field of view covers the target inside the aircraft cabin. The in-situ measuring device is a crucial detachable measuring fixture with a high-precision target pattern (such as a specific arrangement of circles or coded markers) that can be stably recognized by the tri-lens vision system. It is temporarily installed at designated measurement points on the simulated or actual suspended object via a standardized interface, transmitting the suspended object's pose information to the vision system through the target.

[0066] Specifically, the control and computing system adopts a distributed architecture and is physically divided into: Lower-level machine: Typically an industrial computer or high-performance PLC installed at the equipment site, responsible for low-level, high real-time tasks. Its integrated software modules have the following functions: Communication module: Manages the communication links of all hardware devices.

[0067] Image processing module: Runs computer vision algorithms to identify and extract target features from tricular images in real time.

[0068] Zero-position calibration module: Executes the system's initial calibration process.

[0069] Measurement Module: This module measures the attitude angles and horizontal / vertical deviations of the loading vehicle in the engineering coordinate system in real time. Normal operation of this module requires that communication connections between all components be established, and that the image processing and zero-point calibration modules be completed. Based on the measurement principle, the obtained data is fed into the measurement model for calculation.

[0070] Error Handling Module: This module detects any abnormalities in hardware device communication and system operation. If an abnormality occurs, the lower-level machine reports the error type to the upper-level machine. The upper-level operator then performs corresponding actions based on the error type; for example, if hardware device communication is abnormal, the machine can reconnect; if system operation is abnormal, the system can be restarted.

[0071] Host computer: Typically a computer located on the control panel, responsible for human-computer interaction. It receives data from the slave computer, displays the attitude angles, translation amounts, and other status information of the six-degree-of-freedom platform in real time on a graphical interface, and provides the operator with control elements such as buttons and sliders to send commands or switch modes.

[0072] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. The above descriptions are only preferred embodiments of this application. It should be noted that due to the limitations of written expression, while there are objectively infinite specific structures, those skilled in the art can make several improvements, modifications, or changes without departing from the principles of this invention, and can also combine the above technical features in an appropriate manner. These improvements, modifications, changes, or combinations, or the direct application of the inventive concept and technical solution to other situations without modification, should all be considered within the scope of protection of this application.

Claims

1. A method for automatically mounting suspended objects using trinocular vision measurement, characterized in that, Includes the following steps: Images of targets inside the aircraft cabin are captured by a three-eye camera mounted on the ammunition loading vehicle to obtain positional information related to the actual suspended objects and aircraft pylons. Based on the pose information, the pose deviation of the loading vehicle relative to the aircraft cabin design axis is calculated in the horizontal, vertical, pitch and rotation directions. Based on the positional deviation, the lifting device and the six-degree-of-freedom platform are controlled to move in coordination to adjust the spatial position and attitude of the actual suspended object, so that the lugs of the actual suspended object can be precisely connected with the hooks of the aircraft rack to complete the automatic mounting. Prior to acquiring the pose information related to the actual suspended objects and aircraft pylons, system calibration is also included, which comprises the following steps: With the six-degree-of-freedom platform in a zero-state state, a simulated suspended object is installed on the six-degree-of-freedom platform, and an in-situ measuring device is installed on the simulated suspended object; The target on the in-situ measuring device is measured by the three-eye camera, and the pose of the simulated suspended object is adjusted according to the measurement data so that its longitudinal position offset is less than or equal to a first preset threshold, its roll angle is less than or equal to a second preset threshold, and the first relative position data between the simulated suspended object and the platform is recorded. The six-degree-of-freedom platform is manipulated to connect the lugs of the simulated suspended object with the aircraft hook. The connection accuracy is controlled such that the longitudinal and lateral relative positions are less than or equal to the third preset threshold, and the height is less than or equal to the fourth preset threshold. The second relative position data of the lugs and the hook are recorded. Based on the first relative position data and the second relative position data, the first position compensation value is calculated; Based on the first position compensation value, the actual suspended object is attached; After calculating the first position compensation value based on the first relative position data and the second relative position data, and before hanging the actual suspended object based on the first position compensation value, the following steps are also included: Determine whether the deviation between the initial pose of the actual suspended object and the zero pose of the six-degree-of-freedom platform exceeds the compensable range of the six-degree-of-freedom platform during the mounting process. The process of attaching the actual suspended object based on the first position compensation value includes the following steps: If not, then the actual suspended object is mounted based on the first position compensation value; After determining whether the deviation between the initial pose of the actual suspended object and the zero-position pose of the six-degree-of-freedom platform exceeds the compensable range of the six-degree-of-freedom platform during the mounting process, the method further includes the following steps: If so, the in-situ measuring device is installed on the actual suspended object; The target on the in-situ measuring device is measured by the three-lens camera, and the pose of the actual suspended object is adjusted according to the measurement data; The actual suspended object is fixed on the six-degree-of-freedom platform, and the third relative position data between the actual suspended object and the six-degree-of-freedom platform is recorded; Based on the first relative position data, the second relative position data, and the third relative position data, a second position compensation value is calculated for mounting the actual suspended object. The control lifting device moves in coordination with the six-degree-of-freedom platform to adjust the spatial position and attitude of the actual suspended object, including the following steps: Based on the posture deviation and / or the second position compensation value, the kinematic model is used to calculate and generate a predetermined upward trajectory for the lug of the actual suspended object to move from its current position to precisely dock with the hook. During the lifting process, the target inside the aircraft cabin is identified in real time by a three-eye camera, and the positional deviation is dynamically updated. The pose deviation calculated in real time is compared with the expected pose of the predetermined ascent trajectory to generate a tracking error; Based on the tracking error, the combined motion of the lifting device and the six-degree-of-freedom platform is controlled in real time, so that the lugs of the actual suspended object move along the predetermined upward trajectory until they dock with the hook. The method also includes the following steps: The operating status of the three-lens camera is monitored in real time, including image clarity, target recognition confidence, or data transmission continuity. The lifting lugs of the actual suspended object move along the predetermined upward trajectory, including the following steps: If the operating state is abnormal, it will automatically switch to the pose prediction and tracking mode based on the inertial measurement unit. The pose prediction and tracking modes include: Acquire acceleration and angular velocity data output by an inertial measurement unit installed on the actual suspended object or six-degree-of-freedom platform; Based on the acceleration and angular velocity data and the effective visual pose of the previous moment, predict the first current pose of the actual suspended object; Based on the first current pose and the desired pose of the predetermined upward trajectory, control commands are generated to cause the lifting device and the six-degree-of-freedom platform to move together, so as to maintain the lug moving along the predetermined upward trajectory. After the three-lens camera returns to normal operation, it automatically switches back to the tracking mode dominated by three-lens vision measurement.

2. The method for automatically mounting suspended objects using trinocular vision measurement according to claim 1, characterized in that: After the system calibration and before acquiring images of the target inside the aircraft cabin using a three-eye camera mounted on the loading vehicle to obtain pose information related to the actual suspended objects and pylons, the process also includes self-homing and positioning walking, which includes the following steps: The three-lens camera identifies preset navigation markings within the cabin. The loading vehicle is controlled to travel automatically or manually along the preset navigation line until it moves directly below the attachment point, and the three-eye camera is able to observe the target inside the aircraft cabin.

3. The method for automatically mounting suspended objects using trinocular vision measurement according to claim 1, characterized in that: The control lifting device, in coordination with the six-degree-of-freedom platform, adjusts the spatial position and attitude of the actual suspended object, and also includes attachment verification; the attachment verification includes the following steps: After the lifting lug and the hook are connected, the six-degree-of-freedom platform is controlled to slowly fall. By monitoring changes in the force state of the platform, it can be determined whether the actual suspended object has been reliably attached to the hook; When an abnormal force condition is detected, the descent of the six-degree-of-freedom platform is paused and an alarm is issued.

4. The method for automatically mounting suspended objects using trinocular vision measurement according to claim 1, characterized in that: The process of controlling the lifting device to move in coordination with the six-degree-of-freedom platform is divided into at least two control stages and different control strategies are adopted based on the real-time distance between the lifting lug and the hook: In the first stage, when the distance between the lug and the hook is greater than a first distance threshold, a first control strategy based on position tracking is adopted to control the actual suspended object to quickly approach the target area. In the second stage, when the distance between the lug and the hook is less than or equal to the first distance threshold, the system switches to a second control strategy dominated by force / position hybrid control. While continuing to track the pose, the system introduces sensitivity to contact force to prepare for compliant docking. The control gain of the second control strategy is lower than that of the first control strategy, in order to meet the compliance requirements required for fine docking.

5. A system for automatically mounting suspended objects using trinocular vision measurement, for implementing the method described in any one of claims 1-4, characterized in that, include: Ammunition loading vehicle, serving as a mobile transport platform; A lifting device, installed on the loading vehicle, is used to achieve vertical lifting and lowering movements; A six-degree-of-freedom platform, mounted on the lifting device, is used to support and adjust the spatial position and attitude of the simulated or actual suspended object; A three-lens camera is installed on the ammunition loading vehicle or a six-degree-of-freedom platform to capture images of targets inside the aircraft cabin. The in-situ measuring device can be detachably mounted on a simulated or actual suspended object, and is equipped with a high-precision target for tri-lens recognition. Control and computing systems, including: The lower-level machine integrates a communication module, an image processing module, a zero-position calibration module, a measurement module, and an anomaly handling module, which are used to calculate the pose deviation in real time and generate control commands; The host computer communicates with the slave computer and provides a visual operation interface for displaying system status information and receiving operator instructions; The control and calculation system controls the lifting device and the six-degree-of-freedom platform to move in coordination based on the tri-vision measurement results, thereby achieving fully automatic and precise mounting of the actual suspended object.

Images