A high-altitude rescue target automatic positioning and alignment engagement system and method based on visual topology correlation

Abstract

This invention discloses an automatic target positioning and alignment system and method for high-altitude rescue based on visual topological association. Building images are acquired through visible light and infrared sensors integrated at the end effector, and a dual-branch deep learning model is used to identify balcony arrays and trapped personnel in parallel. Geometric features of the balconies are extracted to construct a two-dimensional topological matrix, and a fusion association algorithm based on overlap weight and center distance compensation is used to automatically determine the balcony number and spatial location of the trapped person. Intrinsic parameters, laser ranging, and boom attitude data are fused, and pixel coordinates are converted into three-dimensional coordinates of the rescue vehicle base through kinematic modeling. During the approach process, visual servo closed-loop control is switched to correct the motion vector in real time, achieving flexible and precise alignment between the platform and the balcony edge, with an alignment accuracy of ≤±5cm. This invention significantly improves the accuracy of target positioning and the safety of automated docking in complex rescue environments.

Description

Technical Field

[0001] This invention belongs to the field of high-altitude rescue technology, specifically relating to a high-altitude rescue target automatic positioning and alignment system and method based on visual topological association. It is applicable to rescue operations for trapped personnel in disaster scenarios such as high-rise building fires and earthquakes, and can realize automatic positioning, precise alignment and flexible connection between the rescue platform and the target balcony, thereby improving the intelligence level and operational safety of high-altitude rescue. Technical Background

[0002] With the development of modern urban construction and emergency rescue technology, the level of intelligence in fire and rescue equipment has significantly improved. High-altitude rescue platforms, as key equipment in this process, are now widely used for rescue operations in open areas such as balconies of high-rise buildings. Automatic alignment and engagement are crucial technologies in high-altitude rescue, directly impacting the efficiency and safety of rescue operations. Traditional manual operation requires operators (rescuers) to continuously monitor the relative position of the rescue platform and the balcony visually, increasing the labor intensity and operational risks for rescuers.

[0003] When a high-altitude rescue platform docks with a target balcony, the docking process is prone to positioning errors and delays in rescue operations. Furthermore, the safety of docking with balconies in complex building environments (such as multi-story buildings with multiple balconies, insufficient lighting, or smoke obstruction) and under the influence of wind and vibration is poor, posing a challenge to automated mechanical rescue. Currently, most high-altitude rescue equipment on the market uses single-vision detection, failing to achieve the correlation between the trapped person and the target balcony, and lacks a flexible docking mechanism with closed-loop control. Therefore, how to optimize the automatic alignment and docking functions of high-altitude rescue platforms through intelligent systems and methods to ensure the positioning accuracy and safety of docking operations has become an urgent problem to be solved by researchers in this field. Summary of the Invention

[0004] To address the aforementioned shortcomings in the existing technology, this invention provides an automatic positioning and alignment method for high-altitude rescue targets based on visual topological association, so as to achieve precise positioning and flexible docking of high-altitude rescue platforms during docking operations.

[0005] A method for automatic localization and alignment of high-altitude rescue targets based on visual topological association, characterized by the following steps:

[0006] Step 1: In complex rescue environments, a visual sensing unit integrating a high-definition camera and an infrared thermal imager acquires visible light and infrared thermal images of the high-rise building facade in real time. A dual-branch deep learning model is used for parallel processing. This model includes a visible light feature extraction branch and an infrared feature extraction branch. The visible light feature extraction branch processes the visible light images acquired by the high-definition camera to extract the geometric contour and texture features of the building facade and balconies. The infrared feature extraction branch processes the infrared thermal images acquired by the infrared thermal imager to extract the thermal radiation features of the target area. A feature fusion module concatenates and fuses the feature maps output by the visible light and infrared feature extraction branches. Based on the fused feature maps, target detection is performed to identify all balcony targets in the image and generate balcony target bounding boxes. Extract the geometric distribution features of each balcony to establish a two-dimensional topological matrix. ;

[0007] Step 2: Use an object detection model to identify trapped individuals in the image and generate bounding boxes for the trapped individuals. Constructing the correlation contribution function Calculate the degree of association between each trapped person target and each balcony target, automatically lock the balcony with the highest degree of association as the target balcony, and determine the target balcony number n*;

[0008] Step 3: Integrate the intrinsic parameters K of the visual sensor, the real-time ranging value d collected by the laser rangefinder, and the real-time attitude data collected by the boom attitude sensor of the rescue vehicle, and solve the real-time transformation matrix between the camera coordinate system and the rescue vehicle base coordinate system through boom kinematic modeling. The image pixel coordinates of the target balcony are converted into three-dimensional points in the camera coordinate system. Then convert to the three-dimensional coordinate system of the rescue vehicle base. , thus obtaining the spatial coordinates of the target balcony;

[0009] Step 4: Based on the converted 3D coordinates, drive the rescue vehicle boom to move the end platform toward the target balcony. When the distance between the platform and the target balcony is less than a preset threshold, switch to local fine recognition mode to extract the edge linear features of the balcony railing. Use a PID-Fuzzy composite visual servo strategy to correct the motion vector in real time. Adjust the platform posture by controlling the hydraulic proportional valve to achieve parallel alignment between the end platform and the balcony railing plane, and finally complete the flexible connection.

[0010] The aforementioned automatic positioning and alignment method for high-altitude rescue targets, by employing a dual-branch deep learning model and a correlation contribution function, achieves accurate correlation and positioning between trapped personnel and the target balcony, thus avoiding false detection and missed detection problems in complex environments.

[0011] The correlation contribution function The calculation formula is: ;

[0012] in, and These are the weighting coefficients, and ; The overlapping area between the target frame of the trapped person and the target frame of the balcony; and These are the geometric center vectors of the target frames for the trapped personnel and the balcony, respectively. Let be the spatial smoothing factor; take The balcony index n* corresponding to the maximum value is used as the target balcony. When the maximum value is less than the preset threshold, it is determined to be a positioning anomaly, triggering a re-identification process.

[0013] The coordinate transformation logic in step 3 specifically includes:

[0014] S3.1: Calculate the 3D point in the camera coordinate system based on the visual sensor intrinsic parameter matrix K and the real-time ranging value d. The calculation formula is: ;

[0015] Where K is the camera intrinsic parameter matrix, The pixel coordinates of the target balcony in the image;

[0016] S3.2: By modeling the boom's kinematics and combining it with the real-time attitude data of the rescue vehicle's boom, solve for the transformation matrix of the camera coordinate system relative to the rescue vehicle's base coordinate system. ;

[0017] S3.3: Obtain the three-dimensional points of the target balcony in the coordinate system of the rescue vehicle base through matrix multiplication. The calculation formula is:

[0018] Furthermore, the local fine recognition and flexible joining logic in step 4 specifically includes: in the local fine recognition mode, an edge detection algorithm is used to extract the linear features of the balcony railing, and a straight line fitting algorithm is used to fit the straight line of the railing edge to calculate the spatial angle between the plane where the railing is located and the front end plane of the end platform; by adjusting the pitch angle and tilt angle of the end platform in real time, the angle between the two planes meets the preset parallelism accuracy requirements to achieve parallel alignment of the two planes; during the flexible joining process, a pressure sensor is used to detect the contact pressure between the platform and the balcony railing in real time, and when the contact pressure reaches the preset safe contact threshold, the feed movement of the platform is stopped.

[0019] The present invention also provides an automatic positioning and alignment system for high-altitude rescue targets based on visual topological association, including a sensing module, a processing module and an execution module.

[0020] The sensing module is used to collect images of building facades, thermal imaging information of trapped personnel, and real-time ranging data; the processing module is used to execute a dual-branch deep learning recognition algorithm, a topological association algorithm, and coordinate transformation operations, receive real-time data from the sensing module, and output motion control commands; the execution module is used to receive motion path commands output by the processing module, drive the rescue vehicle boom and end platform to move, achieve target alignment and flexible engagement, and simultaneously feed back real-time motion status data to the processing module to form closed-loop control.

[0021] The perception module includes a high-definition camera, an infrared thermal imager, and a laser rangefinder; the processing module integrates a deep learning inference engine and a microprocessor; the execution module is connected to the rescue vehicle chassis and boom controller, and includes a hydraulic proportional valve, a servo motor, and an attitude adjustment mechanism. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the module connections of the high-altitude rescue automatic alignment system of the present invention;

[0023] Figure 2 This is a flowchart of the steps of the method of the present invention;

[0024] Figure 3 This is a schematic diagram illustrating the construction of a two-dimensional topological matrix for a building facade.

[0025] Figure 4 A schematic diagram illustrating the calculation logic of the correlation contribution function;

[0026] Figure 5 This is a flowchart illustrating the spatial coordinate transformation process.

[0027] Figure 6 This is a schematic diagram illustrating the physical alignment and connection between the end platform and the target balcony. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] like Figure 1 The high-altitude rescue automatic alignment and engagement system shown is mounted on a high-altitude rescue vehicle. The system includes a sensing module, a processing module, and an execution module, which achieve real-time data interaction and full closed-loop control through industrial Ethernet.

[0031] like Figure 1 In the high-altitude rescue automatic alignment and engagement system shown, the structure and deployment of the sensing module are as follows: The sensing module is fixedly installed at the end of the rescue vehicle boom 501 and rigidly connected to the end platform 502. Its core components include a high-definition camera, an infrared thermal imager, and a laser rangefinder. The high-definition camera and infrared thermal imager are arranged coaxially with parallel optical axes, capturing a field of view that completely covers the building facade 301 and all balcony areas 302, used for real-time synchronous acquisition of visible light and infrared thermal images of the high-rise building facade 301. The laser rangefinder's measurement direction is the same as the camera's optical axis, used to acquire the real-time distance value d between the end platform 502 and the target balcony 302.

[0032] Specifically, in this embodiment, the high-definition camera adopts a global shutter industrial camera, which has high resolution and high frame rate imaging performance, and can adapt to complex lighting environments such as strong light, backlight, and night, ensuring the clarity and real-time performance of building facade image acquisition; the infrared thermal imager adopts an uncooled infrared detector, which has wide range temperature measurement performance, and can effectively identify the thermal radiation characteristics of trapped personnel in harsh environments such as smoke obscuration and low visibility, ensuring the effectiveness of target detection in complex disaster scenarios; the laser rangefinder adopts a phase-type laser rangefinder sensor, which has a large range and high precision ranging performance, and can meet the full-process ranging requirements of long-distance coarse positioning and close-range fine alignment in rescue operations, providing stable and reliable real-time distance data for target spatial coordinate calculation.

[0033] As shown in Figure 1, in the automatic alignment and engagement system for high-altitude rescue, the processing module is installed inside the onboard control cabinet of the high-altitude rescue vehicle, integrating a deep learning inference engine and an industrial-grade microprocessor. The microprocessor possesses multi-task real-time processing capabilities, meeting the real-time control requirements of disaster rescue scenarios; the deep learning inference engine can stably perform real-time inference operations on the edge side using a dual-branch deep learning model and a personnel target detection model. The input end of the processing module is connected to the sensing module via an industrial Ethernet network, receiving visible light images, infrared thermal images, real-time ranging values ​​(d), and boom attitude data; the output end of the processing module is communicatively connected to the execution module, outputting the calculated motion control commands and attitude adjustment commands.

[0034] like Figure 1In the high-altitude rescue automatic alignment and engagement system shown, the execution module is communicatively connected to the rescue vehicle chassis and boom controller. Core components include a hydraulic proportional valve, a servo motor, an attitude adjustment mechanism, and a pressure sensor 602. The hydraulic proportional valve and servo motor drive the multi-joint movement of the rescue vehicle boom 501, enabling the end platform 502 to complete a large-range spatial displacement. The attitude adjustment mechanism is installed at the connection end between the rescue vehicle boom 501 and the end platform 502, used to independently adjust the pitch, roll, and yaw angles of the end platform 502 to achieve high-precision attitude alignment. The pressure sensor 602 is located on the front surface of the end platform's anti-collision beam 503, used to collect the contact pressure when the end platform 502 contacts the balcony railing 601 in real time, providing closed-loop feedback for flexible engagement.

[0035] Specifically, in this embodiment, the attitude adjustment mechanism has high-precision angle adjustment capability and wide-range attitude adaptability, which can accurately complete the pitch and tilt attitude adjustment of the end platform to meet the attitude control requirements of the entire alignment and joining process; the pressure sensor 602 has wide range and high-sensitivity pressure detection performance adapted to rescue operation scenarios, which can accurately collect the contact pressure data between the platform and the balcony in real time, effectively avoid the risk of rigid collision during the joining process, and ensure the safety of the joining operation.

[0036] like Figure 1 The high-altitude rescue automatic alignment and engagement system shown includes an execution module that also includes attitude sensors installed on each joint of the rescue vehicle boom 501. These sensors are used to collect attitude data such as rotation angle, pitch angle, and yaw angle of each joint in real time and feed them back to the processing module in real time, providing real-time data input for boom kinematic modeling and coordinate transformation.

[0037] The working principle of the automatic positioning and alignment joining mode is as follows:

[0038] When a high-rise building experiences a fire, earthquake, or other disaster, the aerial rescue vehicle arrives at the rescue site. The system activates an automatic rescue mode, using a high-definition camera and infrared thermal imager in the sensing module to collect visible light and infrared thermal images of the building facade 301 in real time. After processing by a dual-branch deep learning model in the processing module, all balconies 302 on the building facade are identified and a two-dimensional topology matrix is ​​constructed. Simultaneously, the trapped person 504 in the image is identified, and the target balcony 302 where the trapped person is located is located through a correlation contribution function. The processing module integrates multi-source sensor data to calculate the three-dimensional spatial coordinates of the target balcony, plans the movement path of the rescue vehicle boom 501, and drives the end platform 502 to move towards the target balcony. When the distance between the platform and the target balcony reaches a preset threshold, the system switches to a local fine recognition mode, using a PID-Fuzzy composite visual servo strategy to achieve parallel alignment between the platform and the balcony railing 601. Finally, a flexible connection is completed through pressure feedback, creating a stable and safe rescue channel for the trapped person 504.

[0039] In one embodiment, to achieve automatic positioning, precise alignment, and flexible connection between the high-altitude rescue platform and the target balcony, a method for automatic positioning and alignment of high-altitude rescue targets based on visual topological association is provided. This method relies on the aforementioned high-altitude rescue automatic positioning and alignment system, and the specific steps are as follows: Figure 2 As shown, it includes:

[0040] Step 1: In complex rescue environments such as fire, smoke, and insufficient lighting, a visual sensing unit integrating a high-definition camera and an infrared thermal imager is used to acquire visible light and infrared thermal images of the high-rise building facade 301 in real time. A dual-branch deep learning model is used for parallel processing. The dual-branch deep learning model includes a visible light feature extraction branch and an infrared feature extraction branch. The visible light feature extraction branch is used to process the visible light images acquired by the high-definition camera to extract the geometric contour and texture features of the building facade 301 and balcony 302. The infrared feature extraction branch is used to process the infrared thermal images acquired by the infrared thermal imager to extract the thermal radiation features of the target area. The feature maps output by the visible light feature extraction branch and the infrared feature extraction branch are cascaded and fused through a feature fusion module. Based on the fused feature map, target detection is performed to identify all balcony 302 targets in the image and generate balcony target bounding boxes. Extract the geometric distribution features of each balcony (302) to establish a two-dimensional topological matrix. The logic for constructing a topological matrix is ​​as follows: Figure 3 As shown;

[0041] Step 2: Use an object detection model to identify the trapped person 504 in the image and generate a bounding box for the trapped person. Constructing the correlation contribution function Calculate the degree of correlation between each trapped person target and each balcony target, automatically lock the balcony with the highest correlation as the target balcony, and determine the target balcony number n*. The calculation logic of the correlation contribution function is as follows: Figure 4 As shown;

[0042] Step 3: Integrate the intrinsic parameters K of the visual sensor, the real-time ranging value d collected by the laser rangefinder, and the real-time attitude data collected by the 501 attitude sensor of the rescue vehicle boom, and solve the real-time transformation matrix between the camera coordinate system and the rescue vehicle base coordinate system through boom kinematic modeling. The image pixel coordinates of the target balcony are converted into three-dimensional points in the camera coordinate system. Then convert to the three-dimensional coordinate system of the rescue vehicle base. The spatial coordinates of the target balcony are obtained, and the coordinate transformation process is as follows: Figure 5 As shown;

[0043] Step 4: Based on the converted 3D coordinates, drive the rescue vehicle boom 501 to move the end platform 502 towards the target balcony 302. When the distance between the platform and the target balcony 302 is less than a preset threshold, switch to local fine recognition mode to extract the edge linear features of the balcony railing 601. Use a PID-Fuzzy composite visual servo strategy to correct the motion vector in real time. Adjust the platform posture by controlling the hydraulic proportional valve to achieve parallel alignment between the end platform 502 and the balcony railing 601 plane, and finally complete the flexible joining. The alignment and joining process is as follows: Figure 6 As shown;

[0044] Step 5: After the engagement operation is completed, the system continuously monitors the attitude and contact pressure of the end platform 502 to maintain the stability of the platform until the rescue operation is completed.

[0045] The aforementioned automatic positioning and alignment method for high-altitude rescue targets based on visual topological association achieves stable identification of balconies and trapped personnel in complex environments through a dual-branch deep learning model, and accurately associates and positions trapped personnel with target balconies through an association contribution function, effectively avoiding false detection and missed detection problems in complex environments such as multiple layers and balconies, and smoke obstruction; it achieves accurate three-dimensional spatial positioning of target balconies through coordinate transformation, and achieves accurate alignment and flexible alignment of the end platform through a PID-Fuzzy composite visual servoing strategy, significantly improving the intelligence level and safety of high-altitude rescue operations.

[0046] The correlation contribution function The calculation formula is: ;

[0047] in, and These are the weighting coefficients, and In this embodiment, α=0.6 and β=0.4 are taken first, and the spatial overlap ratio between the target frame of the trapped person and the target frame of the balcony is given priority. The area of ​​overlap between the target frame of the trapped person and the target frame of the balcony; and These are the geometric center vectors of the target frames for the trapped personnel and the balcony, respectively. The spatial smoothing factor is set to σ=20 pixels in this embodiment to adapt to image inputs of different resolutions.

[0048] In the above formula, the first term represents the spatial proportion of trapped person 504 within the area of ​​balcony 302, and the second term represents the similarity of the center distance between trapped person 504 and balcony 302. The weighted sum of the two terms yields the final correlation value. This process iterates through all combinations of trapped persons and balconies, taking... The balcony index n* corresponding to the maximum value is used as the target balcony, thus completing the topological association and locking between the trapped person and the target balcony. Simultaneously, a preset threshold for the association degree is set; in this embodiment, the threshold is set to 0.5. When the calculated maximum value... If the value is less than the preset threshold, it is determined to be a positioning anomaly. The system immediately triggers the re-identification process and returns to step 1 to re-identify the balcony target and calculate the correlation.

[0049] The coordinate transformation logic in step 3 specifically includes the following sub-steps:

[0050] S3.1: Calculate the 3D point in the camera coordinate system based on the visual sensor intrinsic parameter matrix K and the real-time ranging value d. The calculation formula is: ;

[0051] Where K is the camera intrinsic parameter matrix, which is pre-calibrated using the Zhang Zhengyou calibration method and stored in the processing module. The intrinsic parameter matrix includes inherent parameters such as the camera's focal length, principal point coordinates, and distortion coefficients; (u,v) are the pixel coordinates of the target balcony 302 in the image. In this embodiment, the target bounding box of the target balcony is taken. The geometric center pixel coordinates are used as the target pixel coordinates;

[0052] S3.2: By modeling the boom's kinematics and combining it with the real-time attitude data of the rescue vehicle's boom 501, solve for the transformation matrix of the camera coordinate system relative to the rescue vehicle's base coordinate system. In this embodiment, the transformation matrix obtained by modeling and solving the forward kinematics of the boom includes the rotation matrix and translation vector of the camera coordinate system relative to the base coordinate system, which can represent the pose state of the camera in the global space in real time.

[0053] S3.3: Obtain the three-dimensional points of the target balcony in the coordinate system of the rescue vehicle base through matrix multiplication. The calculation formula is: The final solution yields the absolute three-dimensional coordinates of the target balcony, providing precise position input for the motion path planning and closed-loop control of the rescue vehicle boom 501.

[0054] In step 4, as the end platform 502 moves toward the target balcony 302, the sensing module updates the image data and ranging data in real time, and the processing module synchronously corrects the spatial coordinates of the target balcony and the movement path of the boom to ensure the positioning accuracy during the movement process. In this embodiment, the preset threshold for switching the local fine recognition mode is set to 0.5m. When the laser rangefinder detects that the distance between the end platform 502 and the target balcony 302 is less than or equal to 0.5m, the system automatically switches to the local fine recognition mode.

[0055] In the local fine recognition mode, an edge detection algorithm is used to extract the linear edge features of the balcony railing 601, and a straight line fitting algorithm is used to fit the straight line of the railing edge. The spatial angle between the plane where the balcony railing 601 is located and the front plane of the end platform 502 is calculated. A PID-Fuzzy composite visual servo strategy is used to calculate the motion correction vector of the end platform 502 in real time. The pitch and tilt angles of the end platform 502 are adjusted in real time by the attitude adjustment mechanism to make the angle between the two planes ≤0.5°, so as to achieve parallel alignment between the end platform 502 and the plane of the balcony railing 601. After the parallel alignment is completed, the system drives the end platform 502 to move towards the balcony railing 601 at a low and uniform speed of 5cm / s. The pressure sensor 602 deployed at the front end of the anti-collision beam 503 of the end platform detects the contact pressure between the platform and the balcony railing 601 in real time. When the contact pressure reaches a preset threshold (in this embodiment, the threshold is set to 50N), the system immediately stops the movement of the boom and the platform, completes the flexible engagement between the end platform 502 and the target balcony 302, and triggers a positioning prompt signal. After the connection is completed, the system continuously monitors the platform status through the pressure sensor 602 and the attitude sensor. If abnormal contact pressure or platform attitude deviation exceeds the threshold occurs, the attitude correction and position holding mechanism will be triggered immediately to resist external interference such as wind and vibration, and ensure continuous safety throughout the rescue process.

[0056] Compared with existing technologies, this invention provides an automatic positioning and alignment system and method for high-altitude rescue targets based on visual topological association. This system achieves fully automatic positioning, precise alignment, and flexible alignment of the high-altitude rescue platform and the target balcony throughout the entire process, significantly improving the intelligence level and safety of high-altitude rescue operations. The advantages and beneficial effects of this invention are as follows:

[0057] This invention employs a dual-branch deep learning model combining visible light and infrared, integrating geometric texture features and thermal radiation features. It can reliably identify balconies and trapped personnel in complex rescue environments such as smoke, insufficient light, and backlighting, effectively solving the problems of single-vision detection being susceptible to environmental interference and having a high rate of false positives and false negatives.

[0058] This invention constructs a correlation contribution function and combines it with a two-dimensional topological matrix of building facade balconies to achieve precise correlation and positioning between trapped personnel and target balconies. This effectively avoids target confusion in multi-story buildings with multiple balconies and significantly improves the accuracy of target positioning.

[0059] This invention integrates multi-source data from visual intrinsic parameters, laser ranging, and boom posture, and completes the precise conversion from two-dimensional pixel coordinates to three-dimensional spatial coordinates through boom kinematic modeling. This provides accurate position input for the motion control of the rescue boom and ensures positioning accuracy during long-distance movement.

[0060] This invention employs a two-level control strategy of coarse positioning and fine alignment. It switches to a local fine recognition mode at close range and achieves parallel alignment between the end platform and the balcony railing through a PID-Fuzzy composite visual servo strategy. Combined with pressure feedback, it achieves flexible joining, effectively avoiding the risk of rigid collisions. At the same time, it can resist external interference such as wind and vibration, ensuring the safety and stability of the joining operation.

[0061] The system of this invention adopts a modular design, with the sensing module, processing module, and execution module working independently yet collaboratively. It can be directly adapted to existing mainstream high-altitude rescue vehicles without requiring significant modifications to the original vehicle structure, and has strong practicality and scalability.

[0062] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0063] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for automatic localization and alignment of high-altitude rescue targets based on visual topological association, characterized in that, Includes the following steps: Step 1: In complex rescue environments, a visual sensing unit integrating a high-definition camera and an infrared thermal imager acquires visible light and infrared thermal images of the high-rise building facade in real time. A dual-branch deep learning model is used for parallel processing. This model includes a visible light feature extraction branch and an infrared feature extraction branch. The visible light feature extraction branch processes the visible light images acquired by the high-definition camera to extract the geometric contour and texture features of the building facade and balconies. The infrared feature extraction branch processes the infrared thermal images acquired by the infrared thermal imager to extract the thermal radiation features of the target area. A feature fusion module concatenates and fuses the feature maps output by the visible light and infrared feature extraction branches. Based on the fused feature maps, target detection is performed to identify all balcony targets in the image and generate balcony target bounding boxes. ; Extract the geometric distribution features of each balcony to establish a two-dimensional topological matrix. ; Step 2: Use an object detection model to identify trapped individuals in the image and generate bounding boxes for the trapped individuals. Constructing the correlation contribution function Calculate the degree of association between each trapped person target and each balcony target, automatically lock the balcony with the highest degree of association as the target balcony, and determine the target balcony number n*; Step 3: Integrate the intrinsic parameters K of the visual sensor, the real-time ranging value d collected by the laser rangefinder, and the real-time attitude data collected by the boom attitude sensor of the rescue vehicle, and solve the real-time transformation matrix between the camera coordinate system and the rescue vehicle base coordinate system through boom kinematic modeling. The image pixel coordinates of the target balcony are converted into three-dimensional points in the camera coordinate system. Then convert to the three-dimensional coordinate system of the rescue vehicle base. , thus obtaining the spatial coordinates of the target balcony; Step 4: Based on the converted 3D coordinates, drive the rescue vehicle boom to move the end platform toward the target balcony. When the distance between the platform and the target balcony is less than a preset threshold, switch to local fine recognition mode to extract the edge linear features of the balcony railing. Use a PID-Fuzzy composite visual servo strategy to correct the motion vector in real time. Adjust the platform posture by controlling the hydraulic proportional valve to achieve parallel alignment between the end platform and the balcony railing plane, and finally complete the flexible connection.

2. The method for automatic positioning and alignment of high-altitude rescue targets based on visual topological association according to claim 1, characterized in that, The correlation contribution function The calculation formula is: in, and These are the weighting coefficients, and ; The overlapping area between the target frame of the trapped person and the target frame of the balcony; and These are the geometric center vectors of the target frames for the trapped personnel and the balcony, respectively. Let be the spatial smoothing factor; take The balcony index n* corresponding to the maximum value is used as the target balcony. When the maximum value is less than the preset threshold, it is determined to be a positioning anomaly, triggering a re-identification process.

3. The method for automatic positioning and alignment of high-altitude rescue targets based on visual topological association according to claim 1, characterized in that, The coordinate transformation logic in step 3 specifically includes: S3.1: Calculate the 3D point in the camera coordinate system based on the visual sensor intrinsic parameter matrix K and the real-time ranging value d. The calculation formula is: Where K is the camera intrinsic parameter matrix, The pixel coordinates of the target balcony in the image; S3.2: By modeling the boom's kinematics and combining it with the real-time attitude data of the rescue vehicle's boom, solve for the transformation matrix of the camera coordinate system relative to the rescue vehicle's base coordinate system. ; S3.3: Obtain the three-dimensional points of the target balcony in the coordinate system of the rescue vehicle base through matrix multiplication. The calculation formula is:

4. The method for automatic positioning and alignment of high-altitude rescue targets based on visual topological association according to claim 1, characterized in that, In step 4, the local fine recognition mode uses an edge detection algorithm to extract the linear features of the balcony railing, fits the railing edge line through a straight line fitting transformation, and calculates the angle between the railing plane and the end platform plane. By adjusting the pitch and tilt angles of the end platform in real time, the angle between the two planes is made ≤0.5° to achieve parallel alignment. During the flexible joint process, the contact pressure between the platform and the balcony is detected in real time by a pressure sensor, and the movement stops when the contact pressure reaches a preset threshold.

5. A high-altitude rescue target automatic positioning and alignment system based on visual topological association, characterized in that, It includes a sensing module, a processing module, and an execution module. The sensing module is used to collect images of building facades, thermal imaging information of trapped personnel, and real-time ranging data. The processing module is used to execute a dual-branch deep learning recognition algorithm, a topological association algorithm, and coordinate transformation operations, receive real-time data from the sensing module, and output motion control commands. The execution module is used to receive motion path commands output by the processing module, drive the rescue vehicle boom and end-effector platform to move, achieve target alignment and flexible engagement, and simultaneously feed back real-time motion status data to the processing module to form a closed-loop control.

6. The automatic positioning and alignment system for high-altitude rescue targets based on visual topological association according to claim 5, characterized in that, The perception module includes a high-definition camera, an infrared thermal imager, and a laser rangefinder; the processing module integrates a deep learning inference engine and a microprocessor; the execution module is connected to the rescue vehicle chassis and boom controller, and includes a hydraulic proportional valve, a servo motor, and an attitude adjustment mechanism.

Images