Fire-fighting unmanned aerial vehicle quick release mechanism docking state multi-source fusion intelligent identification method

Abstract

The application relates to the field of fire-fighting unmanned plane state monitoring and intelligent identification, and provides a multi-source fusion intelligent identification method for a fire-fighting unmanned plane quick-release mechanism docking state.The method comprises the following steps: collecting position sensor data, force sensor data and visual image data of the quick-release mechanism of the fire-fighting unmanned plane; identifying the working state of the quick-release mechanism based on the collected position sensor data, force sensor data and visual image data, wherein the working state comprises a pre-docking state, a guiding positioning state, a locking positioning state, an unlocking positioning state, an abnormal jamming state and a loosening disengaging state; detecting defect data of the quick-release mechanism of the fire-fighting unmanned plane based on the collected visual image data, wherein the defect data comprises a locking surface wear defect and a hook deformation defect; and generating and outputting health state and graded alarm information of the quick-release mechanism.The application improves the accuracy of state identification under a complex fire-fighting operation environment and reduces the misjudgment rate.

Description

Technical Field

[0001] This application relates to the field of fire-fighting drone status monitoring and intelligent identification technology, and in particular to a multi-source fusion intelligent identification method for the docking status of a fire-fighting drone's quick-release mechanism. Background Technology

[0002] Firefighting drones often use quick-release mechanisms to rapidly change firefighting and reconnaissance payloads. Ball-and-socket type double-end docking quick-release mechanisms are widely used due to their high docking efficiency and compact structure. Current technologies for status identification of quick-release mechanisms mostly rely on single sensor data, which can only roughly determine whether the quick-release mechanism has completed docking, presenting the following clear technical problems: 1. It cannot distinguish the subdivided working states such as pre-connection, guiding and positioning, and locking in place in the entire process of quick-release mechanism docking, making it difficult to meet the status monitoring needs of the entire operation process; 2. A single data source is greatly affected by the fire-fighting environment (smoke interference, load fluctuations, sensor drift), resulting in low accuracy of status identification and a high false judgment rate; 3. Existing technologies do not combine the real-time working status identification of quick-release mechanisms with the detection of core structural defects, and cannot output the health status and graded alarm information of quick-release mechanisms, making it difficult to detect potential safety hazards of quick-release mechanisms in advance.

[0003] Therefore, a method is urgently needed to solve at least one of the above problems. Summary of the Invention

[0004] This application provides a multi-source fusion intelligent recognition method for the docking status of a quick-release mechanism for firefighting drones, aiming to solve the problem that there is no technical solution in the existing technology that can fuse multi-source data of position, force, and vision, and simultaneously realize the identification of subdivided working status, defect detection, and generation of graded health alarms for ball-and-socket quick-release mechanisms.

[0005] In a first aspect, embodiments of this application provide a multi-source fusion intelligent recognition method for the docking status of a quick-release mechanism for a fire-fighting drone. The quick-release mechanism for the fire-fighting drone is a ball-and-socket type double-end docking quick-release mechanism. The method includes: Collect position sensor data, force sensor data, and visual image data of the quick-release mechanism of the firefighting drone; Based on the collected position sensor data, force sensor data, and visual image data, the working state of the quick-release mechanism is identified. The working state includes pre-docking state, guided insertion state, locked in place state, unlocked in place state, abnormal jamming state, and loosening and disengaging state. Using deep learning algorithms, based on collected visual image data, defect data of the quick-release mechanism of the fire-fighting drone is detected. The defect data includes wear defects on the locking surface and deformation defects on the hook. Based on the identified working status and the detected defect data, the health status and graded alarm information of the quick-release mechanism are generated and output.

[0006] In some embodiments, the acquisition of position sensor data, force sensor data, and visual image data of the quick-release mechanism of the fire-fighting drone includes: synchronously acquiring position sensor data, force sensor data, and visual image data at a preset fixed sampling frequency; filtering and denoising the acquired position sensor data and force sensor data, and performing distortion correction and grayscale normalization on the acquired visual image data; and aligning the processed position sensor data, force sensor data, and visual image data with timestamps to generate aligned multi-source data.

[0007] In some embodiments, identifying the working state of the quick-release mechanism based on the collected position sensor data, force sensor data, and visual image data includes: inputting the aligned multi-source data into a preset multi-feature fusion state recognition model to extract position features, force features, and visual features; matching the extracted multi-dimensional features with preset feature threshold intervals corresponding to each working state; outputting the working state with the highest matching degree as the recognition result, while recording the duration of the corresponding working state.

[0008] In some embodiments, the method employs a deep learning algorithm to detect defect data of the quick-release mechanism of the fire-fighting drone based on the collected visual image data. The defect data includes locking surface wear defects and hook deformation defects. The method includes: inputting pre-processed visual image data into a pre-trained defect detection network model; extracting image features of the locking surface region and hook region in the visual image through the defect detection network model; identifying the defect type based on the extracted image features; calculating the size parameters of the defect; and outputting the corresponding defect data.

[0009] In some embodiments, generating and outputting health status and graded alarm information of the quick-release mechanism based on the identified working status and detected defect data includes: generating a working status health score based on the abnormal type and duration of the identified working status; generating a structural health score based on the detected defect type and size parameters; generating a comprehensive health status of the quick-release mechanism by combining the working status health score and the structural health score; and generating and outputting graded alarm information of the corresponding level according to the preset correspondence between health score range and alarm level.

[0010] In some embodiments, the method further includes: inputting the identified working status into the flight controller and the motion controller of the quick-release mechanism of the fire-fighting drone in real time; when a pre-docking state or a guided positioning state is identified, generating a docking posture adjustment command based on position sensor data and visual image data, sending it to the flight controller, and adjusting the docking posture of the fire-fighting drone; when an abnormal jamming state is identified, generating an unlocking and retraction command, sending it to the motion controller of the quick-release mechanism, and controlling the quick-release mechanism to perform an unlocking and retraction action.

[0011] In some embodiments, the method further includes: collecting multi-source data and corresponding manual annotation results during each docking and unlocking operation to construct an incremental learning dataset; when the number of samples in the incremental learning dataset reaches a preset sample threshold, starting incremental training of the defect detection network model and the multi-feature fusion state recognition model; after completing the incremental training, updating and replacing the currently used defect detection network model and multi-feature fusion state recognition model.

[0012] In some embodiments, the method further includes: continuously collecting working status data and defect data for each operation throughout the entire life cycle of the quick-release mechanism to construct a full life cycle operation dataset; fitting the changing trends of the wear degree of the locking surface and the deformation degree of the hook based on the full life cycle operation dataset; predicting the remaining reliable service life of the quick-release mechanism based on the fitted changing trends; and outputting an early maintenance prompt message when the predicted remaining reliable service life is lower than a preset service life threshold.

[0013] In some embodiments, the method further includes: uploading multi-source data collected by the device, identified working status data, and detected defect data to the fire operation command platform via a wireless communication link; receiving cluster operation defect statistics and abnormal status statistics of the same type of quick-release mechanism issued by the fire operation command platform; and updating the pre-set feature threshold range and alarm level correspondence of the device based on the cluster statistics to optimize the device's status identification and defect detection accuracy.

[0014] In some embodiments, the method further includes: before the fire-fighting drone performs fire-fighting operations, initiating a full-process self-inspection process of the quick-release mechanism, collecting multi-source data during the self-inspection process, identifying the working status during the self-inspection process, detecting defects in the mechanism, and generating a self-inspection report; when the self-inspection report shows that the quick-release mechanism has no abnormalities, granting the fire-fighting drone operating permissions; after each docking or unlocking operation is completed, encrypting and storing the full-process data, status identification results, defect detection results, health status, and alarm information of this operation to generate an operation traceability file.

[0015] This application addresses the operational characteristics of quick-release mechanisms in firefighting drones by integrating multi-source data (position, force, and vision) for identification. This overcomes the limitations of single-source data sources, which are susceptible to smoke interference, load fluctuations, and sensor drift, significantly improving the accuracy of status identification in complex firefighting environments and reducing the false alarm rate. This application enables precise identification of six subdivided working states throughout the entire operation of the quick-release mechanism. Compared to existing solutions that only determine whether docking is in place, this more refined state classification supports status monitoring and process control throughout the entire operation. Based on working state identification, and combined with two core defect data—locking surface wear and hook deformation—obtained through deep learning detection, this application integrates both types of information to output health status and tiered alarms. This ensures smooth docking operations and allows for early detection of potential safety hazards in the quick-release mechanism, effectively reducing the risk of attachment detachment and mechanism failure during firefighting operations, thus improving the safety and reliability of firefighting drone operations.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic flowchart illustrating the steps of a multi-source fusion intelligent identification method for the docking status of a quick-release mechanism of a fire-fighting drone, provided in one embodiment of this application. Figure 2 This is a structural schematic diagram of a quick-release mechanism for a fire-fighting drone provided in one embodiment of this application; Figure 3 This is a schematic block diagram of a multi-source fusion intelligent identification system for the docking status of a quick-release mechanism for a fire-fighting drone, provided in one embodiment of this application. Figure 4 This is a schematic block diagram of the structure of a computer device provided in an embodiment of this application.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0022] It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0023] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0024] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0025] Firefighting drones often use quick-release mechanisms to rapidly change firefighting and reconnaissance payloads. Ball-and-socket type double-end docking quick-release mechanisms are widely used due to their high docking efficiency and compact structure. Current technologies for status identification of quick-release mechanisms mostly rely on single sensor data, which can only roughly determine whether the quick-release mechanism has completed docking, presenting the following clear technical problems: 1. It cannot distinguish the subdivided working states such as pre-connection, guiding and positioning, and locking in place in the entire process of quick-release mechanism docking, making it difficult to meet the status monitoring needs of the entire operation process; 2. A single data source is greatly affected by the fire-fighting environment (smoke interference, load fluctuations, sensor drift), resulting in low accuracy of status identification and a high false judgment rate; 3. Existing technologies do not combine the real-time working status identification of quick-release mechanisms with the detection of core structural defects, and cannot output the health status and graded alarm information of quick-release mechanisms, making it difficult to detect potential safety hazards of quick-release mechanisms in advance.

[0026] Therefore, a method is urgently needed to solve at least one of the above problems.

[0027] To solve the above problem, please refer to Figure 1 This application provides a multi-source fusion intelligent identification method for the docking status of a fire-fighting drone's quick-release mechanism. The method is applied to computer equipment. This computer equipment can be deployed on a single server, a server cluster, a handheld terminal, a laptop, a wearable device, a fire-fighting drone's onboard controller, or a fire-fighting robot. All data acquisition, transmission, storage, and processing operations involved in this method are performed with the authorization of the relevant users and in compliance with relevant data security laws and regulations, ensuring user privacy and data security throughout the process.

[0028] like Figure 2 As shown, the quick-release mechanism for fire-fighting drones targeted by this method is a ball-and-socket type double-end docking quick-release mechanism. The ball-and-socket type double-end docking quick-release mechanism includes two sets of symmetrically arranged ball-and-socket docking components, two sets of electric push rod components, and an intermediate mounting connector. Each set of ball-and-socket docking components includes a ball, a ball flange, a limiting piece, an upper hook section, a lower hook section, and a ball fixing ring, which can realize the double-end synchronous docking and quick assembly / disassembly of the fire-fighting drone with the fire-fighting load and supply equipment.

[0029] The provided method for multi-source fusion intelligent identification of the docking status of the quick-release mechanism of a fire-fighting drone includes steps S101 to S103. Details are as follows: Step S101. Collect position sensor data, force sensor data, and visual image data of the quick-release mechanism of the fire-fighting drone.

[0030] Specifically, this step is the basic data acquisition stage of the method, used to obtain multi-source heterogeneous data during the docking and operation of the quick-release mechanism, providing data support for subsequent status identification and defect detection.

[0031] Position sensors are installed at the output ends of the two sets of electric push rods of the ball-and-socket double-end docking quick-release mechanism, force sensors are installed on the hook locking force surface and the ball-and-socket docking contact surface of the two sets of ball-and-socket docking components, and an image acquisition device is installed on the complete docking area of ​​the quick-release mechanism directly opposite the fuselage of the fire-fighting drone. The field of view of the image acquisition device completely covers the entire range of motion and key structural areas of the two sets of ball-and-socket docking components.

[0032] When the flight controller of the firefighting drone outputs docking, unlocking, or self-test commands, the data acquisition operation in this step is triggered simultaneously. When the quick-release mechanism is in standby mode, periodic data acquisition is performed according to the preset standby sampling interval to achieve all-time status monitoring.

[0033] Through a preset communication bus, real-time data on the electric push rod extension stroke, hook locking stroke, and relative position of the two docking ends of the quick-release mechanism are obtained from position sensors; real-time data on contact pressure during docking, locking force in the locked state, and running resistance during movement are obtained from force sensors; and real-time video streams of the entire docking process of the quick-release mechanism and high-definition static image data of key structural areas are obtained from image acquisition devices.

[0034] Step S102. Based on the collected position sensor data, force sensor data and visual image data, identify the working state of the quick-release mechanism. The working state includes pre-docking state, guided insertion state, locked in place state, unlocked in place state, abnormal jamming state and loosening and disengaging state.

[0035] Specifically, this step is the core state identification step of the method. Based on the fusion analysis of multi-source data, it realizes the accurate identification of the working status of the quick-release mechanism throughout the entire process, covering the normal and abnormal states of the entire process of docking, locking, and unlocking.

[0036] By pre-calibrating the six working states of the quick-release mechanism, the changes in positional data, force data, and the pose characteristics of the mechanism in the visual image corresponding to each working state are clarified. Specifically: Pre-docking state corresponds to the two ends of the docking entering the preset docking range without physical contact; Guided positioning state corresponds to the two ends of the docking making contact and the hook entering the locking stroke along the ball-and-socket guide structure; Locked in place state corresponds to the hook completing the locking stroke, the locking force reaching the preset rated range, and the mechanism being in a stable locked state; Unlocked in place state corresponds to the hook completing the unlocking stroke, returning to the initial unlocking position, and the two ends of the docking being separable; Abnormal jamming state corresponds to the abnormal state where the running resistance during the mechanism's movement exceeds the preset threshold and the stroke change stops; Loosening and disengaging state corresponds to the abnormal state where the locking force is lower than the preset safety threshold and the hook shows displacement during the locking state.

[0037] By synchronously correlating and analyzing the collected position sensor data, force sensor data, and visual image data, three types of feature data are extracted: the stroke change rate and absolute value of the position data; the pressure amplitude and change rate of the force data; and the relative pose of the mechanism and the motion position of the components in the visual image.

[0038] The extracted three types of feature data are matched with the features of six pre-labeled working states to determine the working state with the highest matching degree with the real-time features, and the corresponding state recognition result is output to realize the real-time state recognition of the entire workflow of the quick-release mechanism.

[0039] Step S103. Using a deep learning algorithm, based on the collected visual image data, detect the defect data of the quick-release mechanism of the fire-fighting drone. The defect data includes wear defects on the locking surface and deformation defects on the hook. Based on the identified working status and the detected defect data, generate and output the health status and graded alarm information of the quick-release mechanism.

[0040] Specifically, this step is the defect detection and health assessment stage of the method. It uses deep learning algorithms to accurately detect structural defects in the organization, and combines the status recognition results to complete the quantitative assessment of the organization's health status and output graded alarms.

[0041] By pre-building and training a deep learning algorithm model targeting structural defects of quick-release mechanisms, the collected visual image data is input into the trained deep learning algorithm model. The model extracts features from the locking surface area and hook area in the image, identifies locking surface wear defects and hook deformation defects within the area, calculates the size parameters of the defects, and outputs the corresponding defect data.

[0042] Based on the working status identification results output in step S102, the number of occurrences and duration of abnormal states are counted to assess the health status of the mechanism's operation. Based on the defect data output by defect detection, the integrity of the mechanism's structure is assessed. By combining the health status of the operation and the integrity of the structure, the overall health status of the quick-release mechanism is generated.

[0043] By pre-setting multiple health status levels and corresponding alarm triggering conditions, the system matches the corresponding health level and alarm triggering conditions based on the generated comprehensive health status, generates alarm information of the corresponding level, and outputs the health status and graded alarm information to the flight controller, airborne display terminal and fire operation command platform of the fire-fighting drone.

[0044] In some embodiments, the acquisition of position sensor data, force sensor data, and visual image data of the quick-release mechanism of the fire-fighting drone includes: synchronously acquiring position sensor data, force sensor data, and visual image data at a preset fixed sampling frequency; filtering and denoising the acquired position sensor data and force sensor data, and performing distortion correction and grayscale normalization on the acquired visual image data; and aligning the processed position sensor data, force sensor data, and visual image data with timestamps to generate aligned multi-source data.

[0045] This embodiment further optimizes the data acquisition operation in step S101. The acquisition of position sensor data, force sensor data, and visual image data from the quick-release mechanism of the fire-fighting drone is achieved by synchronously triggering data acquisition operations from the position sensor, force sensor, and image acquisition device according to a preset fixed sampling frequency. The sampling frequency can be preset according to the operating scenario of the fire-fighting drone; it is set to 50 times per second for normal operating scenarios and 100 times per second for high-speed docking operations, ensuring that the acquired multi-source data remain synchronized in the time dimension.

[0046] By using a moving average filtering algorithm to filter and denoise the collected position sensor data and force sensor data, environmental interference noise and hardware random noise during the data acquisition process are removed, resulting in smoothed and effective sensor data. For the collected visual image data, distortion correction is performed using camera calibration parameters to eliminate image errors caused by lens distortion of the image acquisition device. At the same time, grayscale normalization is performed on the corrected image to uniformly map the grayscale values ​​of the image to a preset fixed value range, eliminating the differences in image brightness under different lighting conditions and obtaining standardized image data.

[0047] By adding a timestamp corresponding to the acquisition time to each set of collected position sensor data, force sensor data, and visual image data, and matching and binding the position sensor data, force sensor data, and visual image data at the same acquisition time according to the timestamp value, the aligned multi-source data is generated. This ensures that the time dimension of the multi-source data is completely consistent during the subsequent fusion analysis process, avoiding recognition errors caused by time sequence misalignment.

[0048] In some embodiments, identifying the working state of the quick-release mechanism based on the collected position sensor data, force sensor data, and visual image data includes: inputting the aligned multi-source data into a preset multi-feature fusion state recognition model to extract position features, force features, and visual features; matching the extracted multi-dimensional features with preset feature threshold intervals corresponding to each working state; outputting the working state with the highest matching degree as the recognition result, while recording the duration of the corresponding working state.

[0049] This embodiment further optimizes the state recognition operation in step S102. Based on the collected position sensor data, force sensor data, and visual image data, the working state of the quick-release mechanism is identified by pre-constructing a multi-feature fusion state recognition model. This model includes a feature extraction branch, a feature fusion branch, and a classification output branch. The feature extraction branch sets up three parallel sub-networks to extract features from the position sensor data, force sensor data, and visual image data, respectively, and outputs the corresponding position features, force features, and visual features. The feature fusion branch concatenates and fuses the three sets of features to generate a fused multi-dimensional comprehensive feature. The classification output branch performs classification calculations on the fused comprehensive feature and outputs the matching probability for the six corresponding working states.

[0050] Multi-source data samples of the quick-release mechanism under six working states were collected in advance. After the samples were labeled, a training dataset was constructed. The multi-feature fusion state recognition model was supervised and trained using the training dataset until the model’s classification and recognition accuracy reached the preset accuracy threshold, thus completing the model’s pre-training and deployment.

[0051] The aligned multi-source data generated in step S101 is input into the pre-trained multi-feature fusion state recognition model. The model extracts the corresponding position features, force features, and visual features, and outputs the matching probabilities corresponding to the six working states. The matching probabilities corresponding to the six working states are matched with the preset feature threshold ranges corresponding to each working state. The working state with the highest matching probability and the value exceeding the preset probability threshold is selected as the final recognition result. At the same time, the duration of the working state from triggering to ending is recorded to provide data support for subsequent health assessment.

[0052] In some embodiments, the method employs a deep learning algorithm to detect defect data of the quick-release mechanism of the fire-fighting drone based on the collected visual image data. The defect data includes locking surface wear defects and hook deformation defects. The method includes: inputting pre-processed visual image data into a pre-trained defect detection network model; extracting image features of the locking surface region and hook region in the visual image through the defect detection network model; identifying the defect type based on the extracted image features; calculating the size parameters of the defect; and outputting the corresponding defect data.

[0053] This embodiment further optimizes the defect detection operation in step S103. A deep learning algorithm is used to detect defect data in the quick-release mechanism of the fire-fighting drone based on collected visual image data. The defect data includes locking surface wear defects and hook deformation defects. A defect detection network model is pre-constructed, comprising a backbone feature extraction network, a region candidate network, and a defect classification and regression network. The backbone feature extraction network performs multi-scale feature extraction on the input visual image to generate an image feature map. The region candidate network extracts candidate regions for the locking surface and hook regions based on the image feature map. The defect classification and regression network analyzes the features of the candidate regions, identifies the defect type, and calculates the defect size parameters.

[0054] Image samples of quick-release mechanisms containing different degrees of locking surface wear defects and hook deformation defects are collected in advance. The defect areas, defect types and defect sizes in the samples are labeled to construct a defect detection training dataset. The defect detection network model is supervised and trained using the training dataset until the defect recognition accuracy and size calculation accuracy of the model both reach the preset accuracy threshold, thus completing the pre-training and deployment of the model.

[0055] The preprocessed visual image data from step S101 is input into the pre-trained defect detection network model. Multi-scale image features of the visual image are extracted through the backbone feature extraction network of the model. The locking surface region and hook region in the image are located through the region candidate network, and the image features of the corresponding regions are extracted. Based on the extracted regional image features, the defect type in the region is identified, and the size parameters corresponding to the length, width, and depth of the defect are calculated. Finally, defect data containing defect type and size parameters are output.

[0056] In some embodiments, generating and outputting health status and graded alarm information of the quick-release mechanism based on the identified working status and detected defect data includes: generating a working status health score based on the abnormal type and duration of the identified working status; generating a structural health score based on the detected defect type and size parameters; generating a comprehensive health status of the quick-release mechanism by combining the working status health score and the structural health score; and generating and outputting graded alarm information of the corresponding level according to the preset correspondence between health score range and alarm level.

[0057] This embodiment further optimizes the health status assessment and graded alarm operation in step S103. Based on the identified working status and detected defect data, the health status and graded alarm information of the quick-release mechanism are generated and output. This is achieved by pre-setting the deduction weights for different abnormal working states and the deduction coefficients for the duration of abnormalities. Based on the working status identified in step S102, the number of occurrences, single duration, and cumulative duration of abnormal working states are statistically analyzed. According to the preset deduction weights and coefficients, the deduction value corresponding to the abnormal state is calculated. Using a maximum score of 100 points as a base, the corresponding deduction value is deducted to generate a working status health score.

[0058] By pre-setting the deduction weights for different defect types and the deduction coefficients for defect sizes, the defect data obtained in step S103 is used to statistically analyze the defect types, quantities, and size parameters. The deduction values ​​corresponding to the defects are calculated according to the preset deduction weights and coefficients. Based on a maximum score of 100, the corresponding deduction values ​​are deducted to generate a structural health score.

[0059] By pre-setting the weight ratios of the working condition health score and the structural health score, the working condition health score and the structural health score are weighted and summed according to their respective weights to generate a comprehensive health score in the range of 0 to 100. Based on the numerical range of the comprehensive health score, the corresponding health status level is divided to generate the comprehensive health status of the quick-disassembly mechanism.

[0060] Four alarm levels are pre-set: normal alert, Level 1 alarm, Level 2 alarm, and Level 3 alarm. A corresponding health score range is also set for each alarm level. The generated comprehensive health score is matched with the pre-set health score range to determine the corresponding alarm level and generate a corresponding tiered alarm message. The comprehensive health status and tiered alarm information are simultaneously output to the flight controller, onboard display terminal, ground handheld control terminal, and fire operation command platform of the fire-fighting drone, achieving synchronized alarm notifications across multiple terminals.

[0061] In some embodiments, the method further includes: inputting the identified working status into the flight controller and the motion controller of the quick-release mechanism of the fire-fighting drone in real time; when a pre-docking state or a guided positioning state is identified, generating a docking posture adjustment command based on position sensor data and visual image data, sending it to the flight controller, and adjusting the docking posture of the fire-fighting drone; when an abnormal jamming state is identified, generating an unlocking and retraction command, sending it to the motion controller of the quick-release mechanism, and controlling the quick-release mechanism to perform an unlocking and retraction action.

[0062] Based on the core steps, this embodiment adds closed-loop control and emergency handling operations for the docking process. By synchronously inputting the working status identified in step S102 into the flight controller and action controller of the quick-release mechanism of the fire-fighting drone through a real-time communication bus, the real-time linkage between the status identification result and the execution mechanism is realized.

[0063] When the pre-docking state or guided positioning state is detected, the relative position deviation data of the two docking ends in the position sensor data and the docking posture deviation data in the visual image data are extracted. Based on the preset posture adjustment control algorithm, the position and attitude adjustment amount of the fire-fighting drone is calculated, and the corresponding docking posture adjustment command is generated. The docking posture adjustment command is sent to the flight controller of the fire-fighting drone. The flight controller controls the drone's power system to adjust the spatial position and docking posture of the fire-fighting drone, eliminate docking deviation, and guide the quick-release mechanism to accurately complete the docking operation.

[0064] When an abnormal jamming state is detected, the current docking or unlocking action is immediately paused. Based on the preset emergency handling logic, an unlocking retraction command is generated. The unlocking retraction command is sent to the motion controller of the quick-release mechanism. The motion controller controls the electric push rod to perform a reverse retraction action, driving the hook back to the initial unlocking position, eliminating jamming stress, and preventing structural damage to the mechanism due to forced movement. At the same time, a jamming abnormality alarm message is generated and output to the relevant control terminal.

[0065] In some embodiments, the method further includes: collecting multi-source data and corresponding manual annotation results during each docking and unlocking operation to construct an incremental learning dataset; when the number of samples in the incremental learning dataset reaches a preset sample threshold, starting incremental training of the defect detection network model and the multi-feature fusion state recognition model; after completing the incremental training, updating and replacing the currently used defect detection network model and multi-feature fusion state recognition model.

[0066] This embodiment adds incremental learning and online optimization operations to the model based on the core steps. After each docking or unlocking operation is completed, multi-source data, corresponding status recognition results, and defect detection results are collected during the operation. At the same time, manual annotation and correction results of the recognition and detection results by the operators are received. The multi-source data and the corresponding manual annotation results are bound and stored to construct an incremental learning dataset.

[0067] By counting the number of valid samples in the incremental learning dataset in real time, the incremental training process is automatically triggered when the number of valid samples reaches the preset sample threshold. The preset sample threshold can be set according to the job scenario. It is set to 500 valid samples in normal scenarios and 200 valid samples in high-frequency job scenarios.

[0068] By employing an incremental learning dataset, the currently used defect detection network model and multi-feature fusion state recognition model are incrementally trained. During training, the model's learned feature extraction capabilities are retained, and the parameters of the model's classification and regression layers are optimized only based on new samples to avoid catastrophic forgetting. After completing the incremental training, the accuracy of the trained model is verified. When the model's accuracy reaches the preset accuracy requirements, the currently used defect detection network model and multi-feature fusion state recognition model are updated and replaced, realizing online self-optimization of the model and continuously improving the model's recognition accuracy and adaptability in different work scenarios.

[0069] In some embodiments, the method further includes: continuously collecting working status data and defect data for each operation throughout the entire life cycle of the quick-release mechanism to construct a full life cycle operation dataset; fitting the changing trends of the wear degree of the locking surface and the deformation degree of the hook based on the full life cycle operation dataset; predicting the remaining reliable service life of the quick-release mechanism based on the fitted changing trends; and outputting an early maintenance prompt message when the predicted remaining reliable service life is lower than a preset service life threshold.

[0070] Based on the core steps, this embodiment adds full life cycle management and remaining service life prediction operations for the mechanism. Starting from the first use of the quick-release mechanism, it continuously collects working status data, defect detection data, number of operations, and cumulative running time during each operation. All data are classified and stored to construct a full life cycle operation dataset for the quick-release mechanism.

[0071] Based on the full lifecycle operation dataset, data on the wear degree of the locking surface and the deformation degree of the hook corresponding to different operation time nodes are extracted. A multinomial fitting algorithm is used to fit the trend curve of the wear degree of the locking surface with the number of operations and the running time, and at the same time, the trend curve of the deformation degree of the hook with the number of operations and the running time is fitted.

[0072] Pre-set safety thresholds for locking surface wear and hook deformation in the quick-release mechanism. When the defect level reaches the safety threshold, the mechanism reaches its service life limit. Based on the fitted trend curves, predict the number of operations and runtime required for the locking surface wear and hook deformation to reach the corresponding safety thresholds. Combined with the number of operations completed and the cumulative runtime, calculate the remaining reliable service life of the quick-release mechanism.

[0073] A lifespan warning threshold is preset. When the predicted remaining reliable lifespan is lower than the preset lifespan warning threshold, an early maintenance reminder is generated. The remaining reliable lifespan prediction result and the early maintenance reminder are output to the fire equipment management platform and the maintenance personnel terminal to guide the maintenance personnel to maintain or replace the quick-release mechanism in advance and avoid safety accidents caused by mechanism failure during operation.

[0074] In some embodiments, the method further includes: uploading multi-source data collected by the device, identified working status data, and detected defect data to the fire operation command platform via a wireless communication link; receiving cluster operation defect statistics and abnormal status statistics of the same type of quick-release mechanism issued by the fire operation command platform; and updating the pre-set feature threshold range and alarm level correspondence of the device based on the cluster statistics to optimize the device's status identification and defect detection accuracy.

[0075] Based on the core steps, this embodiment adds cluster data linkage and model parameter optimization operations. By uploading multi-source data collected locally, identified working status data, detected defect data, and health status data to the fire operation command platform in real time or periodically through an encrypted wireless communication link, the platform summarizes and stores the operation data of all connected quick-release mechanisms of the same model to build a cluster operation database.

[0076] The system receives cluster operation defect statistics and abnormal status statistics of the same type of quick-release mechanism from the fire operation command platform. The cluster operation defect statistics include common defect types, defect occurrence patterns, and corresponding characteristic thresholds of the same type of quick-release mechanism. The abnormal status statistics include common abnormal status types, abnormal occurrence patterns, and corresponding characteristic threshold ranges of the same type of quick-release mechanism.

[0077] Based on the received cluster statistics, the preset working status feature threshold range, defect identification feature threshold, and alarm level correspondence of the local machine are updated and optimized. Based on the optimized parameters, the inference parameters of the local machine's multi-feature fusion status identification model and defect detection network model are adjusted to optimize the local machine's status identification and defect detection accuracy, so that the local machine's identification and detection capabilities are adapted to the overall operating rules of the same type of organization, and the identification error of individual equipment is reduced.

[0078] In some embodiments, the method further includes: before the fire-fighting drone performs fire-fighting operations, initiating a full-process self-inspection process of the quick-release mechanism, collecting multi-source data during the self-inspection process, identifying the working status during the self-inspection process, detecting defects in the mechanism, and generating a self-inspection report; when the self-inspection report shows that the quick-release mechanism has no abnormalities, granting the fire-fighting drone operating permissions; after each docking or unlocking operation is completed, encrypting and storing the full-process data, status identification results, defect detection results, health status, and alarm information of this operation to generate an operation traceability file.

[0079] This embodiment, based on the core steps, adds pre-operation self-inspection control and post-operation traceability and archiving operations. Before the fire-fighting drone performs fire-fighting operations, it receives a self-inspection command issued by the operator and initiates the full-stroke self-inspection process of the quick-release mechanism. It controls the quick-release mechanism to perform the complete locking and unlocking strokes. During the self-inspection process, it collects multi-source data of the entire self-inspection process according to step S101, identifies the working status of the self-inspection process according to step S102, and detects structural defects of the mechanism according to step S103. Finally, it generates a self-inspection report that includes self-inspection process data, status identification results, defect detection results, and health status.

[0080] The generated self-inspection report is compared with the preset operation access conditions. When the self-inspection report shows that the quick-release mechanism has no structural defects, no abnormal conditions, and its health status meets the operation requirements, the fire-fighting drone's fire-fighting operation permission is granted, allowing the drone to perform subsequent fire-fighting operations. When the self-inspection report shows that the quick-release mechanism has abnormalities or defects, the fire-fighting drone's operation permission is locked, and corresponding alarm information is output, prohibiting the drone from performing fire-fighting operations, thus avoiding the safety risks of operating with faults from the source.

[0081] After each docking or unlocking operation is completed, the multi-source data, status identification results, defect detection results, health status data, and alarm information of the entire operation process are encrypted and stored using a preset encryption algorithm to generate an unalterable operation traceability file. The operation traceability file is then synchronously stored on the airborne storage device and the storage server of the fire operation command platform to provide complete and traceable data support for subsequent operation review, accident tracing, and equipment maintenance.

[0082] In some embodiments, this embodiment addresses the technical problems of interrupted status recognition and defect detection, and sharp drop in accuracy caused by the failure or loss of data from single or multiple types of sensors in extremely harsh environments such as dense smoke, high temperature, water mist, and strong electromagnetic interference at fire-fighting operation sites. It achieves multi-source data redundancy completion and robust recognition under extreme environments. Real-time sensor health and data validity assessment is performed by simultaneously evaluating the operating status and output data validity of position sensors, force sensors, and image acquisition devices during the multi-source data acquisition process in step S101. For position sensors and force sensors, the output amplitude, fluctuation frequency, and signal-to-noise ratio of the data are detected in real time. When the data continuously exceeds a preset reasonable range, the signal-to-noise ratio is lower than a preset threshold, or continuous fixed values ​​are output, the corresponding sensor data is determined to be faulty. For visual image data, the image clarity, number of effective feature points, and contrast are calculated in real time. When the image clarity is lower than a preset threshold, the number of effective feature points is less than a preset number, or the contrast exceeds a reasonable range, the corresponding visual image data is determined to be faulty.

[0083] When a single type of sensor data is determined to be faulty, the corresponding level of redundancy completion strategy is activated. If visual image data is determined to be faulty, a time-series prediction model is constructed based on the mapping relationship between valid position sensor data, force sensor data and corresponding working states within a historical time period. The time-series prediction model generates completed visual feature data based on real-time acquired valid position and force sensor data, replacing the faulty visual image data in subsequent state recognition. If a single type of position sensor or force sensor is determined to be faulty, feature data of the faulty sensor is generated based on the mapping relationship between historical data and corresponding features of the remaining two types of valid sensors, and participates in subsequent state recognition. If two types of sensor data are determined to be faulty, completed feature data of the other two types of sensors is generated based on the real-time data of the remaining valid sensor, combined with the pre-stored mechanism kinematics model and historical operating data, ensuring that the state recognition process is not interrupted.

[0084] When sensor data failure and completion operations occur, the system automatically switches to the extreme environment robust recognition mode, adjusts the feature weights of the multi-feature fusion state recognition model, increases the weight ratio of features corresponding to valid sensors, and reduces the weight ratio of completed features. Simultaneously, for defect detection operations, when visual image data fails, the system indirectly identifies changes in force characteristics caused by wear on the locking surface and deformation of the hook based on the changing patterns of force sensor data, replacing visual inspection to achieve indirect defect identification, ensuring the continuity and reliability of mechanism state recognition and defect detection in extreme and harsh environments.

[0085] In some embodiments, to address the risks of mechanism collisions and structural damage caused by load inertia, environmental wind disturbances, and attitude deviations during the docking process of a fire-fighting drone, dynamic safety boundary control and proactive collision prevention intervention are implemented. The dynamic safety boundary construction process for the docking scenario is initiated when the pre-docking state is identified in step S102. Based on the three-dimensional structural dimensions of the quick-release mechanism, the relative positions of the docking ends, the flight speed and attitude angle of the fire-fighting drone, and the wind speed and direction data of the surrounding environment, a three-dimensional dynamic safety boundary for the docking process is constructed. This three-dimensional dynamic safety boundary is divided into three levels: a safe operating area, a warning buffer zone, and a dangerous collision zone. The safe operating area is the normal operating range of the docking process, the warning buffer zone is the warning range indicating an approaching collision risk, and the dangerous collision zone is the prohibited entry area where a collision is imminent.

[0086] Throughout the docking process, based on the position sensor data and visual image data collected in step S101, the relative pose, relative motion speed, and relative motion acceleration of the docking ends are calculated in real time. The distance between the current mechanism pose and the dynamic safety boundary is calculated simultaneously to obtain the real-time safety margin. At the same time, combined with the payload weight and flight inertia of the fire-fighting drone, the motion trajectory of the mechanism within a preset time period is predicted, and it is predicted whether the trajectory will enter the warning buffer zone or the dangerous collision zone.

[0087] According to the real-time safety margin and the trajectory prediction result, hierarchical active intervention is performed. When it is predicted that the pose of the mechanism will enter the warning buffer zone, a deceleration adjustment instruction is generated and sent to the flight controller of the fire drone to reduce the docking approach speed of the drone. At the same time, a pose warning prompt is output. When it is predicted that the pose of the mechanism will enter the dangerous collision area, or the real-time safety margin is lower than the preset safety threshold, an emergency braking and reverse avoidance instruction is immediately generated and sent to the flight controller to control the drone to stop the docking approach action and perform a reverse avoidance operation. At the same time, the locking action of the quick-release mechanism is suspended to avoid rigid collision damage of the mechanism. When the pose of the mechanism returns to the safe operation area, the intervention is解除, and the normal docking process is restored.

[0088] In some embodiments, for the actual combat scenario of multi-drone collaborative operation at the fire rescue site, the state linkage control and task dynamic scheduling of the multi-drone quick-release mechanism are realized to improve the overall efficiency and safety of multi-drone collaborative operation. The multi-drone state cluster aggregation and unified control receive the state data, defect data, health status data, operation times, and remaining service life data of the quick-release mechanisms uploaded by all fire drones in the operation area in real time through the fire operation command platform, construct a multi-drone quick-release mechanism cluster state database, and classify and grade the health status of the quick-release mechanisms of all drones into three levels: core operation level, regular operation level, and standby maintenance level. Among them, the core operation level is the mechanism with excellent health status, no defects, and no abnormal records; the regular operation level is the mechanism with good health status, minor defects that do not affect safety, and no abnormal status records; the standby maintenance level is the mechanism with poor health status, obvious defects, or abnormal status records.

[0089] When the fire operation command platform receives the on-site fire extinguishing operation task, the task is disassembled into multiple subtasks, including the fire extinguishing load docking task, the fire extinguishing agent replenishment docking task, the rescue equipment delivery docking task, and the on-site inspection task. At the same time, the docking frequency, load weight, and operation risk level corresponding to each subtask are clarified.

[0090] Based on the requirements of the disassembled sub-tasks and the health status levels of the multi-drone quick-disassembly mechanisms, dynamic task scheduling is performed. High-frequency, high-load, and high-risk core docking sub-tasks are assigned to core operation-level drones; low-frequency, medium-load, and low-risk routine docking sub-tasks are assigned to routine operation-level drones; standby maintenance-level drones are assigned on-site inspection tasks without docking requirements, and maintenance prompts are output, with maintenance scheduled during work breaks; when a drone's quick-disassembly mechanism malfunctions or its health status deteriorates during task execution, its current task is immediately transferred to a backup drone of the same level, and the malfunctioning drone is moved to the standby maintenance level and recalled to the ground base station to avoid operating with faults, thus achieving full-process status linkage control and dynamic task optimization for multi-drone collaborative operations.

[0091] In some embodiments, to address the safety risk of sudden failures during the operation of the quick-release mechanism, early latent fault prediction and proactive maintenance intervention based on multi-source time-series data correlation mining are implemented. This allows for early identification of faults in their initial stages, preventing sudden failures during operation. The construction of the full-cycle time-series feature database involves continuously collecting multi-source data output in step S101, working status time-series data output in step S102, and defect detection time-series data output in step S103 during each operation cycle, starting from the first use of the quick-release mechanism. This extracts the slope of position data changes, force data fluctuation characteristics, docking stroke duration, locking force rise rate, unlocking force change pattern, and minute changes in the mechanism's fit clearance in visual images within each operation cycle. All feature data are stored in chronological order of operation time to construct the full-cycle time-series feature database for the quick-release mechanism.

[0092] Pre-collect time-series feature samples of the same model of quick-release mechanism from brand new to the appearance of obvious faults throughout its entire life cycle. Label the early feature change patterns corresponding to latent faults in the samples, including the continuous decrease in the rate of increase of locking force, the continuous increase in the docking stroke time, the slow increase in the mating gap, and the slight continuous increase in motion resistance. Based on the labeled samples, construct and train a time-series feature association mining model. The model can identify the small offsets and trend changes related to latent faults in the time-series features.

[0093] The time-series features extracted from the current work cycle are compared with historical features in the full-cycle time-series feature database. At the same time, the trained time-series feature association mining model is input. The model analyzes the changing trends and deviations of the features to identify whether there are early latent faults, including early wear of the locking surface, metal fatigue and micro-deformation of the hook, increased clearance of the transmission structure, and performance degradation of the electric push rod, which are latent faults that have not yet reached the threshold of explicit defects. The model also outputs the type, degree of development, and development trend of latent faults.

[0094] Based on the identified types and development levels of latent faults, corresponding proactive maintenance intervention plans are generated. For minor latent faults, the self-inspection frequency of the mechanism is increased, and targeted special inspections are added before each operation. For moderate latent faults, the docking load weight and docking frequency of the mechanism are limited to reduce operational risks. For severe latent faults, high-risk operation permissions of the mechanism are locked, emergency maintenance prompts are output, and maintenance personnel are guided to immediately disassemble, inspect, and maintain the mechanism. Intervention is completed before the fault develops into an explicit defect and causes sudden failure of the mechanism, ensuring operational safety.

[0095] In some embodiments, to address the issues of varying anti-interference capabilities of different sensors, real-time changes in data confidence levels, and decreased recognition accuracy due to fixed-weight fusion in complex operating environments, heterogeneous multi-sensor confidence assessment and adaptive dynamic adjustment of fusion weights are implemented to improve the anti-interference capability and accuracy of multi-source fusion recognition. The real-time confidence quantification assessment of heterogeneous multi-sensors involves constructing corresponding confidence quantification assessment systems for three types of heterogeneous sensors—position sensors, force sensors, and visual image acquisition devices—during the multi-source data acquisition process in step S101, and calculating the output data confidence level of each type of sensor in real time. For position sensors, the confidence level is calculated based on the linearity, repeatability, and deviation from the theoretical value output by the mechanism's kinematic model, with a value range of 0 to 1. For force sensors, the confidence level is calculated based on the zero-point drift, fluctuation stability, and matching degree with the preset force law, with a value range of 0 to 1. For visual image acquisition devices, the confidence level is calculated based on image clarity, number of effective feature points, ambient light intensity, and smoke obstruction level, with a value range of 0 to 1.

[0096] In the multi-feature fusion state recognition model, the initial weight proportions of position features, force features, and visual features are pre-set, and upper and lower limits for weight adjustment are set to avoid recognition bias caused by excessively high or low weights for a single type of feature. Based on the real-time calculated confidence values ​​of the three types of sensors, the fusion weights of corresponding features are adaptively adjusted. When the confidence of a certain type of sensor increases, the fusion weight of its features is increased; when the confidence of a certain type of sensor decreases, its fusion weight is decreased. During the weight adjustment process, the total weight of the three types of features remains a fixed value, and the weight of each type of feature does not exceed the preset upper and lower limits.

[0097] Model Adaptive Inference and Accuracy Optimization: The adjusted feature weights are updated in real time to the multi-feature fusion state recognition model and the defect detection network model. The model performs feature fusion and inference calculations based on the updated weights. Simultaneously, based on the deviation between the manually labeled results and the recognition results after each operation, the confidence assessment system and weight adjustment logic are iteratively optimized. This allows the weight adjustment strategy to adapt to different operating environments and sensor states, continuously improving the anti-interference capability and recognition accuracy of multi-source fusion recognition. Even when some sensors are affected by environmental interference or decreased confidence, the accuracy of state recognition and defect detection can still be guaranteed.

[0098] Please see Figure 3 As shown, Figure 3 This is a schematic diagram of the structure of the multi-source fusion intelligent identification system 200 for the docking status of a fire-fighting drone's quick-disassembly mechanism, provided in this embodiment. This system 200 is used to execute the steps of the multi-source fusion intelligent identification method for the docking status of a fire-fighting drone's quick-disassembly mechanism as shown in the above embodiments. The system 200 can be a single server or a server cluster, or it can be a terminal, such as a handheld terminal, a laptop computer, a wearable device, or a robot.

[0099] like Figure 3 As shown, the multi-source fusion intelligent identification system 200 for the docking status of the quick-disassembly mechanism of the fire-fighting drone includes: Data acquisition unit 201 is used to acquire position sensor data, force sensor data and visual image data of the quick-release mechanism of the fire-fighting drone; The status recognition unit 202 is used to identify the working status of the quick-release mechanism based on the collected position sensor data, force sensor data and visual image data. The working status includes pre-docking status, guided insertion status, locked in place status, unlocked in place status, abnormal jamming status and loosening and disengaging status. The defect detection unit 203 is used to detect defect data of the quick-release mechanism of the fire-fighting drone based on the collected visual image data using a deep learning algorithm. The defect data includes wear defects on the locking surface and deformation defects on the hook. Based on the identified working status and the detected defect data, the unit generates and outputs the health status and graded alarm information of the quick-release mechanism.

[0100] In some embodiments, the acquisition of position sensor data, force sensor data, and visual image data of the quick-release mechanism of the fire-fighting drone includes: synchronously acquiring position sensor data, force sensor data, and visual image data at a preset fixed sampling frequency; filtering and denoising the acquired position sensor data and force sensor data, and performing distortion correction and grayscale normalization on the acquired visual image data; and aligning the processed position sensor data, force sensor data, and visual image data with timestamps to generate aligned multi-source data.

[0101] In some embodiments, identifying the working state of the quick-release mechanism based on the collected position sensor data, force sensor data, and visual image data includes: inputting the aligned multi-source data into a preset multi-feature fusion state recognition model to extract position features, force features, and visual features; matching the extracted multi-dimensional features with preset feature threshold intervals corresponding to each working state; outputting the working state with the highest matching degree as the recognition result, while recording the duration of the corresponding working state.

[0102] In some embodiments, the method employs a deep learning algorithm to detect defect data of the quick-release mechanism of the fire-fighting drone based on the collected visual image data. The defect data includes locking surface wear defects and hook deformation defects. The method includes: inputting pre-processed visual image data into a pre-trained defect detection network model; extracting image features of the locking surface region and hook region in the visual image through the defect detection network model; identifying the defect type based on the extracted image features; calculating the size parameters of the defect; and outputting the corresponding defect data.

[0103] In some embodiments, generating and outputting health status and graded alarm information of the quick-release mechanism based on the identified working status and detected defect data includes: generating a working status health score based on the abnormal type and duration of the identified working status; generating a structural health score based on the detected defect type and size parameters; generating a comprehensive health status of the quick-release mechanism by combining the working status health score and the structural health score; and generating and outputting graded alarm information of the corresponding level according to the preset correspondence between health score range and alarm level.

[0104] In some embodiments, the method further includes: inputting the identified working status into the flight controller and the motion controller of the quick-release mechanism of the fire-fighting drone in real time; when a pre-docking state or a guided positioning state is identified, generating a docking posture adjustment command based on position sensor data and visual image data, sending it to the flight controller, and adjusting the docking posture of the fire-fighting drone; when an abnormal jamming state is identified, generating an unlocking and retraction command, sending it to the motion controller of the quick-release mechanism, and controlling the quick-release mechanism to perform an unlocking and retraction action.

[0105] In some embodiments, the method further includes: collecting multi-source data and corresponding manual annotation results during each docking and unlocking operation to construct an incremental learning dataset; when the number of samples in the incremental learning dataset reaches a preset sample threshold, starting incremental training of the defect detection network model and the multi-feature fusion state recognition model; after completing the incremental training, updating and replacing the currently used defect detection network model and multi-feature fusion state recognition model.

[0106] In some embodiments, the method further includes: continuously collecting working status data and defect data for each operation throughout the entire life cycle of the quick-release mechanism to construct a full life cycle operation dataset; fitting the changing trends of the wear degree of the locking surface and the deformation degree of the hook based on the full life cycle operation dataset; predicting the remaining reliable service life of the quick-release mechanism based on the fitted changing trends; and outputting an early maintenance prompt message when the predicted remaining reliable service life is lower than a preset service life threshold.

[0107] In some embodiments, the method further includes: uploading multi-source data collected by the device, identified working status data, and detected defect data to the fire operation command platform via a wireless communication link; receiving cluster operation defect statistics and abnormal status statistics of the same type of quick-release mechanism issued by the fire operation command platform; and updating the pre-set feature threshold range and alarm level correspondence of the device based on the cluster statistics to optimize the device's status identification and defect detection accuracy.

[0108] In some embodiments, the method further includes: before the fire-fighting drone performs fire-fighting operations, initiating a full-process self-inspection process of the quick-release mechanism, collecting multi-source data during the self-inspection process, identifying the working status during the self-inspection process, detecting defects in the mechanism, and generating a self-inspection report; when the self-inspection report shows that the quick-release mechanism has no abnormalities, granting the fire-fighting drone operating permissions; after each docking or unlocking operation is completed, encrypting and storing the full-process data, status identification results, defect detection results, health status, and alarm information of this operation to generate an operation traceability file.

[0109] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the multi-source fusion intelligent identification system for the docking status of the fire-fighting drone quick-disassembly mechanism and each module described above can be referred to the corresponding content in the various embodiments of the multi-source fusion intelligent identification method for the docking status of the fire-fighting drone quick-disassembly mechanism, and will not be repeated here.

[0110] The aforementioned multi-source fusion intelligent recognition method for the docking status of the quick-disassembly mechanism of firefighting drones can be implemented as a computer program, which can, for example... Figure 3 It runs on the device shown.

[0111] Please see Figure 4 , Figure 4 This is a schematic block diagram of the structure of a computer device provided in an embodiment of this application. The computer device includes a processor, a memory, and a network interface connected via a device bus, wherein the memory may include a storage medium and internal memory.

[0112] The storage medium can store operating devices and computer programs. The computer program includes program instructions that, when executed, cause the processor to perform any multi-source fusion intelligent recognition method for the docking status of a fire-fighting drone's quick-release mechanism.

[0113] The processor provides computing and control capabilities, supporting the operation of the entire computer device.

[0114] The internal memory provides an environment for the execution of computer programs in non-volatile storage media. When the computer program is executed by the processor, it enables the processor to execute any multi-source fusion intelligent recognition method for the docking status of the quick-release mechanism of a fire-fighting drone.

[0115] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the terminal to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0116] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0117] In one embodiment, the processor is configured to run a computer program stored in memory to perform the following steps: Collect position sensor data, force sensor data, and visual image data of the quick-release mechanism of the firefighting drone; Based on the collected position sensor data, force sensor data, and visual image data, the working state of the quick-release mechanism is identified. The working state includes pre-docking state, guided insertion state, locked in place state, unlocked in place state, abnormal jamming state, and loosening and disengaging state. Using deep learning algorithms, based on collected visual image data, defect data of the quick-release mechanism of the fire-fighting drone is detected. The defect data includes wear defects on the locking surface and deformation defects on the hook. Based on the identified working status and the detected defect data, the health status and graded alarm information of the quick-release mechanism are generated and output.

[0118] In some embodiments, the acquisition of position sensor data, force sensor data, and visual image data of the quick-release mechanism of the fire-fighting drone includes: synchronously acquiring position sensor data, force sensor data, and visual image data at a preset fixed sampling frequency; filtering and denoising the acquired position sensor data and force sensor data, and performing distortion correction and grayscale normalization on the acquired visual image data; and aligning the processed position sensor data, force sensor data, and visual image data with timestamps to generate aligned multi-source data.

[0119] In some embodiments, identifying the working state of the quick-release mechanism based on the collected position sensor data, force sensor data, and visual image data includes: inputting the aligned multi-source data into a preset multi-feature fusion state recognition model to extract position features, force features, and visual features; matching the extracted multi-dimensional features with preset feature threshold intervals corresponding to each working state; outputting the working state with the highest matching degree as the recognition result, while recording the duration of the corresponding working state.

[0120] In some embodiments, the method employs a deep learning algorithm to detect defect data of the quick-release mechanism of the fire-fighting drone based on the collected visual image data. The defect data includes locking surface wear defects and hook deformation defects. The method includes: inputting pre-processed visual image data into a pre-trained defect detection network model; extracting image features of the locking surface region and hook region in the visual image through the defect detection network model; identifying the defect type based on the extracted image features; calculating the size parameters of the defect; and outputting the corresponding defect data.

[0121] In some embodiments, generating and outputting health status and graded alarm information of the quick-release mechanism based on the identified working status and detected defect data includes: generating a working status health score based on the abnormal type and duration of the identified working status; generating a structural health score based on the detected defect type and size parameters; generating a comprehensive health status of the quick-release mechanism by combining the working status health score and the structural health score; and generating and outputting graded alarm information of the corresponding level according to the preset correspondence between health score range and alarm level.

[0122] In some embodiments, the method further includes: inputting the identified working status into the flight controller and the motion controller of the quick-release mechanism of the fire-fighting drone in real time; when a pre-docking state or a guided positioning state is identified, generating a docking posture adjustment command based on position sensor data and visual image data, sending it to the flight controller, and adjusting the docking posture of the fire-fighting drone; when an abnormal jamming state is identified, generating an unlocking and retraction command, sending it to the motion controller of the quick-release mechanism, and controlling the quick-release mechanism to perform an unlocking and retraction action.

[0123] In some embodiments, the method further includes: collecting multi-source data and corresponding manual annotation results during each docking and unlocking operation to construct an incremental learning dataset; when the number of samples in the incremental learning dataset reaches a preset sample threshold, starting incremental training of the defect detection network model and the multi-feature fusion state recognition model; after completing the incremental training, updating and replacing the currently used defect detection network model and multi-feature fusion state recognition model.

[0124] In some embodiments, the method further includes: continuously collecting working status data and defect data for each operation throughout the entire life cycle of the quick-release mechanism to construct a full life cycle operation dataset; fitting the changing trends of the wear degree of the locking surface and the deformation degree of the hook based on the full life cycle operation dataset; predicting the remaining reliable service life of the quick-release mechanism based on the fitted changing trends; and outputting an early maintenance prompt message when the predicted remaining reliable service life is lower than a preset service life threshold.

[0125] In some embodiments, the method further includes: uploading multi-source data collected by the device, identified working status data, and detected defect data to the fire operation command platform via a wireless communication link; receiving cluster operation defect statistics and abnormal status statistics of the same type of quick-release mechanism issued by the fire operation command platform; and updating the pre-set feature threshold range and alarm level correspondence of the device based on the cluster statistics to optimize the device's status identification and defect detection accuracy.

[0126] In some embodiments, the method further includes: before the fire-fighting drone performs fire-fighting operations, initiating a full-process self-inspection process of the quick-release mechanism, collecting multi-source data during the self-inspection process, identifying the working status during the self-inspection process, detecting defects in the mechanism, and generating a self-inspection report; when the self-inspection report shows that the quick-release mechanism has no abnormalities, granting the fire-fighting drone operating permissions; after each docking or unlocking operation is completed, encrypting and storing the full-process data, status identification results, defect detection results, health status, and alarm information of this operation to generate an operation traceability file.

[0127] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the steps of the multi-source fusion intelligent recognition method for the docking status of the quick-release mechanism of a fire-fighting drone as provided in any embodiment of this application.

[0128] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device.

[0129] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A multi-source fusion intelligent recognition method for the docking status of a quick-release mechanism for a fire-fighting drone, wherein the quick-release mechanism for the fire-fighting drone is a ball-and-socket type double-end docking quick-release mechanism, characterized in that... include: Collect position sensor data, force sensor data, and visual image data of the quick-release mechanism of the firefighting drone; Based on the collected position sensor data, force sensor data and visual image data, the working state of the quick-release mechanism is identified. The working state includes pre-docking state, guided insertion state, locked in place state, unlocked in place state, abnormal jamming state and loosening and disengaging state. Using deep learning algorithms, based on collected visual image data, defect data of the quick-release mechanism of the fire-fighting drone is detected. The defect data includes wear defects on the locking surface and deformation defects on the hook. Based on the identified working status and the detected defect data, the health status and graded alarm information of the quick-release mechanism are generated and output.

2. The method according to claim 1, characterized in that, The data collected from the position sensor, force sensor, and visual image of the quick-release mechanism of the firefighting drone includes: According to a preset fixed sampling frequency, position sensor data, force sensor data, and visual image data are collected synchronously. The acquired position sensor data and force sensor data are filtered and denoised, and the acquired visual image data are subjected to distortion correction and grayscale normalization. The processed position sensor data, force sensor data, and visual image data are timestamped to generate aligned multi-source data.

3. The method according to claim 2, characterized in that, The identification of the working status of the quick-release mechanism based on the collected position sensor data, force sensor data, and visual image data includes: The aligned multi-source data is input into a preset multi-feature fusion state recognition model to extract position features, force features and visual features. The extracted multi-dimensional features are matched with the preset feature threshold ranges corresponding to each working state; the working state with the highest matching degree is output as the recognition result, and the duration of the corresponding working state is recorded.

4. The method according to claim 3, characterized in that, The method employs a deep learning algorithm to detect defect data in the quick-release mechanism of the fire-fighting drone based on collected visual image data. This defect data includes wear defects on the locking surface and deformation defects on the hook. The preprocessed visual image data is input into a pre-trained defect detection network model; Image features of the locking surface region and hook region in the visual image are extracted using a defect detection network model; Based on the extracted image features, the defect type is identified, the size parameters of the defect are calculated, and the corresponding defect data is output.

5. The method according to claim 4, characterized in that, Based on the identified working status and detected defect data, the system generates and outputs health status and graded alarm information for the quick-release mechanism, including: Based on the identified abnormal types and durations of work status, a work status health score is generated. Based on the detected defect types and size parameters, a structural health score is generated. By combining the operational health score and the structural health score, the overall health status of the quick-release mechanism is generated; Based on the preset correspondence between health score ranges and alarm levels, generate and output the corresponding level of graded alarm information.

6. The method according to claim 1, characterized in that, The method further includes: The identified working status is input in real time into the flight controller and the action controller of the quick-release mechanism of the fire-fighting drone; When the pre-docking state or the guided positioning state is detected, a docking posture adjustment command is generated based on position sensor data and visual image data and sent to the flight controller to adjust the docking posture of the firefighting drone. When an abnormal jamming state is detected, an unlocking and retraction command is generated and sent to the motion controller of the quick-release mechanism to control the quick-release mechanism to perform the unlocking and retraction action.

7. The method according to claim 4, characterized in that, The method further includes: Collect multi-source data and corresponding manually labeled results during each docking and unlocking operation to construct an incremental learning dataset; When the number of samples in the incremental learning dataset reaches the preset sample threshold, incremental training of the defect detection network model and the multi-feature fusion state recognition model is initiated. After completing incremental training, update and replace the currently used defect detection network model and multi-feature fusion state recognition model.

8. The method according to claim 1, characterized in that, The method further includes: Continuously collect working status data and defect data for each operation throughout the entire life cycle of the quick-release mechanism to construct a full life cycle operation dataset; Based on the full life cycle operation dataset, the changing trends of the wear degree of the locking surface and the deformation degree of the hook are fitted; Based on the fitted trend, the remaining reliable service life of the quick-release mechanism is predicted. When the predicted remaining reliable service life is lower than the preset service life threshold, an early maintenance prompt message is output.

9. The method according to claim 1, characterized in that, The method further includes: The multi-source data collected by the machine, the identified working status data, and the detected defect data are uploaded to the fire operation command platform via a wireless communication link. Receive statistical data on defects and abnormal statuses of cluster operations of the same type of quick-release mechanism from the fire operation command platform; Based on cluster statistics, update the correspondence between the preset feature threshold range and alarm level on the local machine, and optimize the accuracy of local status recognition and defect detection.

10. The method according to claim 1, characterized in that, The method further includes: Before the firefighting drone performs firefighting operations, the quick-release mechanism initiates a full-process self-inspection process, collects multi-source data during the self-inspection process, identifies the working status during the self-inspection process, detects defects in the mechanism, and generates a self-inspection report. When the self-inspection report shows that there are no abnormalities in the quick-release mechanism, grant the fire-fighting drone operating permission; After each docking or unlocking operation is completed, the entire process data, status identification results, defect detection results, health status and alarm information of this operation are encrypted and stored to generate an operation traceability file.

Images