Acousto-optic fusion detection method and system for underwater vehicle

Abstract

The invention provides an acousto-optic fusion detection method and system for an underwater vehicle, and the method comprises the steps: obtaining the initial detection information of a suspicious target in an underwater region, and dynamically selecting a sensing strategy according to the initial detection information; and performing cooperative processing on data from different detection loads based on the sensing strategy, and continuously updating the detection information of the suspicious target through a processing result. The underwater camera, the laser camera and the foresight sonar are integrated on the bow section of the aircraft, a multi-source load cooperative detection framework is constructed, the system firstly utilizes the wide area coverage capability of the foresight sonar to carry out comprehensive scanning measurement on an underwater area, the initial position information of a suspicious target is quickly locked, and a sensing strategy is dynamically adjusted according to the target distance. According to the method, the contradiction between the limited distance of optical equipment and insufficient resolution of sonar equipment in traditional underwater detection is effectively solved, and the target recognition precision and the system robustness in a complex water area environment are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of underwater vehicle detection technology, and in particular to an acoustic-optical fusion detection method and system for underwater vehicles. Background Technology

[0002] Traditional underwater detection systems often employ a single detection payload, which has significant limitations: while sonar devices possess long-range detection capabilities, their resolution is limited; optical devices can provide high-precision images, but they are susceptible to water quality and have limited detection range. Existing technologies often employ fixed-weight fusion for acoustic-optical fusion, failing to dynamically adjust the perception strategy based on target distance, resulting in insufficient detail of near-range targets or missed detection of far-range targets.

[0003] Therefore, developing an acoustic-optical fusion detection method, especially an acoustic-optical fusion detection method for underwater vehicles, is of great significance for the development of autonomous detection technology for underwater vehicles and for improving the intelligence level of underwater vehicles. Summary of the Invention

[0004] The purpose of this invention is to provide an acoustic-optical fusion detection method and system for underwater vehicles, aiming to solve the above-mentioned problems in the prior art.

[0005] This invention provides an acoustic-optical fusion detection method for underwater vehicles, comprising: Acquire preliminary detection information of suspicious targets in the underwater area, and dynamically select a perception strategy based on the preliminary detection information; Based on the aforementioned perception strategy, data from different detection payloads are processed collaboratively, and the detection information of the suspicious target is continuously updated based on the processing results.

[0006] This invention provides an acoustic-optical fusion detection system for underwater vehicles, comprising: The dynamic perception and control module is used to acquire preliminary detection information of suspicious targets in the underwater area and dynamically select a perception strategy based on the preliminary detection information. The multi-source fusion detection module is used to collaboratively process data from different detection payloads based on the perception strategy, and continuously update the detection information of the suspicious target based on the processing results.

[0007] This invention also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the above-described acoustic-optical fusion detection method for underwater vehicles.

[0008] This invention also provides a computer-readable storage medium storing an information transmission implementation program, which, when executed by a processor, implements the steps of the above-described acoustic-optical fusion detection method for underwater vehicles.

[0009] The following beneficial effects can be achieved by adopting the embodiments of the present invention: The embodiments of the present invention provide an acoustic-optical fusion detection method for underwater vehicles, which combines the wide-area scanning of forward-looking sonar with the detailed capture of optical equipment to achieve efficient target detection across the entire range. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart of the acoustic-optical fusion detection method for underwater vehicles according to an embodiment of the present invention; Figure 2 This is a schematic diagram of an acoustic-optical fusion detection system for underwater vehicles according to an embodiment of the present invention. Detailed Implementation

[0012] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.

[0013] Method Implementation Examples According to embodiments of the present invention, an acoustic-optical fusion detection method for underwater vehicles is provided. Figure 1 This is a flowchart of the acoustic-optical fusion detection method for underwater vehicles according to an embodiment of the present invention, as follows: Figure 1 As shown, the acoustic-optical fusion detection method for underwater vehicles according to an embodiment of the present invention specifically includes: Step S101: Obtain preliminary detection information of suspicious targets in the underwater area, and dynamically select a perception strategy based on the preliminary detection information, specifically including: The vehicle uses forward-looking sonar to obtain preliminary location information of suspicious targets in the underwater area, calculates the target distance based on the preliminary location information, and selects to switch to a hybrid acoustic-optical sensing mode in which forward-looking sonar and underwater camera work together, or a single payload sensing mode in which only forward-looking sonar works, based on the comparison result of the target distance and a preset close-range threshold. If the target distance is less than or equal to a preset near-distance threshold, the sound and light hybrid sensing mode is activated. If the target distance is greater than the preset near-distance threshold, then the single load sensing mode is activated.

[0014] Step S102 involves collaboratively processing data from different detection payloads based on the perception strategy, and continuously updating the detection information of the suspicious target using the processing results. Specifically, this includes: In the aforementioned acoustic-optical hybrid sensing mode, the underwater camera and the forward-looking sonar work collaboratively to perform image enhancement processing on the optical images acquired by the underwater camera. Target detection is then performed on the enhanced optical images to obtain optical detection results. These optical detection results are then matched with the acoustic detection results from the forward-looking sonar. For acoustic and optical detection results matching the same target, a dynamic weighting method is used for weighted fusion. The target confidence level is updated based on the fusion result, and a fused detection result containing target location information and the updated confidence level is output. Specifically, this includes: For targets that are successfully matched and merged, the confidence level is increased by a preset reward value; for targets that are not successfully matched, the confidence level is decreased by a preset penalty value; and the confidence level value is constrained to the interval [0, 1]. The image enhancement process includes at least one of the following: dark channel prior dehazing algorithm, histogram equalization processing, or deep learning-based deblurring method. The fusion weights determined by the dynamic weight allocation method are calculated based on image quality weights, distance weights, and target detection confidence weights. The calculation formula is as follows: Formula 1; Formula 2; Formula 3; Formula 4; Formula 5; Where w represents the fusion weight, Indicates the corresponding coefficient. The image quality weights are represented by u, and the image quality threshold is represented by u. Indicates image quality evaluation metrics. Indicates distance weight, and These represent the minimum and maximum weights of the distance weights, respectively. This represents the distance adaptation coefficient, and k represents the slope parameter. Indicates distance, Indicates the distance from the center point. represents the confidence weight, and c represents the confidence score of the target detection result; The detection results include the target's three-dimensional position coordinates, the updated target confidence level, and target category information; the three-dimensional position coordinates are obtained by triangulation or coordinate fusion of successfully matched acoustic ranging information and the pixel positions of optical imaging. In the single-payload sensing mode, the forward-looking sonar is used for detection, and the detection information of the suspicious target is updated based on the detection results.

[0015] The following describes in detail the above-mentioned technical solutions of the present invention with reference to the specific circumstances of the acoustic-optical fusion detection method for underwater vehicles in the embodiments of the present invention.

[0016] This invention proposes an acoustic-optical fusion detection method for underwater vehicles, comprising the following steps: Step 1: Encapsulate underwater cameras, laser cameras, forward-looking sonar, and other acoustic and optical payloads in the bow section of the underwater vehicle; Step 2: The forward-looking sonar equipment performs a comprehensive scan of the underwater area, detects and locates underwater targets, and uses its wide-area coverage capability to quickly lock the preliminary location information of suspicious targets. The subsequent perception strategy is dynamically adjusted based on the location information of the suspicious targets. Among them, the target detection technology is a one-stage detection method based on deep learning. The dynamic adjustment of the subsequent perception strategy includes switching between single-source acoustic and optical payload / multi-source payload modes according to the target distance. Step 3: When the suspicious target is close, the acoustic-optical hybrid sensing mode is activated, and the optical camera and forward-looking sonar work together to optimize the optical image using image enhancement technology; among which, the image enhancement technology includes dark channel prior dehazing algorithm, histogram equalization processing or deep learning-based deblurring method; Step 4: When the optical camera detects a suspicious target, the acoustic and optical detection results are matched: the detection results that match the same suspicious target are weighted and fused, and the confidence of the suspicious target is increased; the confidence of targets that do not match optical detection results is reduced, while the optical detection results with high confidence are retained; the weighted fusion is a dynamic weight allocation method. Step 5: When the target is far away, detection is carried out using only forward-looking sonar equipment; Step 6: Repeat the above steps and output the detection results.

[0017] The present invention provides an acoustic-optical fusion detection method for underwater vehicles. Considering the poor imaging quality of underwater images, a deep learning-based detection model (YOLOv5 depth detection algorithm) is used. This method simultaneously meets the requirements of real-time performance, high accuracy, and robustness in underwater detection missions of autonomous underwater vehicles. The method includes: The underwater vehicle encapsulates acoustic and optical payloads such as underwater cameras, laser cameras, and forward-looking sonar into the bow section. The forward-looking sonar equipment performs a comprehensive scan of the underwater area, quickly locking in the preliminary location information of suspicious targets using its wide-area coverage capability, and dynamically adjusting subsequent perception strategies based on the location information of the suspicious targets.

[0018] When a suspicious target is close, the system will activate a hybrid acousto-optical sensing mode—where the optical camera and the forward-looking camera work together—to obtain more accurate detection results, as optical equipment can provide more detailed information than sonar within the visible range. To improve optical detection performance and range, image enhancement technology is used to optimize the optical image. When the optical camera detects a suspicious target, a clustering method (DBSCAN algorithm with parameters including neighborhood radius and minimum sample size) is used to match the acousto-optical detection results. Detection results matching the same suspicious target are weighted and fused, increasing the confidence level of that target. Targets without a matching optical detection result have their confidence level reduced. High-confidence optical detection results are retained.

[0019] When the target is far away, it is beyond the field of view of the underwater camera and laser camera, so the system will rely solely on forward-looking sonar equipment for detection.

[0020] Repeat the above steps and output the detection results.

[0021] The present invention will be further described below with reference to specific embodiments: The purpose of this invention is to provide an acoustic-optical fusion detection method for underwater vehicles, enabling online, real-time detection and positioning of guided targets in complex underwater environments, providing the location of docking devices for autonomous docking and recovery of underwater vehicles. The detection method includes the following steps: 1. The underwater camera, laser camera, forward-looking sonar, and other acoustic and optical payloads are encapsulated in the bow section of the underwater vehicle.

[0022] 2. The forward-looking sonar equipment performs a comprehensive scan of the underwater area, utilizing its wide-area coverage capability to quickly locate the preliminary position information of suspicious targets. The forward-looking sonar emits a fan-shaped beam, and the echo signal is processed to generate a sonar image. Suspicious target bounding boxes are obtained through target detection methods, and the target distance D, azimuth angle θ, and confidence level Conf are calculated. Subsequent sensing strategies are dynamically adjusted based on the location information of the suspicious targets. Among them, the object detection technology is a one-stage detection method based on deep learning (deep detection algorithm YOLOv5). Dynamically adjusting subsequent perception strategies includes switching between single-source and multi-source acoustic and optical payload modes based on target distance: when the suspected target is close, the acoustic and optical hybrid perception mode is activated; when the target is far away, only forward-looking sonar equipment is used for detection. In the acoustic-optical hybrid sensing mode, the optical camera and forward-looking sonar work together, and image enhancement technology is used to optimize the optical image. Image enhancement techniques include dark channel prior dehazing algorithms, histogram equalization, or deep learning-based deblurring methods.

[0023] 3. When the optical camera detects a suspicious target, the acoustic and optical detection results are matched; The matching method is the DBSCAN clustering algorithm, whose parameters include neighborhood radius and minimum number of samples.

[0024] 4. Perform weighted fusion on detection results that match the same suspicious target, and increase the confidence level of the suspicious target; Among them, weighted fusion is a dynamic weight allocation method: based on the complementarity and reliability of acoustic and optical information, the fusion weight of acoustic and optical detection results needs to comprehensively consider multiple factors such as image quality, distance, and target detection confidence. Therefore, it is necessary to calculate the weights of image quality, distance, and confidence.

[0025] Image quality is a core indicator of the reliability of optical detection results, and weight allocation should prioritize the contribution of high-resolution, low-noise images. The underwater color image quality assessment index is adopted, with a set image quality threshold u and image quality weights. for: ; in, (Underwater Colour Image Quality Evaluation) is a no-reference image quality evaluation metric specifically designed for underwater imaging. It assesses the quality of underwater images by quantifying their chroma, saturation, and contrast, and is particularly suitable for addressing common underwater image problems such as non-uniform color cast, blurriness, and low contrast.

[0026] Distance Information: Acoustic detection has a long range, and its accuracy decreases less with distance. Optical detection, on the other hand, has high reliability at short range and can be assigned a higher weight, but its accuracy decreases significantly with distance. Therefore, the Sigmoid activation function is used to simulate the distance response characteristics of the acoustic and optical detection payloads respectively. By adjusting the parameters of the Sigmoid function, the weight allocation for acoustic and optical detection at different distances can be precisely controlled. Distance Weights for: ; ; Where k represents the slope parameter, which controls the steepness of the curve; the larger the absolute value of k, the steeper the curve changes, and the more rapidly the weight changes with distance. Indicates distance, It represents the distance from the center point and determines the position of the curve on the horizontal axis, that is, the distance at which the weight begins to change significantly (decline). It is a distance adaptation coefficient, which is based on the current detection distance. Relative to a certain center distance The distance is considered, and a value between 0 and 1 is output to control the weight of acoustic or optical detection results in the fusion. This is achieved by adjusting k and... This allows control over the steepness and center position of the transition, thus simulating the response characteristics of different sensors at different distances. This can be achieved by setting minimum weights. and maximum weight This ensures that the weights are neither too high nor too low, improving the robustness of the system. Based on the different characteristics of acoustic and optical detection, different parameters are set so that acoustic detection results maintain a high and stable weight at long distances with slow decay; optical detection results are given a high weight at close distances, which decreases rapidly with increasing distance.

[0027] Target detection confidence: Based on the confidence level of the target detection results, high-confidence results are assigned a dominant weight to avoid low-confidence detection results interfering with the fusion decision. Confidence weight for: ; Where c represents the confidence level of the target detection result.

[0028] In summary, the fusion weight is the sum of the weights mentioned above: ; in, Let be the coefficient and w be the fusion weight. The fusion weights of detection results that successfully match the same target are normalized, and the detection results are then summed according to the fusion weights. In addition, high-confidence optical detection results will be retained.

[0029] 5. For targets that do not match optical detection results, their confidence level is reduced, while high-confidence optical detection results are retained.

[0030] 6. The confidence level of the detection results is dynamically adjusted. The confidence level of successfully matched and fused detection results is increased by a reward value (e.g., +0.2), but the final upper limit of confidence level is strictly constrained to 1. For unmatched acousto-optical detection results, a penalty mechanism is implemented (e.g., confidence level is reduced by 0.2), and the lower limit of confidence level after penalty is guaranteed to be 0. This can effectively fuse high-quality detection data and suppress low-reliability results, significantly improving the robustness and accuracy of underwater target detection.

[0031] 7. Repeat the above steps and output the detection results.

[0032] System Implementation Examples According to embodiments of the present invention, an acoustic-optical fusion detection system for underwater vehicles is provided. Figure 2 This is a schematic diagram of an acoustic-optical fusion detection system for underwater vehicles according to an embodiment of the present invention, as shown below. Figure 2 As shown, the acoustic-optical fusion detection system for underwater vehicles according to an embodiment of the present invention specifically includes: The dynamic perception control module 20 is used to acquire preliminary detection information of suspicious targets in the underwater area and dynamically select a perception strategy based on the preliminary detection information. The multi-source fusion detection module 22 is used to collaboratively process data from different detection payloads based on the perception strategy, and continuously update the detection information of the suspicious target based on the processing results.

[0033] The embodiments of the present invention are system embodiments corresponding to the above method embodiments. The specific operation of each module can be understood by referring to the description of the method embodiments, and will not be repeated here.

[0034] In summary, this invention proposes an acoustic-optical fusion detection method for underwater vehicles. By integrating an underwater camera, a laser camera, and a forward-looking sonar into the bow section of the vehicle, a multi-source payload collaborative detection architecture is constructed. The system first utilizes the wide-area coverage capability of the forward-looking sonar to comprehensively scan the underwater area, quickly locking the preliminary location information of suspicious targets. The sensing strategy is then dynamically adjusted based on the target distance: when the target is close, an acoustic-optical fusion sensing mode is activated, with the optical camera and forward-looking sonar working collaboratively. Image enhancement algorithms are used to de-haze and enhance the optical image, and the acoustic-optical detection results are matched. Successfully matched targets of the same type are weighted and fused. When the target is far away, the system relies solely on the forward-looking sonar for detection. This method, by adjusting the sensing strategy and dynamic weight allocation mechanism, effectively solves the contradiction between the limited range of optical equipment and the insufficient resolution of sonar equipment in traditional underwater detection, significantly improving target recognition accuracy and system robustness in complex aquatic environments. Compared with existing technologies, the beneficial effects of this invention are: 1. This invention proposes an acoustic-optical fusion sensing strategy, which can effectively improve the underwater target detection range, alleviate the problem of single payload failure in complex underwater environments, enable it to carry out underwater operations efficiently and autonomously, and promote the intelligent development of underwater unmanned equipment. 2. This invention features real-time performance, low computational cost, and high robustness, enabling underwater vehicles to efficiently and in real-time complete autonomous docking and recovery.

[0035] Device Example 1 This invention provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it performs the steps described in the method embodiment.

[0036] Device Example 2 This invention provides a computer-readable storage medium storing an information transmission implementation program, which, when executed by a processor, performs the steps described in the method embodiment.

[0037] The computer-readable storage media described in this embodiment include, but are not limited to, ROM, RAM, disk, or optical disk.

[0038] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for acoustic-optical fusion detection of underwater vehicles, characterized in that, include: Acquire preliminary detection information of suspicious targets in the underwater area, and dynamically select a perception strategy based on the preliminary detection information; Based on the aforementioned perception strategy, data from different detection payloads are processed collaboratively, and the detection information of the suspicious target is continuously updated based on the processing results.

2. The method according to claim 1, characterized in that, Acquiring preliminary detection information of suspicious targets in an underwater area, and dynamically selecting a sensing strategy based on the preliminary detection information, specifically includes: The vehicle uses forward-looking sonar to obtain preliminary location information of suspicious targets in the underwater area, calculates the target distance based on the preliminary location information, and selects to switch to a hybrid acoustic-optical sensing mode in which forward-looking sonar and underwater camera work together, or a single payload sensing mode in which only forward-looking sonar works, based on the comparison result of the target distance and a preset close-range threshold. If the target distance is less than or equal to a preset near-distance threshold, the sound and light hybrid sensing mode is activated. If the target distance is greater than the preset near-distance threshold, then the single load sensing mode is activated.

3. The method according to claim 2, characterized in that, Based on the aforementioned perception strategy, data from different detection payloads are collaboratively processed, and the detection information of the suspicious target is continuously updated based on the processing results. Specifically, this includes: In the aforementioned acoustic-optical hybrid sensing mode, the underwater camera and the forward-looking sonar work together to perform image enhancement processing on the optical images acquired by the underwater camera, and then perform target detection on the enhanced optical images to obtain optical detection results. The optical detection results are then matched with the acoustic detection results of the forward-looking sonar. For the acoustic and optical detection results that match the same target, a dynamic weight allocation method is used to perform weighted fusion. Based on the fusion result, the target confidence is updated, and a fused detection result containing target location information and the updated confidence is output. In the single-payload sensing mode, the forward-looking sonar is used for detection, and the detection information of the suspicious target is updated based on the detection results.

4. The method according to claim 3, characterized in that, The image enhancement process includes at least one of the following: dark channel prior dehazing algorithm, histogram equalization, or deep learning-based deblurring method.

5. The method according to claim 3, characterized in that, The fusion weights determined by the dynamic weight allocation method are calculated based on image quality weights, distance weights, and target detection confidence weights. The calculation formula is as follows: Official 1; Official 2; Official 3; Official 4; Official 5; Where w represents the fusion weight, Indicates the corresponding coefficient. The image quality weights are represented by u, and the image quality threshold is represented by u. Indicates image quality evaluation index, Indicates distance weight, and These represent the minimum and maximum weights of the distance weights, respectively. This represents the distance adaptation coefficient, and k represents the slope parameter. Indicates distance, Indicates the distance from the center point. denoted by , where c represents the confidence weight and c represents the confidence score of the target detection result.

6. The method according to claim 3, characterized in that, Updating the target confidence based on the fusion results specifically includes: The confidence level of a successfully matched and merged target is increased by a preset reward value; the confidence level of a target that is not successfully matched is decreased by a preset penalty value; and the confidence level value is constrained to the interval [0, 1].

7. The method according to claim 6, characterized in that, The detection results include the target's three-dimensional position coordinates, the updated target confidence level, and target category information; The three-dimensional position coordinates are obtained by triangulation or coordinate fusion of the successfully matched acoustic ranging information and the pixel position of the optical imaging.

8. An acoustic-optical fusion detection system for underwater vehicles, characterized in that, include: The dynamic perception and control module is used to acquire preliminary detection information of suspicious targets in the underwater area and dynamically select a perception strategy based on the preliminary detection information. The multi-source fusion detection module is used to collaboratively process data from different detection payloads based on the perception strategy, and continuously update the detection information of the suspicious target based on the processing results.

9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the acoustic-optical fusion detection method for underwater vehicles as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an implementation program for information transmission, which, when executed by a processor, implements the steps of the acoustic-optical fusion detection method for underwater vehicles as described in any one of claims 1-7.

Images