A visual guidance system and method for unmanned surface vessels

Abstract

This application provides an unmanned surface vessel (USV) visual guidance system and method, which creates a USV visual guidance system that can achieve high-precision target detection of a mother ship swaying on the sea surface, and can also obtain the precise position and attitude of the USV relative to the mother ship. This can provide real-time and accurate automated guidance for USV recovery, ensuring that under all weather conditions and complex sea conditions, it can provide strong data support for guiding the USV to safely berth, follow the track and navigate, and be successfully recovered, thus meeting the requirements of high-quality USV recovery work.

Description

Technical Field

[0001] This application relates to the field of unmanned surface vessels (USVs), specifically to a visual guidance system and method for USVs. Background Technology

[0002] Unmanned surface vessels (USVs) are vessels capable of autonomous navigation. In practical applications, they can be equipped with various functional modules according to mission requirements to complete a series of tasks autonomously or semi-autonomously, thus possessing significant application value.

[0003] The inventors of this application have discovered that the current recovery of unmanned surface vessels can only be carried out manually at low speeds and in low sea conditions, which obviously has the problem of low recovery efficiency. Furthermore, since it involves manual operation, it also carries a certain degree of danger. Summary of the Invention

[0004] This application provides an unmanned surface vessel (USV) visual guidance system and method, which creates a USV visual guidance system that can achieve high-precision target detection of a mother ship swaying on the sea surface, and can also obtain the precise position and attitude of the USV relative to the mother ship. This can provide real-time and accurate automated guidance for USV recovery, ensuring that under all weather conditions and complex sea conditions, it can provide strong data support for guiding the USV to safely berth, follow the track and navigate, and be successfully recovered, thus meeting the requirements of high-quality USV recovery work.

[0005] In the first aspect, this application provides an unmanned surface vessel (USV) visual guidance system, which includes an USV and a mother ship. The USV is equipped with a passive infrared vision system and an active infrared vision system, and the mother ship is equipped with a reflective near-infrared cooperative target at a preset mother ship docking point.

[0006] When the unmanned surface vessel is more than 30 meters away from the mother ship, in passive working mode, it emits mid-to-far infrared light in the 8-12 μm band to the mother ship through a passive infrared vision system to detect the mother ship and guide itself to approach the mother ship based on the target detection results.

[0007] When the unmanned surface vessel (USV) is within 30 meters of the mother ship, it illuminates the docking area of ​​the mother ship with 900nm near-infrared light through its active vision system in active working mode. This causes the reflective near-infrared cooperative target to emit high-brightness infrared light, thus obtaining a corresponding high-brightness near-infrared cooperative target image. Then, by combining the high-brightness near-infrared cooperative target image and the geometric information of the reflective near-infrared cooperative target, it calculates its own position and attitude relative to the mother ship. Based on its position and attitude, it guides itself to approach the mother ship to complete the docking and recovery mission.

[0008] Secondly, this application provides a visual guidance method for unmanned surface vessels (USVs). This method is applied to a USV visual guidance system, which includes an USV and a mothership. The USV is equipped with a passive infrared vision system and an active infrared vision system. The mothership has a reflective near-infrared cooperative target positioned at a pre-defined docking point. The USV visual guidance method includes:

[0009] When the unmanned surface vessel is more than 30m away from the mother ship, in passive working mode, it emits mid-to-far infrared light in the 8-12um band through a passive infrared vision system to detect the mother ship and guide itself to approach the mother ship based on the target detection results.

[0010] When the unmanned surface vessel is within 30m of the mother ship, in active working mode, it illuminates the docking part of the mother ship with 900nm band near-infrared light through the active vision system, causing the reflective near-infrared cooperative target to emit high-brightness infrared light, and obtains the corresponding high-brightness near-infrared cooperative target image.

[0011] The unmanned surface vessel (USV) combines high-brightness near-infrared cooperative target images and geometric information of reflective near-infrared cooperative targets to determine its own position and attitude relative to the mother ship.

[0012] Based on its position and attitude, the unmanned surface vessel guides itself toward the mother ship to complete the docking and recovery mission.

[0013] Thirdly, this application provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute the methods provided in the second aspect of this application.

[0014] From the above, it can be concluded that this application has the following beneficial effects:

[0015] For the recovery of unmanned surface vessels (USVs), this application develops a visual guidance system for USVs. This system can achieve high-precision target detection of a mother ship that is swaying on the sea surface, and can also obtain the precise position and attitude of the USV relative to the mother ship. This provides real-time and accurate automated guidance for the recovery of USVs, ensuring that the system can provide strong data support for guiding the USV to safely berth, follow the track, and be successfully recovered under all weather conditions and complex sea conditions, thus meeting the requirements for high-quality USV recovery operations. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the system structure of the unmanned surface vessel visual guidance system of this application;

[0018] Figure 2 This is a schematic diagram of a scenario for the visual guidance system of the unmanned surface vessel of this application;

[0019] Figure 3 This is a schematic diagram of a scene for denoising and enhancing infrared target images according to this application;

[0020] Figure 4 A schematic diagram of a scenario for constructing invariant moments for this application;

[0021] Figure 5 This is a schematic diagram illustrating a scenario of reprojection error in this application;

[0022] Figure 6 This is a schematic diagram of a scenario for position and attitude measurement of cooperative targets based on reflective near-infrared radar, as described in this application.

[0023] Figure 7 This is a schematic diagram of a scenario for the semi-physical simulation experiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those explicitly listed, but may include other steps or modules not explicitly listed or inherent to such processes, methods, products, or devices. The naming or numbering of steps appearing in this application does not imply that the steps in the method flow must be performed in the chronological / logical order indicated by the naming or numbering. The execution order of named or numbered process steps can be changed according to the desired technical purpose, as long as the same or similar technical effect is achieved.

[0026] The module division described in this application is a logical division. In practical applications, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the coupling or direct coupling or communication connection between modules shown or discussed may be through some interfaces, and the indirect coupling or communication connection between modules may be electrical or other similar forms, none of which are limited in this application. Furthermore, the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed in multiple circuit modules. Some or all of the modules may be selected to achieve the purpose of the solution in this application according to actual needs.

[0027] See Figure 1 The diagram shows a system structure of the unmanned surface vessel (USV) visual guidance system provided in this application. The USV visual guidance system mainly consists of two parts: the USV and the mother ship. The USV is equipped with a passive infrared vision system and an active infrared vision system specially designed in this application. The mother ship is equipped with a reflective near-infrared cooperative target at a preset docking point (mother ship recovery compartment). It can be understood that the design of these components is for the purpose of USV recovery. The following dynamic workflow will be used to better understand the processing involved when the USV visual guidance system performs automatic USV recovery.

[0028] (1) When the unmanned surface vessel is more than 30m away from the mother ship, in passive working mode, it emits 8-12um mid-far infrared light to the mother ship through the passive infrared vision system to carry out target detection on the mother ship, and guides itself to approach the mother ship based on the target detection results.

[0029] It's important to note that during this phase, the unmanned surface vessel (USV) doesn't necessarily need to know the exact distance between itself and the mother ship. In practice, the USV's deployment / operation location is usually not too far from the mother ship, remaining within 200 meters, and when recovery is required, it will be at least 30 meters away.

[0030] In this case, the passive infrared vision system specially configured in this application can be used first. When the unmanned vessel recovery operation is triggered, it will continue to emit mid-to-far infrared light in the 8-12µm band towards the mother ship in passive working mode. When emitting this mid-to-far infrared light, it is not necessarily directly aimed at the mother ship. Instead, it will identify, locate and track the mother ship based on target detection using mid-to-far infrared light. At the same time, the direction of the emitted mid-to-far infrared light can also be adjusted. Of course, the emission direction of the mid-to-far infrared light can also be fixed, which can be adjusted according to the configuration of the passive infrared vision system.

[0031] Thus, by using a passive infrared vision system, the direction of the mother ship can be identified at a distance, and the unmanned surface vessel can approach the mother ship through its own propulsion system until the distance between itself and the mother ship is within 30m, triggering an active working mode to achieve more precise and delicate approach control, in order to meet the needs of unmanned surface vessel recovery operations.

[0032] Understandably, since passive infrared vision systems use the mid-to-far infrared band (8-12µm), they have strong penetration capabilities in all-weather conditions and complex sea conditions, including rain, fog, and smoke. This all-weather detection capability provides excellent adaptability and ensures stable, efficient, and accurate processing results in the identification, positioning, and tracking of mother ships at medium to long distances.

[0033] (2) When the unmanned surface vessel is within 30m of the mother ship, it illuminates the docking part of the mother ship with 900nm near-infrared light through the active vision system in active working mode, so that the reflective near-infrared cooperative target emits high-brightness infrared light and obtains the corresponding high-brightness near-infrared cooperative target image; then, by combining the high-brightness near-infrared cooperative target image and the geometric information of the reflective near-infrared cooperative target, it solves its own position and attitude relative to the mother ship, and then guides itself to approach the mother ship based on its position and attitude to complete the docking and recovery task.

[0034] It is understandable that as the unmanned surface vessel (USV) approaches, the distance between the USV and the mother ship gets closer and closer until it reaches 30m (the distance can be measured by target detection in the passive working mode or by other ranging methods). At this point, the active working mode can be triggered. The active vision system actively emits / irradiates 300nm near-infrared light towards the pre-set docking point of the mother ship, causing the reflective near-infrared cooperative target configured at the docking point to emit / respond with bright infrared light (relative to the background). This obtains a high-brightness near-infrared cooperative target image, avoiding the need for special noise reduction processing. At this point, the imaging geometry information of the pre-determined reflective near-infrared cooperative target can be combined to accurately calculate the position and attitude of the USV relative to the docking point of the mother ship in real time (including position and attitude information), providing data support for guiding the USV to complete the docking and recovery mission of the mother ship.

[0035] This process can also be combined with Figure 2 The following is a schematic diagram of a scenario for the visual guidance system of the unmanned surface vessel of this application, which will be shown for a more vivid understanding.

[0036] Understandably, because the docking point of the mother ship adopts a near-infrared reflective cooperative target design, the mother ship itself does not emit infrared light and does not need to communicate with the unmanned surface vessel. Therefore, this can also increase the stealth performance of the mother ship. At the same time, in order to avoid the interference of the infrared heat source on the mother ship to the cooperative target, a reflective near-infrared cooperative target design is adopted, which can also greatly improve the imaging performance and detection reliability of the cooperative target. No special noise reduction processing is required, thus further enhancing the adaptability to different complex situations and achieving a more intelligent autonomous recovery effect for the unmanned surface vessel.

[0037] Thus, for Figure 1 As can be seen from the embodiment described, this application has developed a visual guidance system for unmanned surface vessels (USVs) to recover targets. This system can achieve high-precision target detection of a mother ship swaying on the sea surface and can also obtain the precise position and attitude of the USV relative to the mother ship. This provides real-time and accurate automated guidance for the recovery of USVs, ensuring that under all weather conditions and complex sea conditions, it can provide strong data support for guiding the USV to safely berth, follow its path, and be successfully recovered from the mother ship, thus meeting the requirements for high-quality USV recovery operations.

[0038] Furthermore, in practical applications, it is understandable that during the process of the unmanned surface vessel (USV) docking with the mother ship, due to the combined effects of factors such as the relatively long distance, the swaying and movement of the mother ship target, different camera imaging angles, weather conditions, and inherent limitations of the infrared imaging system itself, the infrared target image of the mother ship detected by the passive infrared vision system on the USV suffers from problems such as low clarity, low contrast, blurred target edges, non-uniformity, and high noise. These problems may result in poor ability of the infrared image to distinguish the shape and edge texture of small targets, causing significant interference to subsequent target detection and tracking, and thus affecting the completion of the USV's real-time precise guidance mission.

[0039] Therefore, this application also conducted a series of studies on the processing accuracy of passive infrared vision systems in order to further improve the performance and application value of the solution.

[0040] Specifically, as an exemplary embodiment, the target detection algorithm for the unmanned surface vessel (USV) to detect the mother ship is configured based on the mother ship target recognition database (which can also be understood as a sample image set of the mother ship). The configuration of the mother ship target recognition database can specifically include:

[0041] Based on the installation height of the passive infrared imaging system on the unmanned surface vessel (USV), images of the USV rocking the mother ship from different angles as it approaches the mother ship are acquired within a range of 200-30m from the USV to the mother ship, in increments of 10m. A target recognition database for the mother ship is then established using these images.

[0042] It is understood that the embodiment here provides a specific implementation scheme for collecting sample images of the mother ship under different complex conditions in order to build a mother ship target recognition database.

[0043] Furthermore, in combination Figure 3 The illustrated scenario of infrared target image denoising enhancement according to this application is, as an exemplary embodiment, further including the configuration of the mother ship target recognition database after its establishment, after which the configuration of the mother ship target recognition database may also include:

[0044] Contourlet transform is performed on the images in the mother ship target recognition database to obtain multi-scale, multi-directional low-pass subbands and bandpass subbands;

[0045] A linear transformation of the minimum and maximum values ​​is applied to the low-pass subband to improve image contrast;

[0046] For the bandpass sub-generation, noise suppression is performed based on the statistical characteristics of image noise. Then, high-frequency information is enhanced through a blur enhancement algorithm to highlight target edges and suppress background and noise.

[0047] The data processed from the low-pass and band-pass subbands are then subjected to inverse Contourlet transform to obtain denoised and enhanced infrared target images, thus updating the mother ship target identification database.

[0048] As can be seen, considering the characteristics of infrared images (mother ship target identification database) such as high target noise interference, blurred target edges, and low contrast, this application analyzes the statistical distribution characteristics between pixels and their noise in infrared images based on the imaging mechanism of infrared images. Combining the basic shape features of the target image, it establishes corresponding image denoising and enhancement evaluation indicators, and then studies the denoising and enhancement methods for infrared target images on the sea surface. Specifically, based on the previously constructed mother ship target identification database, it studies an infrared image denoising and enhancement algorithm based on Contourlet transform and fuzzy theory, and provides the above-mentioned very specific implementation scheme. This can achieve efficient and effective denoising and enhancement of infrared images (mother ship target identification database), which will help to configure more accurate and efficient corresponding target detection algorithms.

[0049] Furthermore, also targeting target detection algorithms based on passive infrared vision systems, this application studies the degradation patterns of real mother ship target images caused by factors such as long target distance, motion and swaying, different camera imaging angles, and uneven thermal imaging. Based on this, and combining image geometric transformation, grayscale degradation (image blurring, photometric transformation, and contrast transformation) and invariant moment theory, a highly reliable target detection algorithm for mother ships swaying on the sea surface is developed. Validation experiments are then conducted, and the stability and real-time performance of the algorithm are continuously improved through data feedback from the validation experiments.

[0050] In this regard, as an exemplary embodiment, reference is made to Figure 4 The diagram shown illustrates a scenario for constructing invariant moments according to this application. The configuration of the target detection algorithm for unmanned surface vessels (USVs) to detect targets on the mother ship, based on the mother ship target recognition database, can include:

[0051] Based on the updated mother ship target recognition database, and on the basis of the invariant moment theory with rotation, translation and scaling invariance, combined with the illumination fuzziness invariance theory, as well as the geometric degradation and grayscale degradation mechanism, and the geometric radiative transformation theory, a recognition moment with illumination, fuzziness and affine invariance is constructed.

[0052] A target detection algorithm is constructed based on recognition moments that are invariant to illumination, blur, and affine transformation.

[0053] It is understood that, for the embodiments described here, this application addresses the problem of degradation of real mother ship target images caused by the combined effects of long distance, different viewing angles, rain and fog, infrared heat halo, and lens distortion. It proposes a fuzzy recognition moment with illumination, blur, and affine invariance to improve the reliability of target recognition in mother ship infrared images.

[0054] Furthermore, as an exemplary embodiment, the configuration of the target detection algorithm for unmanned surface vessels to conduct target detection on the mother ship based on the mother ship target recognition database may also include:

[0055] Sampling is performed on the updated mothership target identification database using the queen template sampling algorithm.

[0056] Based on the sampling results, an algorithm is configured based on the grayscale of the feature points corresponding to the reflective near-infrared cooperative target and the optical flow characteristics of the reflective near-infrared cooperative target as it sways with the mother ship.

[0057] It is understandable that the Queen Template Sampling Algorithm / Chess Queen Template Rules can meet the needs of rapid interpretation of images / videos based on the Queen Template, so as to facilitate more efficient configuration of object detection algorithms.

[0058] Furthermore, at a distance of 30 meters, it can be seen that the active infrared imaging of the reflective near-infrared cooperative target at the docking point of the mother ship is already relatively clear. Therefore, the cooperative target point can be directly extracted based on the high gray level of the cooperative point. In order to enhance reliability, taking advantage of the characteristic that the cooperative point moves with the mother ship, and based on its optical flow characteristics, this application presents a near-infrared cooperative target detection method based on the combination of feature point gray level and optical flow characteristics.

[0059] Furthermore, this application also considers the situation where waves may obscure part of the cooperative target points. Therefore, as an exemplary embodiment, the configuration of the target detection algorithm for unmanned surface vessels (USVs) to detect targets on the mother ship based on the mother ship target recognition database may further include:

[0060] Based on the known layout patterns of reflective near-infrared cooperative targets, an automatic adjustment algorithm for target point tracking boxes with occlusion adaptability is configured.

[0061] Thus, by using this target point tracking box auto-adjustment algorithm to predict the characteristics of the cooperative target behind the occlusion, the tracking reliability of the cooperative target can be further improved.

[0062] Furthermore, focusing on the active infrared vision system, it is understandable that the high-precision measurement of the relative position and attitude between the mother ship docking point and the unmanned surface vessel is the key to the success or failure of the docking. The number and layout of cooperative targets placed at the mother ship docking point have an important impact on the accuracy and stability of the pose measurement. Usually, the space at the docking point on the ship is limited, and it is not permissible to arbitrarily and excessively place infrared reflective cooperative targets.

[0063] In this context, as an exemplary embodiment, the configuration of the target detection algorithm involved in the unmanned surface vessel's (USV) process of determining its position and attitude relative to the mother ship may include:

[0064] (1) Based on the camera imaging model and PnP pose calculation principle (EPnP algorithm), combined with the constraints of the docking part of the mother ship, the number of cooperative target points and the distribution position of cooperative targets are used as optimization variables, and the minimum error between the position and attitude calculated by the PnP algorithm and the actual position and attitude is used as the optimization index. The distribution position of the reflective near-infrared cooperative targets at the docking part of the mother ship is obtained by optimizing through the genetic algorithm.

[0065] The specific constraints include: requiring the cooperative targets to be distributed as close as possible to the edge of the docking area (recovery compartment) of the mother ship, so that the distance between the cooperative points is large, which can further improve the accuracy of docking posture measurement.

[0066] (2) Explore the linear and nonlinear constraints in the imaging process, and use the parameterized imaging model and the minimum deviation between the image coordinates of the cooperative target imaging and the image coordinates obtained by reprojection through the pose measurement model as the index. Establish a cost function for nonlinearly solving the pose from the perspective of global optimization, and use the non-iterative optimization theory algorithm as the basis to solve the position and attitude measurement algorithm with high stability, high accuracy and high real-time performance.

[0067] Regarding the image coordinate deviation involved here, it can also be combined with Figure 5 The following is a schematic diagram illustrating a scenario of reprojection error in this application for a more intuitive understanding.

[0068] For this part of the processing, it can also be combined with Figure 6 The schematic diagram shown here illustrates a scenario based on reflective near-infrared cooperative target position and attitude measurement, which can be used to provide a more intuitive understanding of this application.

[0069] Specific nonlinear constraints include: the orthogonality of the rotation matrix during imaging, and the property that the translation matrix can be linearly represented.

[0070] Non-iterative optimization algorithms can specifically employ algorithms such as Grobner basis theory.

[0071] The so-called high stability, high precision, and high real-time performance specifically include: a measurement accuracy of 0.1m within 30m, a single run time of less than 1ms on a regular industrial control computer, and the ability to perform stable measurements when the deviation angle between the unmanned surface vessel and the mother ship is relatively large.

[0072] It is understood that, for this embodiment, this application improves the pose measurement accuracy by optimizing the number of cooperative target points and the layout based on the constraints of the mother ship docking location and the PNP pose measurement theory. Furthermore, considering that the accuracy and stability of existing simple linear pose calculation methods need to be improved, and that iterative pose calculation methods are prone to getting trapped in local optima, this application, in combination with the characteristics of the close-range mother ship docking location, analyzes the constraint relationships in the imaging process based on the physical and mathematical models of the imaging system and the principles of the position and attitude calculation method. It studies the optimization problem of the number of cooperative target points and the spatial layout of reflective near-infrared cooperative targets and proposes a new method for solving pose with high accuracy from a global perspective through imaging physical and mathematical models and non-iterative optimization theory, so as to further improve the pose measurement accuracy and reliability.

[0073] Furthermore, for the series of target detection algorithms mentioned above, or for the different target detection algorithms involved throughout the process, this application can also combine semi-physical simulation experiments for verification and optimization in practical applications, so as to further ensure the performance of the algorithms.

[0074] For this, please refer to Figure 7The schematic diagram of a semi-physical simulation experiment of this application is shown as an exemplary embodiment. For different target detection algorithms involved throughout the process, the corresponding configuration work may include:

[0075] In the semi-physical simulation experiment, the simulated components of the mother ship docking part were set on the ground at a certain distance from the three-axis turntable. An active infrared vision system was installed on the three-axis turntable. The three-axis turntable simulated sea conditions and shook the active infrared vision system on the simulated components of the unmanned surface vessel. The position between the mother ship docking part and the simulated components of the unmanned surface vessel was obtained by measuring with a laser total station. The attitude parameters were obtained by measuring the parameters of the three-axis turntable itself. Rain and fog weather were achieved by spraying water and fog with a spray machine.

[0076] Based on the experimental results of the semi-physical simulation experiment, we will continue to optimize the different target detection algorithms involved in the entire process.

[0077] Thus, the above settings are used to advance the semi-physical simulation experiments involved in this application, verify the accuracy, stability and other indicators of the relevant target detection algorithms, and further improve the algorithm content based on the problems that arise.

[0078] The above is an introduction to the unmanned surface vessel (USV) visual guidance system provided in this application. Based on this, this application also provides an USV visual guidance method from the perspective of the system's workflow. This USV visual guidance method is applied to the USV visual guidance system, which includes an USV and a mothership. The USV is equipped with a passive infrared vision system and an active infrared vision system. The mothership has a reflective near-infrared cooperative target positioned at a pre-defined docking point. The USV visual guidance method includes the following steps:

[0079] (1) When the unmanned surface vessel is more than 30m away from the mother ship, in passive working mode, it emits 8-12um mid-far infrared light to the mother ship through the passive infrared vision system to detect the target of the mother ship, and guides itself to approach the mother ship based on the target detection results.

[0080] (2) When the unmanned surface vessel is within 30m of the mother ship, it illuminates the docking part of the mother ship with 900nm band near-infrared light through the active vision system in active working mode, so that the reflective near-infrared cooperative target emits high-brightness infrared light and obtains the corresponding high-brightness near-infrared cooperative target image.

[0081] (3) The unmanned surface vessel combines the high-brightness near-infrared cooperative target image and the geometric information of the reflective near-infrared cooperative target to solve its position and attitude relative to the mother ship.

[0082] (4) The unmanned surface vessel guides itself toward the mother ship based on its position and attitude in order to complete the docking and recovery task.

[0083] In one exemplary embodiment, the target detection algorithm for the unmanned surface vessel (USV) to detect the mother ship is configured based on the mother ship target recognition database. The configuration of the mother ship target recognition database includes:

[0084] Based on the installation height of the passive infrared imaging system on the unmanned surface vessel (USV), images of the USV rocking the mother ship from different angles as it approaches the mother ship are acquired within a range of 200-30m from the USV to the mother ship, in increments of 10m. A target recognition database for the mother ship is then established using these images.

[0085] In yet another exemplary embodiment, after establishing the mother ship target identification database, the configuration of the mother ship target identification database further includes:

[0086] Contourlet transform is performed on the images in the mother ship target recognition database to obtain multi-scale, multi-directional low-pass subbands and bandpass subbands;

[0087] A linear transformation of the minimum and maximum values ​​is applied to the low-pass subband to improve image contrast;

[0088] For the bandpass sub-generation, noise suppression is performed based on the statistical characteristics of image noise. Then, high-frequency information is enhanced through a blur enhancement algorithm to highlight target edges and suppress background and noise.

[0089] The data processed from the low-pass and band-pass subbands are then subjected to inverse Contourlet transform to obtain denoised and enhanced infrared target images, thus updating the mother ship target identification database.

[0090] In yet another exemplary embodiment, the configuration of the target detection algorithm for the unmanned surface vessel (USV) to conduct target detection on the mother ship based on the mother ship target recognition database includes:

[0091] Based on the updated mother ship target recognition database, and on the basis of the invariant moment theory with rotation, translation and scaling invariance, combined with the illumination fuzziness invariance theory, as well as the geometric degradation and grayscale degradation mechanism, and the geometric radiative transformation theory, a recognition moment with illumination, fuzziness and affine invariance is constructed.

[0092] A target detection algorithm is constructed based on recognition moments that are invariant to illumination, blur, and affine transformation.

[0093] In yet another exemplary embodiment, the configuration of the target detection algorithm for the unmanned surface vessel to conduct target detection on the mother ship based on the mother ship target recognition database further includes:

[0094] Sampling is performed on the updated mothership target identification database using the queen template sampling algorithm.

[0095] Based on the sampling results, an algorithm is configured based on the grayscale of the feature points corresponding to the reflective near-infrared cooperative target and the optical flow characteristics of the reflective near-infrared cooperative target as it sways with the mother ship.

[0096] In yet another exemplary embodiment, the configuration of the target detection algorithm for the unmanned surface vessel to conduct target detection on the mother ship based on the mother ship target recognition database further includes:

[0097] Based on the known layout patterns of reflective near-infrared cooperative targets, an automatic adjustment algorithm for target point tracking boxes with occlusion adaptability is configured.

[0098] In yet another exemplary embodiment, the configuration of the target detection algorithm involved in the unmanned surface vessel's (USV) process of determining its position and attitude relative to the mother ship includes:

[0099] Based on the camera imaging model and PnP pose calculation principle, combined with the constraints of the mother ship docking site, the number of cooperative target points and the distribution position of cooperative targets are used as optimization variables, and the minimum error between the position and attitude calculated by the PnP algorithm and the actual position and attitude is used as the optimization index. The distribution position of reflective near-infrared cooperative targets at the mother ship docking site is obtained by optimizing through genetic algorithm.

[0100] By exploring the linear and nonlinear constraints in the imaging process, and using a parameterized imaging model, with the minimum deviation between the image coordinates of the cooperative target imaging and the image coordinates obtained by reprojection through the pose measurement model as the criterion, a cost function for nonlinearly solving the pose from a global optimization perspective is established. Based on non-iterative optimization theory algorithms, a position and attitude measurement algorithm with high stability, high accuracy, and high real-time performance is obtained.

[0101] In yet another exemplary embodiment, the configuration work for the different target detection algorithms involved throughout the process includes:

[0102] In the semi-physical simulation experiment, the simulated components of the mother ship docking part were set on the ground at a certain distance from the three-axis turntable. An active infrared vision system was installed on the three-axis turntable. The three-axis turntable simulated sea conditions and shook the active infrared vision system on the simulated components of the unmanned surface vessel. The position between the mother ship docking part and the simulated components of the unmanned surface vessel was obtained by measuring with a laser total station. The attitude parameters were obtained by measuring the parameters of the three-axis turntable itself. Rain and fog weather were achieved by spraying water and fog with a spray machine.

[0103] Based on the experimental results of the semi-physical simulation experiment, we will continue to optimize the different target detection algorithms involved in the entire process.

[0104] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the unmanned surface vessel visual guidance method described above can be referred to the description of the unmanned surface vessel visual guidance method in the above embodiments, and will not be repeated here.

[0105] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0106] To this end, this application provides a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute the steps of the unmanned surface vessel visual guidance method in the above embodiments. For specific operations, please refer to the description of the unmanned surface vessel visual guidance method in the above embodiments, which will not be repeated here.

[0107] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0108] Since the instructions stored in the computer-readable storage medium can execute the steps of the unmanned surface vessel visual guidance method in the above embodiments, the beneficial effects that the unmanned surface vessel visual guidance method in the above embodiments can achieve can be realized, as detailed in the preceding description, and will not be repeated here.

[0109] The above provides a detailed description of the unmanned surface vessel visual guidance system, method, and computer-readable storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A visual guidance system for unmanned surface vessels, characterized in that, The unmanned surface vessel (USV) visual guidance system includes an USV and a mothership. The USV is equipped with a passive infrared vision system and an active infrared vision system. The mothership is equipped with a reflective near-infrared cooperative target at a preset docking point. When the unmanned surface vessel is more than 30m away from the mother ship, in passive working mode, it emits mid-far infrared light in the 8-12um band to the mother ship through the passive infrared vision system to detect the mother ship and guide itself to approach the mother ship based on the target detection results. When the unmanned surface vessel (USV) is within 30 meters of the mother ship, in active operating mode, it illuminates the docking area of ​​the mother ship with 900nm near-infrared light through the active vision system, causing the reflective near-infrared cooperative target to emit high-brightness infrared light, thus obtaining a corresponding high-brightness near-infrared cooperative target image. Then, by combining the high-brightness near-infrared cooperative target image and the geometric information of the reflective near-infrared cooperative target, it calculates its own position and attitude relative to the mother ship. Based on the position and attitude, it guides itself to approach the mother ship to complete the berthing and recovery task.

2. The unmanned surface vessel visual guidance system according to claim 1, characterized in that, The target detection algorithm for the unmanned surface vessel (USV) to detect targets on the mother ship is configured based on the mother ship target recognition database. The configuration of the mother ship target recognition database includes: Based on the installation height of the passive infrared imaging system on the unmanned surface vessel (USV), images of the USV swaying the mother ship from different angles as it approaches the mother ship are acquired by the passive infrared vision system within a range of 200-30m from the USV, in increments of 10m. A target recognition database for the mother ship is then established using these images of the USV swaying the mother ship from different angles.

3. The unmanned surface vessel visual guidance system according to claim 2, characterized in that, After establishing the mother ship target identification database, the configuration of the mother ship target identification database also includes: Contourlet transform is performed on the images in the mother ship target recognition database to obtain multi-scale, multi-directional low-pass subbands and bandpass subbands; A linear transformation of the minimum and maximum values ​​is applied to the low-pass subband to improve image contrast; For the bandpass sub-band, noise suppression is performed based on the statistical characteristics of image noise, and then high-frequency information is enhanced through a blur enhancement algorithm to highlight the target edge and suppress the background and its noise. The data processed by the low-pass subband and the band-pass subband are subjected to inverse Contourlet transform to obtain a denoised and enhanced infrared target image, thereby updating the mother ship target identification database.

4. The unmanned surface vessel visual guidance system according to claim 3, characterized in that, The configuration of the target detection algorithm for the unmanned surface vessel (USV) to detect targets on the mother ship based on the mother ship target recognition database includes: Based on the updated mother ship target recognition database, and on the basis of the invariant moment theory with rotation, translation and scaling invariance, combined with the illumination fuzziness invariance theory, as well as the geometric degradation and grayscale degradation mechanism, and the geometric radiative transformation theory, a recognition moment with illumination, fuzziness and affine invariance is constructed. The target detection algorithm is constructed based on the recognition moments that have illumination, blur and affine invariance.

5. The unmanned surface vessel visual guidance system according to claim 4, characterized in that, The configuration of the target detection algorithm for the unmanned surface vessel (USV) to detect targets on the mother ship based on the mother ship target recognition database also includes: For the updated mothership target identification database, sampling is performed based on the queen template sampling algorithm; Based on the sampling results, an algorithm is configured based on the grayscale of the feature points corresponding to the reflective near-infrared cooperative target and the optical flow characteristics of the reflective near-infrared cooperative target as the mother ship sways.

6. The unmanned surface vessel visual guidance system according to claim 5, characterized in that, The configuration of the target detection algorithm for the unmanned surface vessel (USV) to detect targets on the mother ship based on the mother ship target recognition database also includes: Based on the known layout patterns of the reflective near-infrared cooperative targets, an automatic adjustment algorithm for the target point tracking box with occlusion adaptability is configured.

7. The unmanned surface vessel visual guidance system according to claim 1, characterized in that, The target detection algorithm involved in the process of the unmanned surface vessel (USV) determining its position and attitude relative to the mother ship includes the following configuration steps: Based on the camera imaging model and PnP pose calculation principle, combined with the constraints of the mother ship docking site, the number of cooperative target points and the distribution position of cooperative targets are used as optimization variables, and the minimum error between the position and attitude calculated by the PnP algorithm and the actual position and attitude is used as the optimization index. The distribution position of the reflective near-infrared cooperative targets at the mother ship docking site is obtained by optimizing through a genetic algorithm. By exploring the linear and nonlinear constraints in the imaging process, and using a parameterized imaging model, with the minimum deviation between the image coordinates of the cooperative target imaging and the image coordinates obtained by reprojection through the pose measurement model as the criterion, a cost function for nonlinearly solving the pose from a global optimization perspective is established. Based on non-iterative optimization theory algorithms, a position and attitude measurement algorithm with high stability, high accuracy, and high real-time performance is obtained.

8. The unmanned surface vessel visual guidance system according to claim 1, characterized in that, For the different object detection algorithms involved throughout the process, the corresponding configuration work includes: In the semi-physical simulation experiment, the simulated component of the mother ship docking point is set on the ground at a certain distance from the three-axis turntable. The active infrared vision system is installed on the three-axis turntable. The three-axis turntable simulates sea conditions by shaking the active infrared vision system on the simulated component of the unmanned surface vessel. The position between the mother ship docking point and the simulated component of the unmanned surface vessel is obtained by measuring with a laser total station. The attitude parameters are obtained by measuring the parameters of the three-axis turntable itself. Rain and fog weather are achieved by spraying water and fog with a spray machine. Based on the experimental results of the semi-physical simulation experiment, the different target detection algorithms involved in the entire process will be further optimized.

9. A visual guidance method for unmanned surface vessels, characterized in that, The unmanned surface vessel (USV) visual guidance method is applied to an USV visual guidance system, which includes an USV and a mothership. The USV is equipped with a passive infrared vision system and an active infrared vision system. The mothership has a reflective near-infrared cooperative target positioned at a pre-designated docking point. The USV visual guidance method includes: When the unmanned surface vessel is more than 30m away from the mother ship, in passive working mode, it emits mid-far infrared light in the 8-12um band to the mother ship through the passive infrared vision system to detect the mother ship and guide itself to approach the mother ship based on the target detection results. When the unmanned surface vessel is within 30m of the mother ship, in active working mode, it illuminates the docking part of the mother ship with 900nm near-infrared light through the active vision system, causing the reflective near-infrared cooperative target to emit high-brightness infrared light, thereby obtaining a corresponding high-brightness near-infrared cooperative target image. The unmanned surface vessel combines the high-brightness near-infrared cooperative target image and the geometric information of the reflective near-infrared cooperative target to solve its own position and attitude relative to the mother ship. Based on its position and attitude, the unmanned surface vessel guides itself toward the mother ship to complete the berthing and recovery mission.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to execute the method of claim 9.

Images