Visual guidance system and method for unmanned ship

Abstract

The invention provides a visual guidance system and method for an unmanned ship, which are used for creating a set of visual guidance system for the unmanned ship, not only can realize high-precision target detection on a mother ship swinging on the sea surface, but also can obtain the accurate position posture of the unmanned ship relative to the mother ship, and can improve the accuracy of the unmanned ship. Therefore, real-time and accurate automatic guidance can be provided for the recovery work of the unmanned ship, powerful data support can be provided for guiding the unmanned ship to safely berth to a mother ship, track navigation and successful recovery under all-weather conditions and complex sea conditions, and the requirement for high-quality unmanned ship recovery work is met.

Description

Technical Field

[0001] The present application relates to the field of unmanned boats, and in particular to a visual guidance system and method for an unmanned boat. Background Art

[0002] An unmanned boat operating on the water surface is a type of automatically navigable boat. In actual applications, it can be equipped with various functional modules according to mission requirements, and can complete a series of tasks autonomously or semi-autonomously, thus having great application value.

[0003] However, the inventor of the present application has discovered that the current recovery of unmanned boats can only be performed by manual operation at low speeds and low sea conditions. Obviously, there is a problem of low recovery efficiency, and because it involves manual operation, it is also accompanied by certain risks. Summary of the invention

[0004] The present application provides an unmanned boat vision guidance system and method, which are used to create an unmanned boat vision guidance system that can not only achieve high-precision target detection of a mother ship swaying on the sea surface, but also obtain the precise position and posture of the unmanned boat relative to the mother ship. This can provide real-time and precise automated guidance for the recovery of the unmanned boat, ensuring that in all-weather conditions and complex sea conditions, strong data support can be provided for guiding the unmanned boat to safely berth to the mother ship, follow the track and successfully recover, thereby meeting the needs of high-quality unmanned boat recovery work.

[0005] In a first aspect, the present application provides an unmanned boat visual guidance system, the unmanned boat visual guidance system comprising an unmanned boat and a mother ship, the unmanned boat is configured with a passive infrared vision system and an active infrared vision system, and the mother ship is configured with a reflective near-infrared cooperative target at a preset mother ship docking position;

[0006] When the unmanned boat is 30m away from the mother ship, in passive working mode, it emits 8-12um mid- and far-infrared light to the mother ship through the passive infrared vision system to detect the mother ship, and guides itself to approach the mother ship based on the target detection results;

[0007] When the unmanned boat is within 30m of the mother ship, it irradiates the docking part of the mother ship with near-infrared light in the 900nm band through the active vision system in the active working mode, so that the reflective near-infrared cooperation target emits high-brightness infrared light and obtains the corresponding high-brightness near-infrared cooperation target image; then, the unmanned boat combines the high-brightness near-infrared cooperation target image and the geometric information of the reflective near-infrared cooperation target to solve its own position and attitude relative to the mother ship, and then guides itself to approach the mother ship based on the position and attitude to complete the berthing and recovery mission.

[0008] In a second aspect, the present application provides an unmanned boat visual guidance method, which is applied to an unmanned boat visual guidance system. The unmanned boat visual guidance system includes an unmanned boat and a mother ship. The unmanned boat is equipped with a passive infrared vision system and an active infrared vision system. The mother ship is equipped with a reflective near-infrared cooperative target at a preset mother ship docking position. The unmanned boat visual guidance method includes:

[0009] When the unmanned boat is 30m away from the mother ship, in the passive working mode, it emits 8-12um mid- and far-infrared light to the mother ship through the passive infrared vision system to detect the mother ship, and guides itself to approach the mother ship based on the target detection results;

[0010] When the unmanned boat is within 30m of the mother ship, in the active working mode, the active vision system irradiates the docking part of the mother ship with near-infrared light of 900nm band, so that the reflective near-infrared cooperation target emits high-brightness infrared light, and obtains the corresponding high-brightness near-infrared cooperation target image;

[0011] The unmanned boat combines the high-brightness near-infrared cooperative target image and the geometric information of the reflective near-infrared cooperative target to solve its position and posture relative to the mother ship;

[0012] Based on its position and posture, the unmanned boat guides itself towards the mother ship to complete the berthing and recovery mission.

[0013] In a third aspect, the present application provides a computer-readable storage medium, which stores a plurality of instructions suitable for loading by a processor to execute the method provided in the second aspect of the present application.

[0014] From the above content, it can be concluded that the present application has the following beneficial effects:

[0015] Aiming at the goal of unmanned boat recovery, this application has created an unmanned boat vision guidance system, which can not only achieve high-precision target detection of the mother ship swaying on the sea, but also obtain the precise position and posture of the unmanned boat relative to the mother ship. This can provide real-time and accurate automated guidance for the recovery of the unmanned boat, ensuring that in all-weather conditions and complex sea conditions, it can provide strong data support for guiding the unmanned boat to safely berth to the mother ship, follow the track and successfully recover, thereby meeting the needs of high-quality unmanned boat recovery work. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 A schematic diagram of the system structure of the unmanned boat vision guidance system of this application;

[0018] Figure 2 A schematic diagram of a scenario of the unmanned boat visual guidance system of this application;

[0019] Figure 3 A schematic diagram of a scenario for denoising and enhancing infrared target images in this application;

[0020] Figure 4 A schematic diagram of a scenario for constructing invariant moments for this application;

[0021] Figure 5 A schematic diagram of a scenario of reprojection error in this application;

[0022] Figure 6 A schematic diagram of a scenario for position and attitude measurement of a reflective near-infrared cooperative target in this application;

[0023] Figure 7 A schematic diagram of a scenario for a semi-physical simulation experiment of this application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0025] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described here can be implemented in an order other than that illustrated or described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The naming or numbering of steps in this application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The process steps that have been named or numbered can change the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0026] The division of modules in this application is a logical division. There may be other division methods when it is implemented in actual applications. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection between modules can be electrical or other similar forms, which are not limited in this application. In addition, the modules or submodules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed in multiple circuit modules, and some or all of the modules may be selected according to actual needs to achieve the purpose of the present application.

[0027] See also Figure 1 A system structure schematic diagram of the unmanned boat vision guidance system of the present application is shown. The unmanned boat vision guidance system provided by the present application mainly includes two parts, the unmanned boat and the mother ship. For the former, i.e. the unmanned boat, it is equipped with a passive infrared vision system and an active infrared vision system specially designed for the present application. For the latter, i.e. the mother ship, it is equipped with a reflective near-infrared cooperative target at the preset mother ship docking position (mother ship recovery cabin). It can be understood that the design of these component structures is to serve the recovery work of the unmanned boat. Next, from the dynamic workflow shown below, we can better understand the processing content that may be involved when the unmanned boat vision guidance system of the present application performs the automatic recovery of the unmanned boat.

[0028] (1) When the unmanned boat is 30 meters away from the mother ship, in passive working mode, it emits 8-12um mid- and far-infrared light to the mother ship through the passive infrared vision system to detect the mother ship and guide itself to approach the mother ship based on the target detection results;

[0029] It should be noted that in this link, the unmanned boat does not necessarily need to know the specific distance between itself and the mother ship. In actual situations, the deployment location / working position of the unmanned boat is usually not too far from the mother ship, kept within 200m, and, when recovery is required, it will also be 30m away.

[0030] In this case, the passive infrared vision system specially configured in the present application can be used first. When the unmanned boat recovery work is triggered, it will keep emitting mid-to-far infrared light in the 8-12um band to the mother ship in the passive working mode. When the mid-to-far infrared light is emitted, it is not necessarily emitted directly to the mother ship. It will identify, locate and track the mother ship based on target detection based on the mid-to-far infrared light. At the same time, the direction of the emitted mid-to-far infrared light can also be adjusted. Of course, the emission direction of the mid-to-far infrared light can also be fixed, which can be adjusted according to the configuration of the passive infrared vision system.

[0031] In this way, the passive infrared vision system can be used to identify the direction of the mother ship at a long distance, and the unmanned boat can approach / get closer to the mother ship through its own propulsion system until the distance between itself and the mother ship is within 30m, triggering the active working mode to achieve more precise and delicate approach control to meet the needs of unmanned boat recovery work.

[0032] It is understandable that since the passive infrared vision system uses the mid- and far-infrared band (8-12um), it has strong penetration capabilities in all-weather conditions and complex sea conditions, including rain, fog, smoke, etc., so the all-weather detection capability can bring very good adaptability, and can ensure stable, efficient and precise processing effects in the identification, positioning and tracking of mother ships at medium and long distances.

[0033] (2) When the unmanned boat is within 30m of the mother ship, it irradiates the docking part of the mother ship with near-infrared light in the 900nm band through the active vision system in the active working mode, so that the reflective near-infrared cooperative target emits high-brightness infrared light and obtains the corresponding high-brightness near-infrared cooperative target image; then, the unmanned boat combines the high-brightness near-infrared cooperative target image and the geometric information of the reflective near-infrared cooperative target to solve its own position and attitude relative to the mother ship, and then guides itself to approach the mother ship based on the position and attitude to complete the berthing and recovery task.

[0034] It can be understood that as the unmanned boat approaches, the distance between the unmanned boat and the mother ship becomes closer and closer until 30m (the distance can be measured by target detection in the previous passive working mode, or by other ranging means). At this time, the active working mode can be triggered, and the active vision system actively emits / irradiates 300nm band near-infrared light to the mother ship docking position pre-set by the mother ship, so that the reflective near-infrared cooperation target configured at the mother ship docking position emits / responds to high-brightness infrared light (relative to the background), so that a high-brightness near-infrared cooperation target image is obtained, avoiding special denoising processing. At this time, the imaging geometry information of the predetermined reflective near-infrared cooperation target can be combined to accurately solve the position and attitude of the unmanned boat relative to the mother ship docking position (including position and attitude information) in real time, providing data support for guiding the unmanned boat to complete the berthing and recovery task to the mother ship.

[0035] For this process, you can also combine Figure 2 A scene schematic diagram of the unmanned boat visual guidance system of the present application is shown for a more vivid understanding.

[0036] It is understandable that since the docking part of the mother ship adopts a near-infrared reflective cooperative target design, the mother ship itself does not emit infrared light, and there is no need for communication between the mother ship and the unmanned boat. Therefore, this can also increase the stealth performance of the mother ship. At the same time, in order to avoid the interference of the infrared heat source on the mother ship to the cooperative target, a reflective near-infrared cooperative target design is adopted. Therefore, the imaging performance and detection reliability of the cooperative target can be greatly improved, and no special denoising processing is required. Therefore, the adaptability to different complex situations is further strengthened, and a more intelligent unmanned boat autonomous recovery effect is achieved.

[0037] So, for Figure 1 From the scheme content of the illustrated embodiment, it can be seen that, in view of the unmanned boat recovery goal, the present application has created a set of unmanned boat visual guidance system, which can not only realize high-precision target detection of the mother ship swaying on the sea surface, but also obtain the precise position and posture of the unmanned boat relative to the mother ship. This can provide real-time and accurate automated guidance for the recovery of the unmanned boat, ensure that in all-weather conditions and complex sea conditions, it can provide strong data support for guiding the unmanned boat to safely berth to the mother ship, track navigation and successfully recover, thereby meeting the needs of high-quality unmanned boat recovery work.

[0038] Furthermore, in specific applications, it is understandable that during the process of the unmanned boat berthing to the mother ship, due to the combined effects of the long distance, the swaying and movement of the mother ship target, the different imaging angles of the camera, weather conditions and the infrared imaging system itself, the infrared target image of the mother ship detected by the passive infrared vision system on the unmanned boat has problems such as low clarity, low contrast, blurred target edges, non-uniformity and high noise. These problems may cause the infrared image to have poor resolution for the shape and edge texture of small targets, which brings great interference to the subsequent target detection and tracking, and thus affects the completion of the unmanned boat's real-time and precise guidance mission.

[0039] Therefore, this application has also conducted a series of research on the processing accuracy of the passive infrared vision system to further improve its solution performance and application value.

[0040] Specifically, as an exemplary embodiment, the target detection algorithm for the unmanned boat to detect the mother ship is configured based on the mother ship target recognition database (which can also be understood by the sample image set of the mother ship), and the configuration work of the mother ship target recognition database can specifically include:

[0041] Taking the installation height of the passive infrared imaging system on the unmanned boat as a reference, the passive infrared vision system is used to obtain images of the mother ship shaking on the water surface at different perspectives when the unmanned boat is far or near, with an increment of 10m, within the range of 200-30m from the mother ship. A mother ship target recognition database is established based on the images of the mother ship shaking on the water surface at different perspectives.

[0042] It can be understood that the embodiment here provides a specific implementation plan for how to collect sample images of the mother ship in different complex situations for use in building a mother ship target recognition database.

[0043] Further, combined with Figure 3 A schematic diagram of a scenario of infrared target image denoising and enhancement of the present application is shown. As an exemplary embodiment, after establishing a mother ship target recognition database, the configuration work of the mother ship target recognition database of the present application may also include:

[0044] Perform Contourlet transform on the images of the mother ship target recognition database to obtain multi-scale and multi-directional low-pass sub-bands and band-pass sub-bands;

[0045] Perform linear transformation of the maximum and minimum values ​​of the low-pass subband to improve the image contrast;

[0046] For the bandpass offspring, noise suppression is performed based on the statistical characteristics of image noise, and then the high-frequency information is enhanced through the fuzzy enhancement algorithm to highlight the edge of the target and suppress the background and its noise;

[0047] The data processed by the low-pass sub-band and the band-pass sub-band are subjected to the inverse Contourlet transform to obtain the denoised and enhanced infrared target image, thus completing the update of the mother ship target recognition database.

[0048] It can be seen that in view of the characteristics of infrared images (mother ship target recognition database) such as large target noise interference, blurred target edges, and low contrast, this application analyzes the statistical distribution characteristics between infrared image pixels and their noise based on the imaging mechanism of infrared images, and combines the external basic shape characteristics of the target image to establish corresponding image denoising and enhancement evaluation indicators, and then studies the denoising and enhancement methods of sea surface infrared target images. Specifically, based on the mother ship target recognition database constructed earlier, the infrared image denoising and enhancement algorithm based on Contourlet transform and fuzzy theory is studied, and the above-mentioned very specific implementation scheme is given, so that the infrared image (mother ship target recognition database) can be efficiently and effectively denoised and enhanced, which is helpful to configure a more accurate and efficient corresponding target detection algorithm.

[0049] Furthermore, also for the target detection algorithm based on the passive infrared vision system, this application also studies the degradation law of the real mother ship target image caused by the long distance, movement and swaying of the mother ship target, different camera imaging angles, uneven thermal imaging, etc., and on this basis, combined with image geometric transformation, grayscale degradation (image blur, luminosity transformation and contrast transformation) and invariant moment theory, research and establish a highly reliable target detection algorithm for the swaying mother ship on the sea surface, and continue to carry out verification experiments. The stability and real-time performance of the algorithm are improved through continuous feedback of data in the verification experiments.

[0050] In this regard, as an exemplary embodiment, refer to Figure 4 The schematic diagram of a scenario for constructing an invariant moment in the present application is shown. The configuration work of the target detection algorithm for configuring an unmanned boat to detect a target on a mother ship based on a mother ship target recognition database in the present application may include:

[0051] Based on the updated mother ship target recognition database, on the basis of the invariant moment theory with rotation, translation and scaling invariance, combined with the illumination blur invariance theory, as well as the geometric degradation and grayscale degradation mechanism, and combined with the geometric radiation transformation theory, the recognition moment with illumination, blur and affine invariance is constructed;

[0052] An object detection algorithm is constructed based on the recognition moments which are invariant to illumination, blur and affine.

[0053] It can be understood that for the embodiments herein, the present application aims at the problem of real mother ship target image degradation caused by the combined effects of long distance, different viewing angles, rainy and foggy weather, infrared thermal blooming and lens distortion, and proposes a fuzzy recognition moment with illumination, blur and affine invariance to improve the reliability of target recognition in the mother ship infrared image.

[0054] In addition, as an exemplary embodiment, the configuration work of the target detection algorithm for configuring the unmanned boat to detect the target of the mother ship based on the mother ship target recognition database in the present application may also include:

[0055] For the updated mother ship target recognition database, sampling is performed based on the queen template sampling algorithm;

[0056] On the basis of the sampling results, the algorithm is configured based on the grayscale of the feature points corresponding to the reflective near-infrared cooperative target and the optical flow characteristics of the reflective near-infrared cooperative target as the mother ship sways.

[0057] It can be understood that the queen template sampling algorithm / chess queen template rule can meet the needs of fast interpretation of images / videos based on queen templates, so as to configure the target detection algorithm more efficiently.

[0058] In addition, at a distance of 30 meters, it can be seen that the active infrared imaging of the reflective near-infrared cooperative target at the docking site of the mother ship is already relatively clear, so the cooperative target point can be directly extracted based on the higher grayscale of the cooperation point. In order to enhance reliability, with the help of the characteristic that the cooperation point sways with the mother ship, according to its optical flow characteristics, this application is based on a near-infrared cooperative target detection method that combines the grayscale of feature points and optical flow characteristics.

[0059] In addition, the present application also considers the situation that the sea surface wind and waves may cause the waves to block some cooperative target points. In this regard, as an exemplary embodiment, the configuration work of the target detection algorithm for configuring the unmanned boat to detect the mother ship based on the mother ship target recognition database may also include:

[0060] According to the known layout rules of reflective near-infrared cooperative targets, an automatic adjustment algorithm for target point tracking frame with occlusion adaptability is configured.

[0061] In this way, the target point tracking frame automatic adjustment algorithm is used to predict the characteristics of the cooperative target behind the occlusion, so as to further improve the tracking reliability of the cooperative target.

[0062] Furthermore, focusing on the active infrared vision system, it can be understood that high-precision measurement of the relative position and posture of the mother ship's docking site and the unmanned boat is the key to the success of the docking. The number and layout of cooperative targets arranged at the mother ship's docking site have an important impact on the accuracy and stability of position and posture measurement. Usually, the space at the docking site on the ship is limited, and arbitrary and excessive arrangement of infrared reflective cooperative targets is not allowed.

[0063] In this case, as an exemplary embodiment, the target detection algorithm involved in the process of the unmanned boat solving its own position and posture relative to the mother ship, the corresponding configuration work may include:

[0064] (1) Based on the camera imaging model and the PnP pose solution principle (EPnP algorithm), combined with the constraints of the mother ship docking position, the number of cooperative target points and the distribution position of the cooperative target are used as optimization variables, and the minimum error between the position and pose calculated by the PnP algorithm and the actual position and pose is used as the optimization index. The distribution position of the reflective near-infrared cooperative target at the mother ship docking position is obtained by genetic algorithm optimization;

[0065] Among them, the specific constraints are: requiring the cooperation targets to be distributed along the border of the mother ship's docking position (recovery cabin) as much as possible, so that the distance between the cooperation points is far, which can further improve the docking posture measurement accuracy.

[0066] (2) The linear and nonlinear constraints in the imaging process are explored. The parameterized imaging model is used to minimize the deviation between the image coordinates of the cooperative target and the image coordinates reprojected by the pose measurement model. A cost function for solving the pose nonlinearly from a global optimization perspective is established. Based on the non-iterative optimization theory algorithm, a position and pose measurement algorithm with high stability, high precision and high real-time performance is solved.

[0067] Among them, for the image coordinate deviation involved here, you can also combine Figure 5 A scene schematic diagram of the reprojection error of the present application is shown for a more vivid understanding.

[0068] For this processing, you can also combine Figure 6 A schematic diagram of a scenario of position and posture measurement of a reflective near-infrared cooperative target is shown for a more vivid understanding of the present application.

[0069] The specific nonlinear constraints include: the orthogonality of the rotation matrix during the imaging process, and the property that the translation matrix can be linearly represented.

[0070] The non-iterative optimization theory algorithm may specifically adopt algorithms such as the Grobner basis theory.

[0071] The so-called high stability, high precision and high real-time performance specifically include: the measurement accuracy reaches 0.1m within 30m, a single operation on an ordinary industrial computer is within 1ms, and stable measurement can be performed when the deviation angle between the docking point of the unmanned boat and the mother ship is relatively large.

[0072] It can be understood that for the embodiments herein, the present application optimizes the number of cooperative target points and the layout optimization method according to the constraints of the mother ship docking part and the PNP posture measurement theory to improve the posture measurement accuracy; and, considering that the accuracy and stability of the existing simple linear posture solution method need to be improved, and the posture solution method using the iterative method is prone to fall into the local optimal problem, the present application combines the characteristics of the close-range mother ship docking part, according to the physical and mathematical model of the imaging system, combined with the principle of the position and attitude solution method, analyzes the constraint relationship in the imaging process, studies the optimization problem of the number of cooperative target points and the spatial layout of the reflective near-infrared, and proposes to establish a new method for solving the posture with high precision in a global sense through the imaging physical and mathematical model and non-iterative optimization theory, so as to further improve the posture measurement accuracy and reliability.

[0073] In addition, for the series of target detection algorithms involved above, or for the different target detection algorithms involved in the entire process, in practical applications, this application can also be combined with semi-physical simulation experiments for verification and optimization to further ensure its algorithm performance.

[0074] For this, see Figure 7A schematic diagram of a scenario of a semi-physical simulation experiment of the present application is shown. As an exemplary embodiment, for different target detection algorithms involved in the whole process, the corresponding configuration work may include:

[0075] In the semi-physical simulation experiment, the simulated parts of the mother ship docking position are set on the ground at a certain distance from the three-axis turntable, and the active infrared vision system is installed on the three-axis turntable. The three-axis turntable simulates the sea conditions to shake the active infrared vision system on the simulated parts of the unmanned boat. The position between the mother ship docking position and the simulated parts of the unmanned boat is measured by a laser total station, and the attitude parameters are obtained by measuring the parameters of the three-axis turntable itself. The rain and fog climate is achieved by spraying water and fog with a spray machine.

[0076] Based on the experimental results of semi-physical simulation experiments, we continue to optimize the different target detection algorithms involved in the whole process.

[0077] In this way, the above settings are used to promote the semi-physical simulation experiment involved in this application, verify the accuracy, stability and other indicators of the relevant target detection algorithm, and further improve the algorithm content according to the problems that arise.

[0078] The above is an introduction to the unmanned boat visual guidance system provided by the present application. On this basis, the present application also provides an unmanned boat visual guidance method from the perspective of the system workflow. The unmanned boat visual guidance method is applied to the unmanned boat visual guidance system. The unmanned boat visual guidance system includes an unmanned boat and a mother ship. The unmanned boat is equipped with a passive infrared vision system and an active infrared vision system. The mother ship is equipped with a reflective near-infrared cooperative target at a preset mother ship docking position. The unmanned boat visual guidance method includes the following steps:

[0079] (1) When the unmanned boat is 30 meters away from the mother ship, in the passive working mode, it emits 8-12um mid- and far-infrared light to the mother ship through the passive infrared vision system to detect the mother ship and guide itself to approach the mother ship based on the target detection results;

[0080] (2) When the unmanned boat is within 30m of the mother ship, the active vision system irradiates the docking position of the mother ship with near-infrared light of 900nm band in the active working mode, so that the reflective near-infrared cooperative target emits high-brightness infrared light, and obtains the corresponding high-brightness near-infrared cooperative target image;

[0081] (3) The unmanned boat combines the high-brightness near-infrared cooperative target image and the geometric information of the reflective near-infrared cooperative target to solve its own position and posture relative to the mother ship;

[0082] (4) Based on its position and posture, the unmanned boat guides itself toward the mother ship to complete the berthing and recovery mission.

[0083] In an exemplary embodiment, the target detection algorithm for the unmanned boat to detect the target of the mother ship is configured based on the mother ship target recognition database. The configuration work of the mother ship target recognition database includes:

[0084] Taking the installation height of the passive infrared imaging system on the unmanned boat as a reference, the passive infrared vision system is used to obtain images of the mother ship shaking on the water surface at different perspectives when the unmanned boat is far or near, with an increment of 10m, within the range of 200-30m from the mother ship. A mother ship target recognition database is established based on the images of the mother ship shaking on the water surface at different perspectives.

[0085] In another exemplary embodiment, after establishing the mother ship target identification database, the configuration work of the mother ship target identification database further includes:

[0086] Perform Contourlet transform on the images of the mother ship target recognition database to obtain multi-scale and multi-directional low-pass sub-bands and band-pass sub-bands;

[0087] Perform linear transformation of the maximum and minimum values ​​of the low-pass subband to improve the image contrast;

[0088] For the bandpass offspring, noise suppression is performed based on the statistical characteristics of image noise, and then the high-frequency information is enhanced through the fuzzy enhancement algorithm to highlight the edge of the target and suppress the background and its noise;

[0089] The data processed by low-pass sub-band and band-pass sub-band are used to obtain the denoised and enhanced infrared target image through inverse Contourlet transform, thus completing the update of the mother ship target recognition database.

[0090] In another exemplary embodiment, the configuration work of configuring the target detection algorithm for configuring the unmanned boat to detect the target of the mother ship based on the mother ship target recognition database includes:

[0091] Based on the updated mother ship target recognition database, on the basis of the invariant moment theory with rotation, translation and scaling invariance, combined with the illumination blur invariance theory, as well as the geometric degradation and grayscale degradation mechanism, and combined with the geometric radiation transformation theory, the recognition moment with illumination, blur and affine invariance is constructed;

[0092] An object detection algorithm is constructed based on the recognition moments which are invariant to illumination, blur and affine.

[0093] In another exemplary embodiment, the configuration work of configuring the target detection algorithm for configuring the unmanned boat to detect the target of the mother ship based on the mother ship target recognition database also includes:

[0094] For the updated mother ship target recognition database, sampling is performed based on the queen template sampling algorithm;

[0095] On the basis of the sampling results, the algorithm is configured based on the grayscale of the feature points corresponding to the reflective near-infrared cooperative target and the optical flow characteristics of the reflective near-infrared cooperative target as the mother ship sways.

[0096] In another exemplary embodiment, the configuration work of configuring the target detection algorithm for configuring the unmanned boat to detect the target of the mother ship based on the mother ship target recognition database also includes:

[0097] According to the known layout rules of reflective near-infrared cooperative targets, an automatic adjustment algorithm for target point tracking frame with occlusion adaptability is configured.

[0098] In another exemplary embodiment, the target detection algorithm involved in the process of the unmanned boat solving its own position and posture relative to the mother ship, and the corresponding configuration work includes:

[0099] Based on the camera imaging model and the PnP pose solving principle, combined with the constraints of the mother ship docking position, the number of cooperative target points and the distribution position of the cooperative target are used as optimization variables, and the minimum error between the position and pose solved by the PnP algorithm and the true position and pose is used as the optimization index. The distribution position of the reflective near-infrared cooperative target at the mother ship docking position is obtained through genetic algorithm optimization.

[0100] The linear and nonlinear constraints in the imaging process are explored, and the parameterized imaging model is used. The minimum deviation between the image coordinates of the cooperative target imaging and the image coordinates obtained by reprojecting the pose measurement model is used as the indicator. A cost function for nonlinearly solving the pose from a global optimization perspective is established. Based on the non-iterative optimization theoretical algorithm, a position and pose measurement algorithm with high stability, high precision and high real-time performance is solved.

[0101] In another exemplary embodiment, for different target detection algorithms involved in the whole process, the corresponding configuration work includes:

[0102] In the semi-physical simulation experiment, the simulated parts of the mother ship docking position are set on the ground at a certain distance from the three-axis turntable, and the active infrared vision system is installed on the three-axis turntable. The three-axis turntable simulates the sea conditions to shake the active infrared vision system on the simulated parts of the unmanned boat. The position between the mother ship docking position and the simulated parts of the unmanned boat is measured by a laser total station, and the attitude parameters are obtained by measuring the parameters of the three-axis turntable itself. The rain and fog climate is achieved by spraying water and fog with a spray machine.

[0103] Based on the experimental results of semi-physical simulation experiments, we continue to optimize the different target detection algorithms involved in the whole process.

[0104] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the unmanned boat visual guidance method described above can refer to the description of the unmanned boat visual guidance method in the above embodiment, and will not be repeated here.

[0105] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0106] To this end, the present application provides a computer-readable storage medium storing a plurality of instructions, which can be loaded by a processor to execute the steps of the unmanned boat visual guidance method in the above embodiment. For specific operations, please refer to the description of the unmanned boat visual guidance method in the above embodiment, which will not be repeated here.

[0107] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0108] Since the instructions stored in the computer-readable storage medium can execute the steps of the unmanned boat visual guidance method in the above embodiment, the beneficial effects that can be achieved by the unmanned boat visual guidance method in the above embodiment can be achieved. Please refer to the previous description for details, which will not be repeated here.

[0109] The unmanned boat visual guidance system, method and computer-readable storage medium provided by the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An unmanned boat visual guidance system, characterized in that: The unmanned boat visual guidance system comprises an unmanned boat and a mother ship, the unmanned boat is equipped with a passive infrared vision system and an active infrared vision system, and the mother ship is equipped with a reflective near-infrared cooperative target at a preset mother ship docking position; When the unmanned boat is 30 meters away from the mother ship, in a passive working mode, the unmanned boat emits mid-to-far infrared light in the 8-12um band to the mother ship through the passive infrared vision system to detect the mother ship, and guides itself to approach the mother ship based on the target detection result; When the unmanned boat is within 30m of the mother ship, the unmanned boat irradiates the docking part of the mother ship with near-infrared light in the 900nm band through the active vision system in the active working mode, so that the reflective near-infrared cooperation target emits high-brightness infrared light, and obtains a corresponding high-brightness near-infrared cooperation target image; then, the unmanned boat combines the high-brightness near-infrared cooperation target image and the geometric information of the reflective near-infrared cooperation target to solve its own position and posture relative to the mother ship, and then, based on the position and posture, guides itself to approach the mother ship to complete the berthing and recovery task.

2. The unmanned boat visual guidance system according to claim 1, characterized in that: The target detection algorithm for the unmanned boat to detect the target of the mother ship is configured based on the mother ship target recognition database. The configuration work of the mother ship target recognition database includes: Taking the installation height of the passive infrared imaging system on the unmanned boat as a reference, from the range of 200-30m from the unmanned boat to the mother ship, with 10m as an increment, the passive infrared vision system is used to obtain images of the water surface shaking the mother ship from far to near at different perspectives, and the mother ship target recognition database is established based on the images of the water surface shaking the mother ship at different perspectives.

3. The unmanned boat visual guidance system according to claim 2, characterized in that: After the mother ship target identification database is established, the configuration work of the mother ship target identification database further includes: Performing Contourlet transform on the image of the mother ship target recognition database to obtain multi-scale and multi-directional low-pass sub-bands and band-pass sub-bands; Performing a linear transformation of the maximum and minimum values ​​on the low-pass sub-band to improve the image contrast; For the bandpass offspring, noise suppression is performed according to the statistical characteristics of image noise, and then high-frequency information is enhanced by a fuzzy enhancement algorithm to highlight the edge of the target and suppress the background and its noise; The data processed by the low-pass sub-band and the band-pass sub-band are subjected to inverse Contourlet transformation to obtain a denoised and enhanced infrared target image, thereby completing the update of the mother ship target recognition database.

4. The unmanned boat visual guidance system according to claim 3, characterized in that: The configuration work of configuring the target detection algorithm for configuring the unmanned boat to detect the target of the mother ship based on the mother ship target recognition database includes: Based on the updated mother ship target recognition database, on the basis of the invariant moment theory with rotation, translation and scaling invariance, combined with the illumination blur invariance theory, combined with the geometric degradation and grayscale degradation mechanism, and combined with the geometric radiation transformation theory, a recognition moment with illumination, blur and affine invariance is constructed; The target detection algorithm is constructed based on the recognition moment with illumination, blur and affine invariance.

5. The unmanned boat visual guidance system according to claim 4, characterized in that: The configuration work of configuring the target detection algorithm for configuring the unmanned boat to detect the target of the mother ship based on the mother ship target recognition database also includes: For the updated mother ship target recognition database, sampling is performed based on the queen template sampling algorithm; On the basis of the sampling results, an algorithm is configured based on the grayscale of the feature points corresponding to the reflective near-infrared cooperative target and the optical flow characteristics of the reflective near-infrared cooperative target as the mother ship sways.

6. The unmanned boat visual guidance system according to claim 5, characterized in that: The configuration work of configuring the target detection algorithm for configuring the unmanned boat to detect the target of the mother ship based on the mother ship target recognition database also includes: According to the known layout rules of the reflective near-infrared cooperative targets, an automatic adjustment algorithm for target point tracking frames with occlusion adaptability is configured.

7. The unmanned boat visual guidance system according to claim 1, characterized in that: The target detection algorithm involved in the process of the unmanned boat solving the position and posture of itself relative to the mother ship, and the corresponding configuration work includes: Based on the camera imaging model and the PnP pose solving principle, combined with the constraints of the mother ship docking position, the number of cooperative target points and the distribution position of the cooperative target are used as optimization variables, and the minimum error between the position and pose solved by the PnP algorithm and the true position and pose is used as the optimization index. The distribution position of the reflective near-infrared cooperative target at the mother ship docking position is obtained by genetic algorithm optimization; The linear and nonlinear constraints in the imaging process are explored, and the parameterized imaging model is used. The minimum deviation between the image coordinates of the cooperative target imaging and the image coordinates obtained by reprojecting the pose measurement model is used as the indicator. A cost function for nonlinearly solving the pose from a global optimization perspective is established. Based on the non-iterative optimization theoretical algorithm, a position and pose measurement algorithm with high stability, high precision and high real-time performance is solved.

8. The unmanned boat visual guidance system according to claim 1, characterized in that: For the different target detection algorithms involved in the whole process, the corresponding configuration work includes: In the semi-physical simulation experiment, the simulated component of the mother ship docking position is set on the ground at a certain distance from the three-axis turntable, and the active infrared vision system is installed on the three-axis turntable. The three-axis turntable simulates the sea conditions to shake the active infrared vision system on the simulated component of the unmanned boat. The position between the mother ship docking position and the simulated component of the unmanned boat is measured by a laser total station, and the attitude parameters are obtained by measuring the parameters of the three-axis turntable itself. The rain and fog climate is achieved by spraying water and fog with a spray machine. Based on the experimental results of the semi-physical simulation experiment, the different target detection algorithms involved in the whole process are further optimized.

9. A visual guidance method for an unmanned boat, characterized in that: The unmanned boat visual guidance method is applied to an unmanned boat visual guidance system, the unmanned boat visual guidance system comprises an unmanned boat and a mother ship, the unmanned boat is equipped with a passive infrared vision system and an active infrared vision system, the mother ship is equipped with a reflective near-infrared cooperative target at a preset mother ship docking position, and the unmanned boat visual guidance method comprises: When the unmanned boat is 30 meters away from the mother ship, in a passive working mode, the unmanned boat emits mid-to-far infrared light in the 8-12um band to the mother ship through the passive infrared vision system to detect the mother ship, and guides itself to approach the mother ship based on the target detection result; When the unmanned boat is within 30m of the mother ship, the active vision system irradiates the docking position of the mother ship with near-infrared light in the 900nm band in the active working mode, so that the reflective near-infrared cooperative target emits high-brightness infrared light, and obtains a corresponding high-brightness near-infrared cooperative target image; The unmanned boat combines the high-brightness near-infrared cooperative target image and the geometric information of the reflective near-infrared cooperative target to solve its own position and posture relative to the mother ship; The unmanned boat guides itself toward the mother ship based on the position and posture to complete the berthing and recovery mission.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor to execute the method of claim 9.

Images