An Unmanned Control Decision-making Method for the Optoelectronic System of an Unmanned Boat

Abstract

The present invention discloses an unmanned control decision-making method for the optoelectronic system of an unmanned boat. Under the condition of external target indication information, the optoelectronic system autonomously responds to the radar target indication. After the target enters the field of view of the image sensor, it switches to intelligent detection and recognition, captures and tracks the target, measures the target, and reports the comprehensive target information. When the target indication target does not enter the field of view, the optoelectronic system will autonomously search for the target within a certain range of the target indication position and then switch to the subsequent process, without any human intervention throughout the process. When the target tracking is stable, a target motion track will be established based on the measurement information. When the tracked target is lost, the optoelectronic system predicts and extrapolates based on the target track within a certain time to quickly and automatically retrieve the target. Compared with the current method, the method of the present invention utilizes multi-source heterogeneous data, enhancing the robustness and tracking success rate of the unmanned control decision-making method for the optoelectronic system of the unmanned boat.

Description

Technical Field

[0001] The present invention belongs to the technical field of automatic control, and relates to a comprehensive information processing method for an unmanned boat optoelectronic system, and particularly to an unmanned control decision-making method for an unmanned boat optoelectronic system, so as to improve the unmanned and intelligent levels of equipment. Background Art

[0002] An unmanned boat optoelectronic system usually consists of a servo pan-tilt head, a visible / infrared camera, and a laser rangefinder. The servo pan-tilt head keeps the system dynamically stable, the visible / infrared camera provides target image information, and the laser rangefinder measures the target distance information.

[0003] For the requirements of the unmanned boat platform for the unmanned control and intelligent decision-making of the optoelectronic system, one typical situation is to quickly track and locate a single point target detected by the radar and report the target information. The existing unmanned control decision-making methods for unmanned boat optoelectronic systems usually only rely on single-state information to implement a simple search-track-range measurement algorithm process, and have the following disadvantages: the autonomous working mode is not perfect, there are deficiencies such as the tracking field of view being unable to adapt, the target loss judgment being untimely, and it being difficult to quickly and automatically retrieve the lost tracking target. Summary of the Invention

[0004] Aiming at the above deficiencies in the prior art, the present invention patent provides an unmanned control decision-making method for an unmanned boat optoelectronic system. By fusing multi-source heterogeneous information, a perfect unmanned control decision-making method framework is established to realize the unmanned control and intelligent decision-making functions of the optoelectronic system under a type of typical requirements, and achieve the effect of fully autonomously and stably tracking the task target of the optoelectronic system under the condition of no human intervention.

[0005] The technical solution adopted by the present invention to solve its technical problems is: an unmanned control decision-making method for an unmanned boat optoelectronic system, including the following steps:

[0006] Step 1, after the system receives external target indication information including azimuth, pitch, and distance and the working mode, it enters the servo turning to target mode, turns the azimuth and pitch, adaptively adjusts the camera field of view according to the target distance and the preset scale of the target, and judges whether the servo target indication is in place according to the optoelectronic state information including servo angle measurement, working mode, camera field of view angle, laser distance value, and target geographical coordinate. Otherwise, continue to turn the azimuth and pitch. If yes, enter the next step;

[0007] Step 2, judge whether the target is detected and extracted according to the image detection algorithm information including the quantity, target pixel coordinates, and type. If not (not detected and extracted within x seconds), then point left and right within the set search range at the target indication azimuth and continue to judge whether the servo target indication is in place. If yes, enter the next step;

[0008] Step 3: Select the target closest to the image center. Based on the navigation information including the heading angle, roll, and pitch, calculate the azimuth and pitch of the detected target in the geographical coordinate system to obtain the target indication information of the detected target including azimuth and pitch. After the servo responds to the target indication information of the detected target, adjust the azimuth and pitch. Then, judge whether the servo target indication is in place according to the optoelectronic state information. Otherwise, continue to judge whether a target is detected and extracted. If so, proceed to the next step;

[0009] Step 4: Calculate and judge whether the detection box and the tracking box coincide according to the image tracking algorithm information including the miss distance and the tracking gate scale. Otherwise, the image tracking algorithm is guided by the detection result to capture the target, and then judge again whether the detection box and the tracking box coincide. If so, proceed to the next step;

[0010] Step 5: Enter the servo-to-tracking mode. According to the optoelectronic state information, select tracking judgment, azimuth angle measurement judgment, calculation of the pitch angle of the optical axis in the geographical coordinate system, or target loss judgment.

[0011] Furthermore, the tracking judgment in Step 5 is to judge whether the tracking is stable according to the optoelectronic state information. Otherwise, stop laser emission; if so, input the azimuth and pitch of the target in the geographical coordinate system into the trajectory prediction module, output the predicted value of the azimuth and pitch of the target in the geographical coordinate system, then emit laser for ranging. Judge whether the laser data is valid according to the optoelectronic state information, and then calculate and judge whether the difference between the laser distance value and the external target indication distance is within the error range according to the external target indication information and the optoelectronic state information. If so, the camera adaptively adjusts the field of view according to the laser distance value information; otherwise, calculate the distance value according to the pixels of the detected target, the camera adaptively adjusts the field of view according to the calculated distance value, and then judge whether a target is detected and extracted according to the image detection algorithm information. If not (no detection and extraction for x seconds), report target loss, perform trajectory extrapolation search according to the predicted value of the azimuth and pitch of the target in the geographical coordinate system, and the servo responds to the predicted target indication value of the azimuth and pitch of the target in the geographical coordinate system, and turns the mode to adjust the azimuth and pitch; if so, select the target closest to the image center, judge whether the detected and extracted target is within the guidance area. Otherwise, set the detection result guidance to invalid. If so, calculate and judge whether the detection box and the tracking box coincide according to the image detection algorithm information and the image tracking algorithm information. If so, set the detection result guidance to invalid. Otherwise, judge whether the target is detected and extracted continuously for x frames. If so, the image tracking algorithm is guided by the detection result to capture the target. Otherwise, set the detection result guidance to invalid.

[0012] Furthermore, the azimuth angle measurement judgment in Step 5 is to judge whether the azimuth angle measurement is in the laser prohibited emission area according to the optoelectronic state information. If so, enter the servo-to-target-indication mode until the laser stops emitting.

[0013] Further, the pitch angle calculation of the optical axis in the geographic coordinate system in step 5 is to calculate the pitch angle of the optical axis in the geographic coordinate system based on the optoelectronic state information and navigation information, determine whether the pitch angle of the optical axis exceeds the threshold, report the loss of the target, perform extrapolation search for the track according to the predicted azimuth and pitch values of the target in the geographic coordinate system, and the servo response predicts the target to the predicted azimuth and pitch values of the target in the geographic coordinate system, and switch the mode to adjust the azimuth and pitch.

[0014] Further, the target loss judgment in step 5 is to judge whether the target is lost according to the optoelectronic state information. If so, report the loss of the target, perform extrapolation search for the track according to the predicted azimuth and pitch values of the target in the geographic coordinate system, and the servo response predicts the target to the predicted azimuth and pitch values of the target in the geographic coordinate system, and switch the mode to adjust the azimuth and pitch.

[0015] Furthermore, after switching the mode to adjust the azimuth and pitch, the servo response predicts the target to the predicted azimuth and pitch values of the target in the geographic coordinate system, and switches the mode to adjust the azimuth and pitch. Select to judge whether the servo target indication is in place according to the optoelectronic state information, or judge whether the extrapolation duration reaches the x-second threshold. If so, the servo response system aims, switches to the aiming mode and adjusts the azimuth and pitch, and judges again whether the target is lost according to the optoelectronic state information.

[0016] The beneficial effects of the present invention are as follows: The control decision-making method of the present invention comprehensively utilizes heterogeneous data such as target indication information, navigation information, tracked target information, optoelectronic state information, and scene information, and realizes optoelectronic comprehensive control and decision-making through a multi-source information fusion processing algorithm based on a multi-temporal and spatial domain associated state machine. Moreover, it comprehensively considers various technologies such as tracking field-of-view adaptability, timely determination of target loss, rapid automatic recovery after target loss, and parametric configuration of algorithm process parameters, and realizes the effect of an optoelectronic fully autonomous stable tracking task target under a typical requirement and without human intervention.

[0017] Description of the drawings.

[0018] Figure 1 It is a partial flow diagram of the control method of the present invention;

[0019] Figure 2 It is another partial flow diagram of the control method of the present invention. Detailed implementation manners

[0020] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0021] In view of the deficiencies in the existing technology of the unmanned control decision-making method for the optoelectronic system of unmanned boats, such as imperfect working modes, inability to adaptively adjust the tracking field of view, untimely judgment of target loss, and difficulty in quickly and automatically retrieving the lost tracking target, a perfect unmanned control decision-making method framework is established by integrating multi-source heterogeneous information, realizing the unmanned control and intelligent decision-making functions of the optoelectronic system under a typical type of requirements, and achieving the effect of fully autonomous and stable tracking of the task target by the optoelectronic system under the condition of no human intervention.

[0022] In the case of having external target indication information, the optoelectronic system autonomously responds to the radar target indication. After the target enters the field of view of the image sensor, it transfers to intelligent detection and recognition, capture and tracking, measurement of the target, and reports the comprehensive target information. In the case where the indicated target does not enter the field of view, the optoelectronic system will autonomously search for the target within a certain range of the indicated position and then transfer to the subsequent process, without any human intervention throughout the process. When the target tracking is stable, a target motion track will be established based on the measurement information. When the tracked target is lost, the optoelectronic system predicts and extrapolates based on the target track within a certain time and quickly and automatically retrieves the target. During the whole process, the tracking field of view is adaptively adjusted according to the target distance and scale.

[0023] As Figure 1 shown, an unmanned control decision-making method for an unmanned boat optoelectronic system disclosed by the present invention includes two logical relationships, serial and parallel, and the serial steps are described as follows.

[0024] Step 1: The system issues external target indication information ① and the working mode, and jumps to Step 2.

[0025] Step 2: The servo responds to the system target indication, transfers to the target indication mode, adjusts the azimuth and elevation, and jumps to Step 3.

[0026] Step 3: Adaptively adjust the camera field of view according to the target indication distance and the preset scale of the target, and jump to Step 4.

[0027] Step 4: According to the optoelectronic status information ②, judge whether the servo target indication is in place. If so, jump to Step 5; otherwise, jump to Step 2.

[0028] Step 5: According to the image detection algorithm information ③, judge whether the target is detected and extracted. If so, jump to Step 7; if not (no detection and extraction within x seconds), jump to Step 6.

[0029] Step 6: Point left and right back and forth within the set search range in the indicated azimuth, and jump to Step 4.

[0030] Step 7: Select the target closest to the center of the image, and jump to Step 8.

[0031] Step 8: According to the navigation information ⑤, calculate the azimuth and elevation of the detected target in the geographical coordinate system, obtain the target indication information ⑥ of the detected target, and jump to Step 9.

[0032] Step 9, the servo responds to detect the target target indication information ⑥, turns the azimuth and pitch, and jumps to Step 10.

[0033] Step 10, based on the optoelectronic state information ②, determine whether the servo target indication is in place. If so, jump to Step 11; otherwise, jump to Step 5.

[0034] Step 11, based on the image tracking algorithm information ④, calculate and determine whether the detection box and the tracking box coincide. If so, jump to Step 13; otherwise, jump to Step 12.

[0035] Step 12, the detection result guides the image tracking algorithm to capture the target, and jumps to Step 11.

[0036] Step 13, the servo switches to the tracking mode, and jumps to Step 14, Step 30, Step 31, Step 35.

[0037] The above are the serial steps of the present invention. Figure 2 The following are the parallel steps of the present invention (where Figure 1 and Figure 2 are connected left and right to form the overall control decision-making steps of this patent application), which are described as follows.

[0038] Step 14, based on the optoelectronic state information ②, determine whether the tracking is stable. If so, jump to Step 15, Step 16, Step 22; otherwise, jump to Step 36.

[0039] Step 15, input the target geographical system azimuth and pitch ⑦ into the trajectory prediction module, and output the predicted value of the target geographical system azimuth and pitch ⑧.

[0040] Step 16, emit laser for ranging, and jump to Step 17.

[0041] Step 17, based on the optoelectronic state information ②, determine whether the laser data is valid. If so, jump to Step 18.

[0042] Step 18, based on the external target indication information ① and the optoelectronic state information ②, calculate and determine whether the difference between the laser distance value and the external target indication distance is within the error range. If so, jump to Step 19; otherwise, jump to Step 20.

[0043] Step 19, adaptively adjust the camera field of view according to the laser distance value information.

[0044] Step 20, calculate the distance value according to the detected target pixels, and jump to Step 21.

[0045] Step 21, adaptively adjust the camera field of view according to the calculated distance value.

[0046] Step 22, based on the image detection algorithm information ③, determine whether the target is detected and extracted. If so, jump to Step 23; if not (no detection and extraction for x seconds), jump to Step 24.

[0047] Step 23, select the target closest to the center of the image and jump to step 25.

[0048] Step 24, report that the target is lost, perform extrapolation search based on the predicted value ⑧ of the target's geographical system azimuth and pitch, and jump to step 33.

[0049] Step 25, determine whether the detected and extracted target is within the guiding area. If yes, jump to step 26; otherwise, jump to step 27.

[0050] Step 26, according to the image detection algorithm information ③ and the image tracking algorithm information ④, calculate and determine whether the detection box and the tracking box coincide. If yes, jump to step 29; otherwise, jump to step 27.

[0051] Step 27, determine whether the target has been detected and extracted for consecutive x frames. If yes, jump to step 28; otherwise, jump to step 29.

[0052] Step 28, the detection result guides the image tracking algorithm to capture the target.

[0053] Step 29, the detection result guide is set to invalid.

[0054] Step 30, according to the optoelectronic state information ②, determine whether the azimuth angle measurement is in the laser prohibited emission area. If yes, jump to step 2 and step 36.

[0055] Step 31, according to the optoelectronic state information ② and the navigation information ⑤, solve the pitch angle of the optical axis in the geographical system and jump to step 32.

[0056] Step 32, determine whether the pitch angle of the optical axis exceeds the threshold. If yes, jump to step 24.

[0057] Step 33, the servo response predicts the target to the predicted value ⑧ of the target's geographical system azimuth and pitch, switches to the mode, adjusts the azimuth and pitch, and jumps to step 4 and step 34.

[0058] Step 34, determine whether the extrapolation duration reaches the x - second threshold. If yes, jump to step 2.

[0059] Step 35, according to the optoelectronic state information ②, determine whether the target is lost. If yes, jump to step 24.

[0060] Step 36, the laser stops emitting, and the unmanned control decision of this patent is completed.

[0061] Figure 1 and Figure 2As shown, ①, ②, ③, ④, and ⑤ are respectively external target information, optoelectronic state information, image detection algorithm information, image tracking algorithm information, and navigation information, which are input into the algorithm process. ⑥ is the detected target target information, obtained by calculation. ⑦ is the azimuth and pitch in the geographical system of the tracked target, obtained by feedback of the optoelectronic state information. ⑧ is the predicted value of the target geographical system azimuth and pitch, extrapolated and calculated by the track prediction algorithm.

[0062] The above embodiments are only illustrative of the principles and effects of the present invention, as well as some applied embodiments. For those of ordinary skill in the art, without departing from the inventive concept of the present invention, several modifications and improvements can still be made, and these all fall within the protection scope of the present invention.

Claims

1. A method for unmanned control decision-making of an optoelectronic system of an unmanned boat, characterized in that: Including the following steps Step 1: After the system receives external target information including azimuth, pitch, and distance, as well as the working mode, it enters the servo target-turning mode, turns the azimuth and pitch, adaptively adjusts the camera field of view according to the target distance and the preset scale of the target, and determines whether the servo target indication is in place based on the optoelectronic state information including servo angle measurement, working mode, camera field of view angle, laser distance value, and target geographic coordinate system. If not, continue to turn the azimuth and pitch; if so, proceed to the next step; Step 2: Determine whether the target is detected and extracted based on the image detection algorithm information including the quantity, target pixel coordinates, and type. If not, turn left and right within the set search range at the indicated azimuth and continue to determine whether the servo target indication is in place. If so, proceed to the next step; Step 3: Select the target closest to the center of the image. Based on the navigation information including the heading angle, roll, and pitch, calculate the azimuth and pitch of the detected target in the geographic coordinate system to obtain the target indication information of the detected target including azimuth and pitch. After the servo responds to the target indication information of the detected target, turn the azimuth and pitch, and determine whether the servo target indication is in place based on the optoelectronic state information. If not, continue to determine whether the target is detected and extracted. If so, proceed to the next step; Step 4: Calculate and determine whether the detection box and the tracking box coincide based on the image tracking algorithm information including the miss distance and the tracking gate scale. If not, use the detection result to guide the image tracking algorithm to capture the target and determine again whether the detection box and the tracking box coincide. If so, proceed to the next step; Step 5: Enter the servo tracking mode. Based on the optoelectronic state information, select tracking judgment, azimuth angle measurement judgment, calculation of the pitch angle of the optical axis in the geographic coordinate system, or target loss judgment: Determine whether the tracking is stable based on the optoelectronic state information. If not, stop emitting the laser; If so, input the azimuth and pitch of the target in the geographic coordinate system into the trajectory prediction module, output the predicted value of the azimuth and pitch of the target in the geographic coordinate system, then emit the laser for ranging, determine whether the laser data is valid based on the optoelectronic state information, and calculate and determine whether the difference between the laser distance value and the external target distance is within the error range based on the external target information and the optoelectronic state information. If so, the camera adaptively adjusts the field of view according to the laser distance value information; Otherwise, calculate the distance value based on the detected target pixels, the camera adaptively adjusts the field of view according to the calculated distance value, and then determine whether the target is detected and extracted based on the image detection algorithm information. If not, report target loss, perform trajectory extrapolation search based on the predicted value of the azimuth and pitch of the target in the geographic coordinate system, and the servo responds to the predicted target indication of the azimuth and pitch of the target in the geographic coordinate system and turns the azimuth and pitch in the mode; If so, select the target closest to the center of the image, and determine whether the detected and extracted target is within the guidance area. If not, set the detection result guidance to invalid. If so, calculate and determine whether the detection box and the tracking box coincide based on the image detection algorithm information and the image tracking algorithm information. If so, set the detection result guidance to invalid. Otherwise, determine whether the target is detected and extracted continuously for x frames. If so, use the detection result to guide the image tracking algorithm to capture the target. Otherwise, set the detection result guidance to invalid.

2. The unmanned control decision-making method for the optoelectronic system of an unmanned boat according to claim 1, wherein, The azimuth angle measurement judgment in Step 5 is to determine whether the azimuth angle measurement is in the laser prohibited area based on the optoelectronic state information. If so, enter the servo target-turning mode until the laser stops emitting.

3. The unmanned control decision-making method for the optoelectronic system of an unmanned boat according to claim 1, characterized in that, The pitch angle solution of the optical axis in the geographic coordinate system in step 5 is to calculate the pitch angle of the optical axis in the geographic coordinate system based on the optoelectronic state information and navigation information, determine whether the pitch angle of the optical axis exceeds the threshold, report the loss of the target, perform extrapolation search for the track according to the predicted azimuth and pitch values of the target in the geographic coordinate system, and the servo response predicts the target to the predicted azimuth and pitch values of the target in the geographic coordinate system, and switch the mode to adjust the azimuth and pitch.

4. The unmanned control decision-making method for the optoelectronic system of an unmanned boat according to claim 1, characterized in that, The target loss judgment in step 5 is to determine whether the target is lost according to the optoelectronic state information. If so, report the loss of the target, perform extrapolation search for the track according to the predicted azimuth and pitch values of the target in the geographic coordinate system, and the servo response predicts the target to the predicted azimuth and pitch values of the target in the geographic coordinate system, and switch the mode to adjust the azimuth and pitch.

5. A method for unmanned control decision-making of an unmanned boat optoelectronic system according to any one of claims 2 to 4, characterized in that After switching the mode to adjust the azimuth and pitch, the servo response predicts the target to the predicted azimuth and pitch values of the target in the geographic coordinate system, and switch the mode to adjust the azimuth and pitch. Select to determine whether the servo target indication is in place according to the optoelectronic state information, or determine whether the extrapolation duration reaches the x-second threshold. If so, the servo response system aims at the target, switches to the target indication mode and adjusts the azimuth and pitch, and determines again whether the target is lost according to the optoelectronic state information.

Images