Monocular camera underwater target detection positioning method based on motion inversion

Abstract

The invention relates to the technical field of underwater robot navigation and computer vision, in particular to a monocular camera underwater target detecting and positioning method based on motion inversion, which comprises the following steps: acquiring an underwater image; a target in the underwater image is detected and compensated, and the actual physical width and the actual physical height of the target are obtained; acquiring a transverse deviation angle of the target center relative to the camera center; acquiring a linear distance from the center of the target to the center of the camera; acquiring the longitudinal height between the center of the target and the center of the camera; and positioning the target. According to the method, the target can be detected, and the size and the position can be stably estimated at the same time, so that the continuity and the accuracy of target detection and positioning are ensured.

Description

Technical Field

[0001] The present invention relates to the field of underwater robot navigation and computer vision technology, and in particular to a monocular camera underwater target detection and positioning method based on motion inversion. Background Art

[0002] Underwater vehicles are indispensable tools in the fields of marine science, submarine pipeline inspection, resource exploration, underwater archaeology, and even reconnaissance. They can perform tasks, collect data, conduct observations, and intervene in deep-sea environments that humans cannot directly reach. When performing underwater tasks, underwater vehicles usually need to be able to accurately identify and locate target objects. Cameras are widely equipped in vehicles as visual perception sensors. They perceive and understand the underwater environment through visual information to perform complex tasks such as target detection, obstacle avoidance, charging and docking, and intervention.

[0003] Although the positioning method based on binocular vision can provide depth information and realize positioning in three-dimensional space, it is relatively expensive and large in size, and is not easy to integrate into small or micro underwater vehicles, and performs poorly in weak light or turbid underwater environments; LiDAR can provide high-precision distance and direction information and performs stably under various lighting conditions, but it is expensive and large in size, and is not suitable for small underwater vehicles. In addition, it suffers from severe attenuation in underwater environments and has a limited effective working range; although traditional positioning methods based on monocular cameras (such as the similar triangle method) are low-cost, easy to install and maintain, they require the camera internal parameters such as focal length to be known in advance, which is usually not feasible in autofocus digital cameras, and information such as relative depth and lateral deflection angle cannot be directly obtained; mainstream computer vision task data sets, such as COCO and VOC, although relatively rich and mature, are all made of objects in the air. In summary, when the target detection model trained with these data sets is used to detect underwater targets, the image is distorted due to water waves, turbulence, and refraction, resulting in the inability to perform continuous and stable detection, thereby affecting the continuity and accuracy of target detection and positioning.

[0004] Therefore, it is necessary to provide a monocular camera underwater target detection and positioning method based on motion inversion to solve the above problems. Summary of the invention

[0005] The present invention provides a method for underwater target detection and positioning using a monocular camera based on motion inversion, so as to solve the problem that when an existing target detection model trained with a data set is used to detect underwater targets, images are distorted due to factors such as water waves, turbulence and refraction, resulting in inability to perform continuous and stable detection, thereby affecting the continuity and accuracy of target detection and positioning.

[0006] The present invention discloses a method for underwater target detection and positioning using a monocular camera based on motion inversion, which adopts the following technical solutions, including: Collect multiple frames of underwater images continuously; Target detection is performed on underwater images using a target detection algorithm. If a target is detected in a current underwater image frame, but a target is not detected in a next underwater image frame of the current underwater image frame, and the interval between the previous underwater image frame and the next underwater image frame is less than a preset time length, a detection missing frame compensation algorithm based on kinematic inversion is used to infer the target center coordinates of the next underwater image frame; if the next two underwater images also fail to detect a target, and the interval between the current underwater image frame and the next two underwater images is less than a preset time length, a detection missing frame compensation algorithm based on kinematic inversion is used to infer the target center coordinates of the next two underwater images until the underwater image detects a target; wherein the target detection result is a target label, a target confidence, a central horizontal coordinate and a central vertical coordinate of the target, and a pixel width and a pixel height of the target in the image; the actual physical width and actual physical height of the target are obtained according to the pixel width and pixel height of the target in the image, the change in the vertical coordinate of the target center in two adjacent frames, and the change in the depth of the vehicle; According to the central abscissa of the target in each frame of underwater image, the central abscissa and the lateral size of each frame of underwater image, and the lateral field of view of the camera, the lateral deviation angle of the center of the target relative to the center of the camera is obtained; According to the lateral size of the underwater image of the frame where the target is located, the width of the target in the image, the actual physical width of the target, and the lateral field of view of the camera, the straight-line distance from the center of the target to the center of the camera is obtained; According to the longitudinal size of the underwater image of the target frame, the longitudinal field of view of the camera, the central longitudinal coordinate of the target, the central longitudinal coordinate of the underwater image of the target frame, and the straight-line distance from the center of the target to the center of the camera, the longitudinal height between the center of the target and the center of the camera is obtained; The target is located according to the longitudinal height between the target center and the camera center, the straight-line distance from the target center to the camera center, and the lateral deviation angle of the target center relative to the camera center.

[0007] Preferably, the steps of calculating the target center coordinates of the next frame of underwater image based on the detection missed frame compensation algorithm of kinematic inversion are: Establish a ground coordinate system and a carrier coordinate system: the ground coordinate system takes the initial buoyancy center of the vehicle as the origin, the x-axis of the ground coordinate system points to the due north, the y-axis of the ground coordinate system is perpendicular to the x-axis and points to the due east, and the z-axis of the ground coordinate system is perpendicular to the x-axis and the y-axis to form a right-handed system; the carrier coordinate system takes the buoyancy center of the vehicle as the origin, the x-axis of the carrier coordinate system is along the length direction of the underwater vehicle and points forward, the y-axis of the carrier coordinate system is perpendicular to the x-axis and points to the starboard, and the z-axis of the carrier coordinate system is perpendicular to the x-axis and the y-axis to form a right-handed system; According to the pixels corresponding to the horizontal unit angle and the vertical unit angle of the current frame underwater image, the central horizontal coordinate and the central vertical coordinate of the target in the current frame underwater image, as well as the rolling angle, pitch angle, navigation angle, depth of the vehicle when collecting two adjacent frames of underwater images, and the distance from the center of the target in the current frame to the vehicle, the target center coordinates of the next frame underwater image are calculated.

[0008] Preferably, the expression of the target center coordinates of the next frame of underwater image is:

[0009] In the formula, Pd x Represents the corresponding pixel of the horizontal unit angle of the underwater image; Pd y Represents the corresponding pixel of the vertical unit angle of the underwater image; Indicates The central ordinate of the target in the frame underwater image; Indicates The central ordinate of the target in the frame underwater image; Indicates The central ordinate of the target in the frame underwater image; Indicates The central horizontal coordinate of the target in the frame underwater image; Indicates collection of Frame underwater image to collect the first The change in the rolling angle of the vehicle when framing underwater images; Indicates collection of Frame underwater image to collect the first The pitch angle change of the vehicle when framing underwater images; Ψ Indicates collection of Frame underwater image to collect the first The change in the yaw angle of the vehicle when capturing underwater images; Indicates collection of Frame underwater image to collect the first The depth change of the vehicle when framing underwater images; Indicates The distance from the frame vehicle to the center of the target in the underwater image.

[0010] Preferably, obtaining the actual physical width and actual physical height of the target includes:

[0011]

[0012] In the formula, Indicates the actual physical width of the target; Indicates the actual physical height of the target; Indicates the pixel width of the target in the underwater image; Indicates the pixel height of the target in the underwater image; Indicates collection of Frame underwater image to collect the first The depth change of the vehicle when framing underwater images; Indicates collection of Frame underwater image to collect the first The change in the vertical coordinate of the target center when framing the underwater image; Indicates the physical distance corresponding to the unit pixel.

[0013] Preferably, obtaining the lateral deviation angle of the target center relative to the camera head center includes:

[0014] In the formula, Indicates the lateral deviation angle of the target center relative to the camera center; Represents the central horizontal coordinate of the underwater image in the image coordinate system; Represents the horizontal coordinate of the target center in the image coordinate system; is the lateral field of view of the camera; is the horizontal pixel size of the underwater image.

[0015] Preferably, obtaining the straight-line distance from the center of the target to the center of the camera head includes:

[0016] In the formula, Indicates the straight-line distance from the center of the target to the center of the camera; is the lateral field of view of the camera; is the horizontal pixel size of the underwater image; Indicates the actual physical width of the target; Represents the pixel width of the target in the underwater image.

[0017] Preferably, obtaining the longitudinal height between the center of the target and the center of the camera head includes:

[0018] In the formula, Indicates the vertical height between the center of the target and the center of the camera head; Indicates the straight-line distance from the center of the target to the center of the camera; is the vertical field of view of the camera; is the vertical pixel size of the underwater image; yc Represents the central ordinate of the underwater image in the image coordinate system; y t Represents the vertical coordinate of the target center in the image coordinate system.

[0019] Preferably, the target detection algorithm adopts the YOLOv4-tiny algorithm.

[0020] The beneficial effects of the present invention are: The target is acquired based on the target detection algorithm. In the continuous underwater image frames, the target is detected in the previous underwater image but not in the next underwater image. Then, the target center coordinates of the next underwater image are acquired and estimated based on the detection missing frame compensation algorithm based on kinematic inversion. The actual physical width and actual physical height of the target are acquired according to the pixel width and pixel height of the target in the image, the change in the longitudinal coordinate of the target center in two adjacent frames and the change in the depth of the vehicle. Finally, the longitudinal height between the target center and the camera center, the straight-line distance from the target center to the camera center, and the lateral deviation angle of the target center relative to the camera center are acquired based on the positioning algorithm. The target is positioned according to the longitudinal height between the target center and the camera center, the straight-line distance from the target center to the camera center, and the lateral deviation angle of the target center relative to the camera center. In summary, the present invention can simultaneously detect the target and robustly estimate the size and position, thereby ensuring the continuity and accuracy of target detection and positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 It is a flow chart of a method for underwater target detection and positioning using a monocular camera based on motion inversion of the present invention; Figure 2 An underwater image of a starfish as a target in an embodiment of the present invention; Figure 3 is a schematic diagram of a ground coordinate system, a carrier coordinate system and an image coordinate system in an embodiment of the present invention; Figure 4 Schematic diagram of the lateral deviation angle of the target center relative to the camera head center in an embodiment of the present invention; Figure 5 Schematic diagram of the straight-line distance from the center of the target to the center of the camera head in an embodiment of the present invention; Figure 6is a schematic diagram of the longitudinal height between the center of the target and the center of the camera head in an embodiment of the present invention; Figure 7 It is a comparison diagram of the detection result of the YOLOv4-tiny algorithm of the underwater image in the embodiment of the present invention and the calculation result of the detection missing frame compensation algorithm of the underwater image through kinematic inversion; Figure 8 It is a result diagram of the lateral position estimation error corresponding to the detection missing frame compensation algorithm of the inversion of the static model and the detection missing frame compensation algorithm of the inversion of the motion model in the experimental process of the embodiment of the present invention; Fig. 9 It is a result diagram of the longitudinal position estimation error corresponding to the detection missing frame compensation algorithm of the inversion of the static model and the detection missing frame compensation algorithm of the inversion of the motion model in the experimental process of the embodiment of the present invention; Fig.10 The present invention is a schematic diagram of an algorithm framework of a monocular camera underwater target detection and positioning method based on motion inversion. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] An embodiment of a monocular camera underwater target detection and positioning method based on motion inversion of the present invention is as follows: Figure 1 and Fig.10 As shown, including: S1, collecting underwater images; Specifically, in this embodiment, a monocular camera of the aircraft is used to collect multiple frames of continuous underwater images.

[0025] S2, detecting and compensating the target in the underwater image, and obtaining the actual physical width and actual physical height of the target; Step 21: The steps of detecting and compensating the target in the underwater image are as follows: like Fig.10As shown, the target detection algorithm is used to detect the target of the underwater image. If the current frame underwater image detects the target, the next frame underwater image of the current frame underwater image does not detect the target, and the interval time between the previous frame underwater image and the next frame underwater image is less than the preset time length, the kinematic inversion detection missing frame compensation algorithm is used to calculate the target center coordinates of the next frame underwater image; if the next two frames of underwater images also do not detect the target, and the interval time between the current frame underwater image and the next two frames of underwater images is less than the preset time length, the kinematic inversion detection missing frame compensation algorithm is used to calculate the target center coordinates of the next two frames of underwater images, until The underwater image detects the target. If the next three frames of underwater images also do not detect the target, and the interval time between the current frame of underwater image and the next three frames of underwater image is less than the preset duration, the kinematic inversion detection missed frame compensation algorithm is used to infer the target center coordinates of the next three frames of underwater image, and so on, until the underwater image detects the target. If the target is not detected subsequently, the interval time is calculated with the most recent frame of underwater image that detects the target. The preset duration in this embodiment is 2s. It should be noted that in this embodiment, the target detection algorithm is yolov4-tiny. The detection frequency of yolov4-tiny when running on jetson nano software is about 15hz, that is, 15 frames of underwater images can be detected in one second, and 2s is 30 frames of underwater images. When no target is detected in 30 consecutive frames of underwater images, it means that the target is out of the camera's shooting range, and no extrapolation is performed at this time; the target detection result includes the target label, target confidence, the target's center horizontal coordinate and center vertical coordinate, and the pixel width and pixel height of the target in the image; the actual physical width and actual physical height of the target are obtained based on the pixel width and pixel height of the target in the image.

[0026] Step 211, in this embodiment, the YOLOv4-tiny algorithm is used to detect targets in underwater images. The YOLO (You Only Look Once) series of algorithms are well-known for their efficient end-to-end detection process. It regards the target detection problem as a regression problem and directly predicts the coordinates and category probabilities of the bounding box in a single neural network. YOLOv4-tiny is a lightweight version of YOLOv4, which uses a network structure similar to YOLOv4, but adopts a series of strategies to reduce model complexity and computational complexity, such as reducing the number of network layers and the number of channels. Since YOLOv4-tiny can achieve efficient target detection in an environment with limited computing resources, improve the detection rate while maintaining high detection accuracy, it is suitable for deployment on single-board computers with limited computing power, and the technology is relatively mature and software development is relatively convenient. This embodiment will not be repeated, so this embodiment selects the YOLOv4-tiny algorithm.

[0027] Step 212, the steps of calculating the target center coordinates of the next frame of underwater image (i.e., missing frame compensation) based on the detection missing frame compensation algorithm of kinematic inversion are as follows: like Figure 3 As shown, in order to describe the motion of the aircraft, a ground coordinate system { n} and the carrier coordinate system { b}, the coordinate axis units are meters. The ground coordinate system is fixed to the earth, where the ground coordinate system { n}Origin o n Select the initial buoyancy center of the spacecraft at the initial time, axis o n x n Pointing due north, the axis o n y n Perpendicular to the axis o n x n and points due east, the axis o n z n Perpendicular to the axis o n x n and o n y n , direction o n x n y n z n Become a right-handed system, pointing directly downward; the carrier coordinate system { b}Fixed to the underwater vehicle, coordinate origin o b Select the center of buoyancy of the aircraft, axis o b x b Along the longitudinal axis of the underwater vehicle and pointing forward, the axis o b y b Perpendicular to the axis o b x b and pointed to starboard, axis o b z b Perpendicular to the axis ob x b and o b y b , direction o b x b y b z b The right-hand system points downward. The aircraft has six degrees of freedom, including forward and backward motion, lateral motion, heave motion, roll motion, pitch motion, and yaw motion. The position of the aircraft relative to the ground coordinate system is given by the coordinates ( x n ,y n ,z n ), and the attitude relative to the ground coordinate system is described by the three-axis Euler angles including roll angle, pitch angle, and yaw angle ( , , ) description; To describe the position of the target in the underwater image, this embodiment establishes an image coordinate system { p}, the unit of the coordinate axis is pixel. The origin of the image coordinate system o Select the upper left corner of the underwater image, axis ox Pointing due right, axis oy Perpendicular to the axis ox Pointing directly downward. In a frame of underwater image, the position of the target center is represented by its coordinates ( x t , y t ) is determined, and the image center coordinates are set to ( x c ,y c ), assuming that the size of the target remains unchanged and its position is fixed, the spacecraft makes a small movement, that is, only changes its three-axis Euler angle and depth in a small range, and the distance from the spacecraft to the center of the target D The target center position is calculated by the following formula:

[0028] in,

[0029] In the formula, Pd x Represents the corresponding pixel of the horizontal unit angle of the underwater image; Pd yRepresents the corresponding pixel of the vertical unit angle of the underwater image; Indicates The central ordinate of the target in the frame underwater image; Indicates The central ordinate of the target in the frame underwater image; Indicates The central ordinate of the target in the frame underwater image; Indicates The central horizontal coordinate of the target in the frame underwater image; Indicates collection of Frame underwater image to collect the first The change in the rolling angle of the vehicle when framing underwater images; Indicates collection of Frame underwater image to collect the first The pitch angle change of the vehicle when framing underwater images; Ψ Indicates collection of Frame underwater image to collect the first The change in the yaw angle of the vehicle when capturing underwater images; Indicates collection of Frame underwater image to collect the first The depth change of the vehicle when framing underwater images; Indicates that the aircraft has arrived The distance between the center of the target in the underwater image of the frame; when the frequency of the "breathing frame" is high, the interval between the image frames of the detected target is short, and the vehicle only changes its three-axis Euler angle and depth within a small range, which can be considered to meet the assumption of small-amplitude movement of the vehicle; as the frequency of the "breathing frame" increases, the interval between the image frames of the detected target increases, the actual situation deviates from the assumption, and the estimation error will gradually increase. When the YOLOv4-tiny algorithm is run on the vehicle, the frame rate is about 15fps. After many tests, it is found that it is more appropriate to set the maximum estimation frame number to 30, that is, when the interval between the image frames of the detected target is less than 2s, the algorithm is used for estimation, otherwise the target is considered to be lost; it should be noted that the "breathing frame" is a frame used by the vehicle to select the target in the image when performing target detection. When the detection is discontinuous, it is reflected that the detection frame on the image is sometimes absent, and this phenomenon is also vividly called the "breathing frame". In the experiment, it was found that the breathing frequency was higher when the vehicle's movement amplitude was smaller, and the breathing frequency was lower when the movement amplitude was larger. The compensation algorithm based on motion inversion is designed to solve the problem of unstable and discontinuous detection. When the target detection algorithm fails to detect the target, the motion information is used to estimate the target position.

[0030] Step 23: The steps of obtaining the actual physical width and actual physical height of the target are: For a single underwater target, its actual physical width is L , the actual physical height is H The target detection result output by the YOLOv4-tiny algorithm is its label, confidence, and the horizontal coordinate of the target center. x t and the vertical coordinate y t , the pixel width of the target in the underwater image w and pixel height h Combined with the vehicle's own motion information, the actual physical width of the target is L and actual physical height H Calculated by the following formula:

[0031] in,

[0032] In the formula, Indicates the actual physical width of the target; Indicates the actual physical height of the target; Indicates the pixel width of the target in the underwater image; Indicates the pixel height of the target in the underwater image; Indicates collection of Frame underwater image to collect the first The depth change of the vehicle when framing underwater images; Indicates collection of Frame underwater image to collect the first The change in the ordinate of the target center when framing the underwater image; Indicates the physical distance corresponding to the unit pixel.

[0033] S3, obtaining the lateral deviation angle of the target center relative to the camera center; Specifically, the lateral deviation angle of the target center relative to the camera center is obtained according to the central lateral coordinate of the target, the central lateral coordinate and lateral size of the underwater image of the frame where the target is located, and the lateral field of view of the camera.

[0034] The schematic diagram of the lateral deviation angle of the target center relative to the camera center is as follows: Figure 4 As shown, the expression of the lateral deviation angle of the target center relative to the camera center is:

[0035] In the formula, Indicates the lateral deviation angle of the target center relative to the camera center; Represents the central horizontal coordinate of the underwater image in the image coordinate system; Indicates the horizontal coordinate of the target center in the image coordinate system; is the lateral field of view of the camera; is the horizontal pixel size of the underwater image.

[0036] S4, obtaining the straight-line distance from the target center to the camera center; Specifically, the straight-line distance from the center of the target to the center of the camera is obtained according to the lateral size of the underwater image of the frame where the target is located, the width of the target in the image, the actual physical width of the target, and the lateral field of view of the camera.

[0037] Among them, the schematic diagram of the straight-line distance from the target center to the camera center is as follows: Figure 5 As shown, the expression of the straight-line distance from the target center to the camera center is:

[0038] In the formula, Indicates the straight-line distance from the center of the target to the center of the camera; is the lateral field of view of the camera; is the horizontal pixel size of the underwater image; Indicates the actual physical width of the target; It represents the pixel width of the target in the underwater image. It should be noted that this calculation method approximates the width of the target as a circular arc, which simplifies the calculation.

[0039] S5, obtaining the longitudinal height between the center of the target and the center of the camera head; Specifically, the longitudinal height between the center of the target and the center of the camera is obtained according to the longitudinal size of the underwater image of the frame where the target is located, the longitudinal field of view of the camera, the central longitudinal coordinate of the target, the central longitudinal coordinate of the underwater image of the frame where the target is located, and the straight-line distance from the center of the target to the center of the camera.

[0040] The schematic diagram of the longitudinal height between the target center and the camera center is as follows: Figure 6 As shown, the expression of the vertical height between the target center and the camera center is:

[0041] In the formula, Indicates the vertical height between the center of the target and the center of the camera head; Indicates the straight-line distance from the center of the target to the center of the camera; is the vertical field of view of the camera; is the vertical pixel size of the underwater image; yc Represents the central ordinate of the underwater image in the image coordinate system; y t Indicates the ordinate of the target center in the image coordinate system S6. Position the target; The target can be positioned based on the longitudinal height between the target center and the camera center, the straight-line distance from the target center to the camera center, and the lateral deviation angle of the target center relative to the camera center.

[0042] The following is described in conjunction with specific embodiments: The positioning method of the present invention is encapsulated as a ROS algorithm node and applied to an underwater vehicle, and the vehicle can detect and locate starfish using the positioning method of the present invention.

[0043] Conduct experiments to verify the detection missing frame compensation algorithm based on kinematic inversion: The algorithm is written in the computer vision module, and a ROS topic named "CvDr" is created. The number of frames detected by the YOLOv4-tiny algorithm, the number of frames calculated by the algorithm, and the deviation information calculated between the static model and the motion model are published for rosbag to subscribe and record. A starfish model sealed in a sealed bag is placed in the pool as the detection target, and the aircraft is placed 40 cm in front of the target and remotely controlled. The remotely controlled aircraft dives to a depth of 0.1m, and the remotely controlled yaw angle is used to bring the target back to the center only when the target is about to deviate from the camera's line of sight. Definition: If the target in this frame is detected by the YOLOv4-tiny algorithm, the detection frame is light blue; if it is calculated by the algorithm, it is black.

[0044] If a target is detected in one underwater image frame, but no target is detected in the next image frame, the detection missed frame compensation algorithm of kinematic inversion is triggered, and the target position is estimated based on the static model and the motion model for the frames without target within the maximum estimated frame number, until a certain underwater image frame detects the target again. The target position output by the underwater image frame is taken as the true value, and the difference is made with the estimated results of the static model and the motion model, and the absolute value is taken as the estimated deviation of the two models. If the estimated frame number exceeds the maximum estimated frame number, it is determined that the target is lost, and the model estimation deviation is not calculated. Figure 7 As shown, Figure 7 The detection results of the YOLOv4-tiny algorithm for a frame of underwater image and the inference results of the detection missing frame compensation algorithm for a frame of underwater image through kinematic inversion are shown. Figure 7 It can be seen that there is basically no difference between the two detection results; MATLAB is used to draw the lateral position estimation errors corresponding to the detection missing frame compensation algorithm of the static model inversion and the detection missing frame compensation algorithm of the motion model inversion during the experiment. Figure 8As shown, MATLAB is used to draw the longitudinal position estimation errors corresponding to the detection missing frame compensation algorithm of the static model inversion and the detection missing frame compensation algorithm of the motion model inversion during the experiment. Fig. 9 As shown in the figure, a motion model of an underwater vehicle is established using the detection missing frame compensation algorithm of kinematic inversion to estimate the target center position. In contrast, a static model (i.e., the vehicle is assumed to be motionless, which is the simplest motion model) is established and estimated. Figure 8 and Fig. 9 It can be seen that the error curve of the static model almost covers the error curve of the motion model, which shows that the motion model established by the present invention can better describe the motion of the underwater vehicle and improve the accuracy of missed frame compensation.

[0045] The performance comparison of the static model and the motion model is shown in Table 1. Compared with the static model, the inference algorithm based on the motion model reduces the average lateral error by about 32% and the average lateral error by about 6%. About 40% of the output detection results come from the inference algorithm, which greatly fills the detection gaps between the "breathing frames".

[0046] Table 1

[0047] An experiment was conducted to test the monocular camera target positioning algorithm based on target detection. A tape measure and a protractor were used to measure the actual position of the target. The position output by the algorithm was recorded five times and the average value was calculated as the algorithm output result. The results are shown in Table 2.

[0048] Table 2

[0049] Experimental results show that the calculation error of the algorithm for lateral deflection angle and longitudinal height difference is less than 10%, and the calculation error of straight-line distance is within 15%.

[0050] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for underwater target detection and positioning using a monocular camera based on motion inversion, characterized in that: include: Collect multiple frames of underwater images continuously; Target detection is performed on underwater images using a target detection algorithm. If a target is detected in a current underwater image frame, but a target is not detected in a next underwater image frame of the current underwater image frame, and the interval between the previous underwater image frame and the next underwater image frame is less than a preset time length, a detection missing frame compensation algorithm based on kinematic inversion is used to infer the target center coordinates of the next underwater image frame; if the next two underwater images also fail to detect a target, and the interval between the current underwater image frame and the next two underwater images is less than a preset time length, a detection missing frame compensation algorithm based on kinematic inversion is used to infer the target center coordinates of the next two underwater images until the underwater image detects a target; wherein the target detection result is a target label, a target confidence, a central horizontal coordinate and a central vertical coordinate of the target, and a pixel width and a pixel height of the target in the image; the actual physical width and actual physical height of the target are obtained according to the pixel width and pixel height of the target in the image, the change in the vertical coordinate of the target center in two adjacent frames, and the change in the depth of the vehicle; According to the central abscissa of the target in each frame of underwater image, the central abscissa and the lateral size of each frame of underwater image, and the lateral field of view of the camera, the lateral deviation angle of the center of the target relative to the center of the camera is obtained; According to the lateral size of the underwater image of the frame where the target is located, the width of the target in the image, the actual physical width of the target, and the lateral field of view of the camera, the straight-line distance from the center of the target to the center of the camera is obtained; According to the longitudinal size of the underwater image of the target frame, the longitudinal field of view of the camera, the central longitudinal coordinate of the target, the central longitudinal coordinate of the underwater image of the target frame, and the straight-line distance from the center of the target to the center of the camera, the longitudinal height between the center of the target and the center of the camera is obtained; The target is located according to the longitudinal height between the target center and the camera center, the straight-line distance from the target center to the camera center, and the lateral deviation angle of the target center relative to the camera center.

2. The method for underwater target detection and positioning using a monocular camera based on motion inversion according to claim 1, characterized in that: The steps of calculating the target center coordinates of the next frame of underwater image by the detection missing frame compensation algorithm based on kinematic inversion are as follows: Establish a ground coordinate system and a carrier coordinate system: the ground coordinate system takes the initial buoyancy center of the vehicle as the origin, the x-axis of the ground coordinate system points to the due north, the y-axis of the ground coordinate system is perpendicular to the x-axis and points to the due east, and the z-axis of the ground coordinate system is perpendicular to the x-axis and the y-axis to form a right-handed system; the carrier coordinate system takes the buoyancy center of the vehicle as the origin, the x-axis of the carrier coordinate system is along the length direction of the underwater vehicle and points forward, the y-axis of the carrier coordinate system is perpendicular to the x-axis and points to the starboard, and the z-axis of the carrier coordinate system is perpendicular to the x-axis and the y-axis to form a right-handed system; According to the pixels corresponding to the horizontal unit angle and the vertical unit angle of the current frame underwater image, the central horizontal coordinate and the central vertical coordinate of the target in the current frame underwater image, as well as the rolling angle, pitch angle, navigation angle, depth of the vehicle when collecting two adjacent frames of underwater images, and the distance from the center of the target in the current frame to the vehicle, the target center coordinates of the next frame underwater image are calculated.

3. The method for underwater target detection and positioning using a monocular camera based on motion inversion according to claim 2, characterized in that: The expression of the target center coordinates of the next frame of underwater image is: In the formula, Pd x Represents the corresponding pixel of the horizontal unit angle of the underwater image; Pd y Represents the corresponding pixel of the vertical unit angle of the underwater image; Indicates The central ordinate of the target in the frame underwater image; Indicates The central ordinate of the target in the frame underwater image; Indicates The central ordinate of the target in the frame underwater image; Indicates The central horizontal coordinate of the target in the frame underwater image; Indicates collection of Frame underwater image to collect the first The change in the rolling angle of the vehicle when framing underwater images; Indicates collection of Frame underwater image to collect the first The pitch angle change of the vehicle when framing underwater images; Ψ Indicates collection of Frame underwater image to collect the first The change in the yaw angle of the vehicle when capturing underwater images; Indicates collection of Frame underwater image to collect the first The depth change of the vehicle when framing underwater images; Indicates that the aircraft has arrived The distance to the center of the target in the frame underwater image.

4. The method for underwater target detection and positioning using a monocular camera based on motion inversion according to claim 1, characterized in that: Get the actual physical width and actual physical height of the target, including: In the formula, Indicates the actual physical width of the target; Indicates the actual physical height of the target; Indicates the pixel width of the target in the underwater image; Indicates the pixel height of the target in the underwater image; Indicates collection of Frame underwater image to collect the first The depth change of the vehicle when framing underwater images; Indicates collection of Frame underwater image to collect the first The change in the ordinate of the target center when framing the underwater image; Indicates the physical distance corresponding to the unit pixel.

5. The method for underwater target detection and positioning using a monocular camera based on motion inversion according to claim 1, characterized in that: Obtaining the lateral deviation angle of the target center relative to the camera center includes: In the formula, Indicates the lateral deviation angle of the target center relative to the camera center; Represents the central horizontal coordinate of the underwater image in the image coordinate system; Represents the horizontal coordinate of the target center in the image coordinate system; is the lateral field of view of the camera; is the horizontal pixel size of the underwater image.

6. The method for underwater target detection and positioning using a monocular camera based on motion inversion according to claim 1, characterized in that: Obtaining the straight-line distance from the center of the target to the center of the camera includes: In the formula, Indicates the straight-line distance from the center of the target to the center of the camera; is the lateral field of view of the camera; is the horizontal pixel size of the underwater image; Indicates the actual physical width of the target; Represents the pixel width of the target in the underwater image.

7. The method for underwater target detection and positioning using a monocular camera based on motion inversion according to claim 1, characterized in that: Getting the vertical height between the target center and the camera center includes: In the formula, Indicates the vertical height between the center of the target and the center of the camera head; Indicates the straight-line distance from the center of the target to the center of the camera; is the vertical field of view of the camera; is the vertical pixel size of the underwater image; y c Represents the central ordinate of the underwater image in the image coordinate system; y t Represents the vertical coordinate of the target center in the image coordinate system.

8. The method for underwater target detection and positioning using a monocular camera based on motion inversion according to claim 1, characterized in that: The target detection algorithm uses the YOLOv4-tiny algorithm.

Images