Target tracking method and system based on visual identification and underwater robot

Abstract

The invention discloses a target tracking method and system based on visual identification and an underwater robot, and relates to the technical field of underwater robots. According to the invention, automatic tracking and monitoring of an underwater target are realized by using a visual identification technology, operations such as an image identification technology of YOLOv5, a motion tracking technology of an optical flow and a Kalman filter, a target re-identification algorithm of deep learning, image enhancement and the like are specifically combined, and the intelligent level and the working efficiency of underwater operation are comprehensively improved. According to the method, the diver can be efficiently tracked in a complex underwater environment, and high recognition and tracking precision is ensured under the condition that the diver disappears or the motion trail is complex.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater robots, and in particular to a target tracking method and system based on visual recognition and an underwater robot. Background Art

[0002] At present, underwater robots have been widely used in deep-sea exploration, salvage and other tasks, usually to assist or replace divers. However, when underwater robots track divers in complex environments, they are limited by environmental factors and technical means, making it difficult to achieve high-precision, real-time target tracking and re-identification. Problems such as light attenuation, color distortion, and low contrast in underwater environments can affect the performance of image recognition systems, resulting in interruption or loss of target tracking.

[0003] Specifically, the main drawbacks of current underwater diver tracking methods include:

[0004] Poor image quality: Due to the rapid attenuation and scattering of light underwater, underwater images often have problems with low contrast and high noise, making target recognition difficult.

[0005] Insufficient motion tracking accuracy: Existing optical flow algorithms cannot effectively track the movement of divers in rapidly changing scenes, resulting in inaccurate motion trajectory capture.

[0006] Difficulty in target re-identification: After a diver disappears for a short time or is obscured, it is difficult to accurately re-identify the target, resulting in tracking failure.

[0007] Therefore, the existing technology still needs to be improved and developed. Summary of the invention

[0008] The technical problem to be solved by the present invention is that, in view of the above-mentioned defects of the prior art, a target tracking method, system and underwater robot based on visual recognition are provided, aiming to solve the problems of target recognition difficulty, inaccurate motion trajectory capture and difficulty in accurately re-identifying the target in the existing underwater diver tracking methods, which leads to tracking failure.

[0009] The technical solution adopted by the present invention to solve the problem is as follows:

[0010] In a first aspect, an embodiment of the present invention provides a target tracking method based on visual recognition, the method comprising:

[0011] Acquire an underwater video stream, and improve the image quality of each video frame in the video stream through an image enhancement operation;

[0012] The detection area corresponding to the target in each video frame is obtained through the lightweight YOLOv5 network;

[0013] Calculating the motion data of the target according to the local image corresponding to the detection area by using a local optical flow algorithm; wherein the motion data includes at least one of the position, motion trajectory and motion speed of the target;

[0014] The motion data is optimized by a Kalman filter; wherein, when the target is blocked or disappears, the Kalman filter outputs predicted motion data based on the motion data to keep tracking the target;

[0015] When the target reappears, the target is re-identified based on the video frame at the time of reproduction through the deep learning model, and the re-identification effect of the target is jointly judged by the Kalman filter and the deep learning model.

[0016] In one embodiment, the image enhancement operation includes:

[0017] At least one operation of illumination compensation, spectrum restoration, color correction, contrast adjustment, and dehazing is performed on each video frame through the back-illumination model.

[0018] In one embodiment, the local optical flow algorithm is used to:

[0019] Obtaining the fuzzy outline position of the target relative to the background based on the local image by using a corner detection algorithm, and generating tracking corner points according to the fuzzy outline position;

[0020] Performing sparse optical flow detection on each of the tracking corner points to obtain a motion vector of each of the tracking corner points;

[0021] The motion data of the target is calculated according to the motion vector of each tracking corner point.

[0022] In one embodiment, the local optical flow algorithm is also used to:

[0023] Determining the coordinates and displacement vector of each tracking corner point according to the motion vector of each tracking corner point;

[0024] The interference data is eliminated based on the coordinates and displacement vector of each tracking corner point by using the K-means clustering algorithm.

[0025] In one embodiment, the local optical flow algorithm is also used to:

[0026] After removing the interference data, the average motion vector is calculated according to the motion vectors of the remaining tracking corner points;

[0027] The moving speed of the target is calculated according to the average motion vector and the time difference between two frames.

[0028] In one embodiment, the Kalman filter is used to:

[0029] When the target is blocked or disappears, the predicted position and predicted speed corresponding to the next frame are output according to the historical position and historical speed of the target.

[0030] In one embodiment, the re-identifying the target based on the reproduced video frame by using the deep learning model, and jointly judging the re-identification effect of the target by using the Kalman filter and the deep learning model, includes:

[0031] Calculating the actual position of the target according to the reproduced video frame, and generating a first confidence indicator according to a deviation value between the actual position and the corresponding predicted position;

[0032] Extracting image segments corresponding to the target according to the reproduced video frames, and obtaining feature vectors corresponding to the image segments and historical image segments of the target respectively through the deep learning model;

[0033] Calculating the cosine similarity of the two feature vectors, and generating a second confidence index according to the cosine similarity;

[0034] The re-identification effect of the target is judged according to the first confidence index and the second confidence index.

[0035] In one embodiment, the method further comprises:

[0036] Analyzing the posture, size and position of the target through the video stream to obtain comprehensive analysis data of the target;

[0037] Acquiring the accumulated motion data of the target by using the local optical flow algorithm and the Kalman filter;

[0038] The speed and direction of the underwater robot are adjusted according to the comprehensive analysis data and the accumulated motion data.

[0039] In a second aspect, an embodiment of the present invention further provides a target tracking system based on visual recognition, the system comprising:

[0040] An image enhancement module, used to obtain an underwater video stream and improve the image quality of each video frame in the video stream through an image enhancement operation;

[0041] The target recognition module is used to obtain the detection area corresponding to the target in each video frame through a lightweight YOLOv5 network;

[0042] A motion capture module, used to calculate the motion data of the target according to the local image corresponding to the detected area by using a local optical flow algorithm; wherein the motion data includes at least one of the position, motion trajectory and motion speed of the target;

[0043] A motion optimization module, configured to optimize the motion data through a Kalman filter; wherein when the target is blocked or disappears, the Kalman filter outputs predicted motion data based on the motion data to keep tracking the target;

[0044] The target re-identification module is used to re-identify the target based on the video frame at the time of reproduction through the deep learning model when the target is reproduced, and to jointly judge the re-identification effect of the target through the Kalman filter and the deep learning model.

[0045] In a third aspect, an embodiment of the present invention further provides an underwater robot, comprising a memory and at least one processor; the memory stores a program; the program includes instructions for executing any of the visual recognition-based target tracking methods described above; and the processor is used to execute the program.

[0046] Beneficial effects of the invention: The embodiment of the invention uses visual recognition technology to achieve automatic tracking and monitoring of underwater targets, specifically combining YOLOv5 image recognition technology, optical flow and Kalman filter motion tracking technology, deep learning target re-identification algorithm and image enhancement operations, which comprehensively improves the intelligence level and work efficiency of underwater operations. The present invention can efficiently track divers in complex underwater environments, and ensure high recognition and tracking accuracy when divers disappear or their motion trajectories are complex. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 It is a flowchart of a target tracking method based on visual recognition provided by an embodiment of the present invention.

[0049] Figure 2 It is a schematic diagram of the internal modules of the target tracking system based on visual recognition provided by an embodiment of the present invention.

[0050] Figure 3 It is a principle block diagram of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The present invention discloses a target tracking method, system and underwater robot based on visual recognition. In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0052] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0053] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as herein.

[0054] In view of the above-mentioned defects of the prior art, the present invention provides a target tracking method based on visual recognition, which obtains an underwater video stream and improves the image quality of each video frame in the video stream through an image enhancement operation; obtains the detection area corresponding to the target in each video frame through a lightweight YOLOv5 network; calculates the motion data of the target according to the local image corresponding to the detection area through a local optical flow algorithm; wherein the motion data includes at least one data of the position, motion trajectory and motion speed of the target; optimizes the motion data through a Kalman filter; wherein, when the target is blocked or disappears, the Kalman filter outputs predicted motion data based on the motion data to keep tracking the target; when the target reappears, the deep learning model is used to re-identify the target based on the video frame at the time of reappearance, and the Kalman filter and the deep learning model are used to jointly judge the re-identification effect of the target. The present invention uses visual recognition technology to achieve automatic tracking and monitoring of underwater targets. It specifically combines YOLOv5 image recognition technology, optical flow and Kalman filter motion tracking technology, deep learning target re-identification algorithm, and image enhancement operations to comprehensively improve the intelligence level and work efficiency of underwater operations. The present invention can efficiently track divers in complex underwater environments, and ensure high recognition and tracking accuracy when divers disappear or their motion trajectories are complex.

[0055] like Figure 1 As shown, the method specifically comprises the following steps:

[0056] Step S100: Acquire an underwater video stream, and improve the image quality of each video frame in the video stream through an image enhancement operation.

[0057] Specifically, the underwater video stream can be recorded by a camera device specially designed for underwater environment. Such a camera device should be waterproof and able to work under low light conditions. In order to further improve the shooting effect, the camera device can be equipped with additional lighting equipment, such as underwater lights, to illuminate the shooting area. The camera equipment can be mounted on an underwater robot, and the desired scene can be captured by controlling the underwater robot. Due to the rapid attenuation and scattering of light underwater, underwater images often have problems of low contrast and high noise. In order to facilitate target recognition, this embodiment also needs to perform image enhancement operations on each video frame in the collected video stream to improve image quality, thereby improving the accuracy of subsequent data analysis and processing.

[0058] In one implementation, the image enhancement operation includes:

[0059] At least one operation of illumination compensation, spectrum restoration, color correction, contrast adjustment, and dehazing is performed on each video frame through the back-illumination model.

[0060] In order to facilitate the rapid optimization of video images, this embodiment pre-constructs a physical model based on the backlighting principle, namely, the backlighting model. The backlighting model analyzes the light attenuation and scattering effects through underwater image enhancement technology, restores the colors and details in the video image, improves the contrast and clarity of the video image, and thus effectively improves the image quality. Specifically, the image enhancement operations that can be implemented by the backlighting model include but are not limited to the following five categories: illumination compensation, spectral restoration, color correction, contrast adjustment, and defogging. In actual application, for each video image to be optimized, the backlighting model can perform one or more image enhancement operations on the video image in a targeted manner according to the quality problems existing in the video image. If different video images have different quality problems, the image enhancement operations performed may be different.

[0061] For example, the functions of the backlighting model include but are not limited to:

[0062] Lighting compensation and physical model calculation: Based on the principle of backlighting, the ambient light intensity and attenuation curve are estimated, and the exposure of low-brightness areas is improved through lighting compensation to make the image brightness more uniform;

[0063] Spectral restoration and color correction: restore the lost red spectral component, correct the color shift, and reconstruct the underwater color close to the real one by separating the attenuation characteristics of the RGB channels;

[0064] Adaptive contrast enhancement: Adjusts the contrast of underwater images to enhance edge information of objects and ensure the clarity of divers and key targets;

[0065] Dehazing based on transmission estimation: By estimating the transmission map and background light, suspended particles and fogging effects are removed to make the image more transparent and the layers more distinct.

[0066] Step S200: Obtain the detection area corresponding to the target in each video frame through a lightweight YOLOv5 network.

[0067] Specifically, YOLOv5 is a lightweight convolutional neural network suitable for deployment on edge devices such as underwater robots. This embodiment uses a lightweight YOLOv5 network as a target recognition model to identify the area where the specified target is located in each video frame, that is, to obtain the detection area of ​​the target. Taking the target as a diver as an example, the YOLOv5 network can detect the diver in real time from the video stream and provide accurate target recognition capabilities. In addition, the YOLOv5 network also has a small computational burden and can effectively process image data in an underwater environment. In practical applications, the model complexity can be further reduced through quantization and pruning techniques, so that it can be efficiently deployed on the robot.

[0068] Step S300, calculating the motion data of the target according to the local image corresponding to the detected area by using a local optical flow algorithm; wherein the motion data includes at least one of the position, motion trajectory and motion speed of the target.

[0069] Specifically, the collected video stream can be regarded as an image sequence. The local optical flow algorithm uses the principle of optical flow field to estimate the motion data of a specified target in the image sequence. The optical flow field is used to reflect the spatial distribution of the motion speed and direction of pixels between consecutive video frames. In order to improve the accuracy of target tracking, this embodiment uses a local optical flow algorithm to refine the detection area of ​​the target to obtain at least one data of the position, motion trajectory and motion speed of the target at different time points. Among them, the position of the target can be the coordinates of the target in each frame, which is obtained by tracking the displacement of a specific point over time; the motion trajectory of the target can be a motion trajectory curve formed by connecting the position of the target at each time point, which can intuitively display the moving path of the target; the motion speed can be calculated by the position change and time interval of the target between adjacent frames.

[0070] In one implementation, the local optical flow algorithm is used to:

[0071] Obtaining the fuzzy outline position of the target relative to the background based on the local image by using a corner detection algorithm, and generating tracking corner points according to the fuzzy outline position;

[0072] Performing sparse optical flow detection on each of the tracking corner points to obtain a motion vector of each of the tracking corner points;

[0073] The motion data of the target is calculated according to the motion vector of each tracking corner point.

[0074] Specifically, the local optical flow algorithm established in this embodiment is an optimization algorithm. First, the corner detection algorithm is used to extract the fuzzy outline of the target from the local image of the designated target detected by YOLOv5, and the key corner points for tracking are determined, that is, a number of tracking corner points are obtained. The sparse optical flow algorithm is applied to these tracking corner points to estimate their motion between consecutive frames. Among them, the input data of the sparse optical flow algorithm are the local images and corner point positions of the previous and next two frames, and the output data is the motion vector of the point. Through the motion vectors of all tracking corner points, the overall motion characteristics of the designated target can be inferred, thereby obtaining the motion data of the designated target.

[0075] In one implementation, the local optical flow algorithm is further used to:

[0076] Determining the coordinates and displacement vector of each tracking corner point according to the motion vector of each tracking corner point;

[0077] The interference data is eliminated based on the coordinates and displacement vector of each tracking corner point by using the K-means clustering algorithm.

[0078] Specifically, the local optical flow algorithm can also use the K-means clustering algorithm to remove interference data. The input data of the K-means clustering algorithm are point coordinates and displacement vectors. The point coordinates can be expressed in two-dimensional rectangular coordinates, and the displacement vector can be expressed in polar coordinates. The output data of the K-means clustering algorithm is the identified interference data. After the interference data is removed, the accuracy and reliability of the subsequent calculation results can be effectively improved.

[0079] In one implementation, the local optical flow algorithm is further used to:

[0080] After removing the interference data, the average motion vector is calculated according to the motion vectors of the remaining tracking corner points;

[0081] The moving speed of the target is calculated according to the average motion vector and the time difference between two frames.

[0082] In practical applications, due to noise, false detection or other factors, the motion vector of the tracking corner point may contain some inaccurate or abnormal data. Eliminating these interfering data can improve the accuracy and reliability of subsequent calculations. Specifically, after eliminating the interfering data, the remaining motion vector can better reflect the actual movement of the target. By averaging these remaining motion vectors, an average motion vector representing the overall movement of the target can be obtained. This average motion vector takes into account the motion information of multiple corner points and is therefore more robust. The average motion vector can reflect the average displacement of the target between two frames, and the time difference between the two frames can reflect the time interval of the displacement. The average motion vector can be combined with the time difference between the two frames to calculate the target's movement speed, thereby achieving a quantitative description and analysis of the target's movement. Since the movement speed is the ratio of displacement to time, the length (or size) of the average motion vector can be divided by the time difference between the two frames to obtain the target's movement speed.

[0083] For example, assuming the target is a diver, after removing interference, the average motion vector is calculated and divided by the time difference between two frames to obtain the diver's speed.

[0084] Step S400, optimizing the motion data through a Kalman filter; wherein, when the target is blocked or disappears, the Kalman filter outputs predicted motion data based on the motion data to keep tracking the target.

[0085] Specifically, in order to improve the accuracy of capturing the target's motion trajectory, this embodiment combines the local optical flow algorithm with the Kalman filter. The local optical flow algorithm analyzes the pixel motion changes between video frames to capture the target's motion trajectory, while the Kalman filter uses a prediction model and a measurement update mechanism to optimize the motion trajectory, reduce noise interference, and ensure accurate tracking of the target's dynamic changes.

[0086] In one implementation, the Kalman filter is used to:

[0087] When the target is blocked or disappears, the predicted position and predicted speed corresponding to the next frame are output according to the historical position and historical speed of the target.

[0088] Specifically, the Kalman filter can make the best estimate of the system state in the case of many uncertainties, so this embodiment applies the Kalman filter to the target tracking process. In actual application, if there is no observation data of the target in the current frame, the Kalman filter cannot rely on new observation data to update the state, and it is necessary to rely on the internal prediction mechanism to update the state, that is, to predict the state of the current frame (predicted position and predicted speed) based on the state of the previous frame (historical position and historical speed). If there is available observation data of the specified target in the current frame, the first method is to use the new observation data to calculate the state of the target, where the state includes position and speed, and the position can be expressed in coordinates; the second method is to predict the state of the current frame based on the state of the previous frame through the Kalman filter, and fuse the new observation data with the prediction data of the Kalman filter, and calculate the state of the target according to the fusion result. The second method takes into account the uncertainty of the mobile detection device (such as the accuracy of the camera) and the uncertainty of the Kalman filter, and can generate a more accurate state estimate.

[0089] For example, the input data of the Kalman filter are the real coordinates (x, y) and velocity v of the target, where the influence of the relative velocity and displacement of the mobile detection device (such as an underwater robot) has been deducted. Assume that the measured real coordinates and velocity are z = [x, y, u x , ν y ] T , where z is the observed data, v x 、v y are the speeds in the x and y directions respectively, and T is the time. The Kalman filter outputs the predicted coordinates and speed of the next frame based on the input data, that is, the predicted state vector The actual state vector is X tThe predicted coordinates can be used to infer the trajectory when the target disappears temporarily. For example, when the camera cannot obtain the image of the target due to environmental interference (such as a large number of steam drums or dust obstruction), the Kalman filter can predict the target position based on the current speed and acceleration information.

[0090] Step S500: When the target reappears, the target is re-identified based on the video frame at the time of reproduction by using the deep learning model, and the re-identification effect of the target is jointly determined by the Kalman filter and the deep learning model.

[0091] During underwater operations, when the target is obscured or disappears for a short time, this embodiment will re-identify the target through the deep learning model and continue tracking to ensure that the tracking task is not interrupted. When the target appears again, this embodiment will comprehensively judge the effect of target re-identification by analyzing the output data of the Kalman filter and the deep learning model. Among them, the input data of the deep learning model is the local image when the target reappears, and the output data is the feature vector extracted based on the local image. The feature vector can be used to calculate the cosine similarity, and the numerical value of the cosine similarity can be used to judge the quality of the re-identification effect.

[0092] In one implementation, the re-identifying the target based on the reproduced video frame by using the deep learning model, and jointly judging the re-identification effect of the target by using the Kalman filter and the deep learning model, includes:

[0093] Calculating the actual position of the target according to the reproduced video frame, and generating a first confidence indicator according to a deviation value between the actual position and the corresponding predicted position;

[0094] Extracting image segments corresponding to the target according to the reproduced video frames, and obtaining feature vectors corresponding to the image segments and historical image segments of the target respectively through the deep learning model;

[0095] Calculating the cosine similarity of the two feature vectors, and generating a second confidence index according to the cosine similarity;

[0096] The re-identification effect of the target is judged according to the first confidence index and the second confidence index.

[0097] Specifically, the target re-identification effect is mainly determined based on the first confidence index and the second confidence index. The first confidence index is calculated based on the deviation between the observed actual position and the predicted position; the second confidence index is to extract the feature vector of the image segment containing the target when the target is reproduced, and calculate the cosine similarity between it and the feature vector of the image segment of the target collected previously. The size of the cosine similarity can reflect the similarity of the two feature vectors, and then determine whether the two image segments contain the same target.

[0098] For example, when the target appears again, the deviation between the actual observed position and the predicted position is used as one of the confidence levels of target re-identification. The calculation formula for the deviation is:

[0099]

[0100] Among them, ∈ is the deviation value, is the predicted state vector, X t+1 is the actual state vector, which contains coordinates and velocity.

[0101] The feature vector of the local image of the target is calculated through a deep learning model, and the cosine similarity calculated based on the feature vector is used as another confidence indicator.

[0102] In one implementation, the method further includes:

[0103] Analyzing the posture, size and position of the target through the video stream to obtain comprehensive analysis data of the target;

[0104] Acquiring the accumulated motion data of the target by using the local optical flow algorithm and the Kalman filter;

[0105] The speed and direction of the underwater robot are adjusted according to the comprehensive analysis data and the accumulated motion data.

[0106] Specifically, by analyzing the target's posture, size, and position, the relative distance and orientation between the target and the mobile detection device can be inferred. Combined with the optical flow and Kalman filter accumulated motion data, the speed and direction of the mobile detection device can be accurately adjusted to maintain the appropriate tracking distance and orientation.

[0107] For example, if the mobile detection device is an underwater robot and the target is a diver, in a deep-sea exploration mission, the underwater robot detects the diver by deploying YOLOv5 and combines optical flow and Kalman filtering to capture the motion trajectory. At the same time, the image enhancement technology based on the backlighting model effectively improves the image quality, allowing the underwater robot to adjust the speed and direction in real time and continue to follow the diver.

[0108] The advantages of the present invention are:

[0109] 1. Image enhancement technology based on physical models improves the clarity and contrast of underwater images and enhances the accuracy of target detection.

[0110] 2. The combination of optical flow algorithm and Kalman filter improves the accuracy of capturing motion trajectory and can accurately track even in complex motion scenes.

[0111] 3. The target re-identification capability of deep learning ensures accurate re-identification of targets after occlusion or disappearance, improving the robustness of the system.

[0112] 4. Infer the distance and direction of the diver in real time, and dynamically adjust the speed and direction of the robot according to the motion trajectory to ensure the stability and accuracy of tracking.

[0113] Based on the above embodiments, the present invention also provides a target tracking system based on visual recognition, such as Figure 2 As shown, the system comprises:

[0114] An image enhancement module 01 is used to obtain an underwater video stream and improve the image quality of each video frame in the video stream through an image enhancement operation;

[0115] The target recognition module 02 is used to obtain the detection area corresponding to the target in each video frame through a lightweight YOLOv5 network;

[0116] The motion capture module 03 is used to calculate the motion data of the target according to the local image corresponding to the detected area by using a local optical flow algorithm; wherein the motion data includes at least one of the position, motion trajectory and motion speed of the target;

[0117] A motion optimization module 04, configured to optimize the motion data through a Kalman filter; wherein, when the target is blocked or disappears, the Kalman filter outputs predicted motion data based on the motion data to keep tracking the target;

[0118] The target re-identification module 05 is used to re-identify the target based on the video frame at the time of reproduction through the deep learning model when the target is reproduced, and to jointly determine the re-identification effect of the target through the Kalman filter and the deep learning model.

[0119] Based on the above embodiment, the present invention further provides a terminal, whose principle block diagram can be shown as follows: Figure 3As shown. The terminal includes a processor, a memory, a network interface, and a display screen connected via a system bus. Among them, the processor of the terminal is used to provide computing and control capabilities. The memory of the terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a target tracking method based on visual recognition is implemented. The display screen of the terminal can be a liquid crystal display screen or an electronic ink display screen.

[0120] Those skilled in the art will understand that Figure 3 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the terminal to which the scheme of the present invention is applied. The specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0121] In one implementation, the memory of the terminal stores one or more programs, and is configured to be executed by one or more processors. The one or more programs include instructions for performing a target tracking method based on visual recognition.

[0122] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0123] In summary, the present invention discloses a target tracking method, system and underwater robot based on visual recognition, and relates to the technical field of underwater robots. The method obtains an underwater video stream and improves the image quality of each video frame in the video stream through an image enhancement operation; obtains the detection area corresponding to the target in each video frame through a lightweight YOLOv5 network; calculates the motion data of the target according to the local image corresponding to the detection area through a local optical flow algorithm; wherein the motion data includes at least one data of the position, motion trajectory and motion speed of the target; optimizes the motion data through a Kalman filter; wherein, when the target is blocked or disappears, the Kalman filter outputs predicted motion data based on the motion data to keep tracking the target; when the target reappears, the deep learning model is used to re-identify the target based on the video frame at the time of reappearance, and the re-identification effect of the target is jointly judged by the Kalman filter and the deep learning model. The present invention uses visual recognition technology to achieve automatic tracking and monitoring of underwater targets. It specifically combines YOLOv5 image recognition technology, optical flow and Kalman filter motion tracking technology, deep learning target re-identification algorithm, and image enhancement operations to comprehensively improve the intelligence level and work efficiency of underwater operations. The present invention can efficiently track divers in complex underwater environments, and ensure high recognition and tracking accuracy when divers disappear or their motion trajectories are complex.

[0124] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A target tracking method based on visual recognition, characterized in that: The method comprises: Acquire an underwater video stream, and improve the image quality of each video frame in the video stream through an image enhancement operation; The detection area corresponding to the target in each video frame is obtained through the lightweight YOLOv5 network; Calculating the motion data of the target according to the local image corresponding to the detection area by using a local optical flow algorithm; wherein the motion data includes at least one of the position, motion trajectory and motion speed of the target; The motion data is optimized by a Kalman filter; wherein, when the target is blocked or disappears, the Kalman filter outputs predicted motion data based on the motion data to keep tracking the target; When the target reappears, the target is re-identified based on the video frame at the time of reproduction through the deep learning model, and the re-identification effect of the target is jointly judged by the Kalman filter and the deep learning model.

2. The target tracking method based on visual recognition according to claim 1, characterized in that: The image enhancement operation includes: At least one operation of illumination compensation, spectrum restoration, color correction, contrast adjustment, and dehazing is performed on each video frame through the back-illumination model.

3. The target tracking method based on visual recognition according to claim 1, characterized in that: The local optical flow algorithm is used to: Obtaining the fuzzy outline position of the target relative to the background based on the local image by using a corner detection algorithm, and generating tracking corner points according to the fuzzy outline position; Performing sparse optical flow detection on each of the tracking corner points to obtain a motion vector of each of the tracking corner points; The motion data of the target is calculated according to the motion vector of each tracking corner point.

4. The target tracking method based on visual recognition according to claim 3 is characterized in that: The local optical flow algorithm is also used to: Determining the coordinates and displacement vector of each tracking corner point according to the motion vector of each tracking corner point; The interference data is eliminated based on the coordinates and displacement vector of each tracking corner point by using the K-means clustering algorithm.

5. The target tracking method based on visual recognition according to claim 4, characterized in that: The local optical flow algorithm is also used to: After removing the interference data, the average motion vector is calculated according to the motion vectors of the remaining tracking corner points; The moving speed of the target is calculated according to the average motion vector and the time difference between two frames.

6. The target tracking method based on visual recognition according to claim 5, characterized in that: The Kalman filter is used to: When the target is blocked or disappears, the predicted position and predicted speed corresponding to the next frame are output according to the historical position and historical speed of the target.

7. The target tracking method based on visual recognition according to claim 6, characterized in that: The re-identifying the target based on the reproduced video frame by the deep learning model, and jointly judging the re-identification effect of the target by the Kalman filter and the deep learning model, includes: Calculating the actual position of the target according to the reproduced video frame, and generating a first confidence indicator according to a deviation value between the actual position and the corresponding predicted position; Extracting image segments corresponding to the target according to the reproduced video frames, and obtaining feature vectors corresponding to the image segments and historical image segments of the target respectively through the deep learning model; Calculating the cosine similarity of the two feature vectors, and generating a second confidence index according to the cosine similarity; The re-identification effect of the target is judged according to the first confidence index and the second confidence index.

8. The target tracking method based on visual recognition according to claim 1, characterized in that: The method further comprises: Analyzing the posture, size and position of the target through the video stream to obtain comprehensive analysis data of the target; Acquiring the accumulated motion data of the target by using the local optical flow algorithm and the Kalman filter; The speed and direction of the underwater robot are adjusted according to the comprehensive analysis data and the accumulated motion data.

9. A target tracking system based on visual recognition, characterized in that: The system comprises: An image enhancement module, used to obtain an underwater video stream and improve the image quality of each video frame in the video stream through an image enhancement operation; The target recognition module is used to obtain the detection area corresponding to the target in each video frame through a lightweight YOLOv5 network; A motion capture module, used to calculate the motion data of the target according to the local image corresponding to the detected area by using a local optical flow algorithm; wherein the motion data includes at least one of the position, motion trajectory and motion speed of the target; A motion optimization module, configured to optimize the motion data through a Kalman filter; wherein when the target is blocked or disappears, the Kalman filter outputs predicted motion data based on the motion data to keep tracking the target; The target re-identification module is used to re-identify the target based on the video frame at the time of reproduction through the deep learning model when the target is reproduced, and to jointly judge the re-identification effect of the target through the Kalman filter and the deep learning model.

10. An underwater robot, characterized in that: The underwater robot includes a memory and at least one processor; the memory stores a program; the program contains instructions for executing the target tracking method based on visual recognition as described in any one of claims 1-8; and the processor is used to execute the program.

Images