An underwater robot target tracking method and system based on machine vision

Abstract

The present invention relates to an underwater robot target tracking method and system based on machine vision, belonging to the technical field of underwater robots. The present invention constructs an object recognition model based on a deep neural network, and uses the object recognition model to recognize the target tracking image information, obtains the action trajectory information of the object within a preset time, and finally constructs the path information of the underwater robot within a preset time according to the action trajectory information of the object within the preset time, controls the underwater robot according to the path information of the underwater robot within the preset time, and monitors the real-time operation status of each underwater robot. The present invention can collect and process underwater images by integrating the environmental characteristics of the water environment, the DCP algorithm, and the genetic algorithm, enabling image enhancement of the original images, improving the tracking accuracy of the robot for the object, and realizing long-term tracking of underwater objects.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater robots, and in particular, to an underwater robot target tracking method and system based on machine vision. Background Art

[0002] The recognition of underwater objects is an important basis for underwater robots to operate in the underwater environment and is also a core issue in underwater robot technology. The robot needs to be able to detect and recognize objects in a complex background environment. After recognizing the object, the underwater robot can calculate the pose of the robot relative to the object, automatically plan a path, control the robot to move to a certain position in front of the object, and finally complete the corresponding operation. The main observation means of an underwater robot based on machine vision is an underwater camera. However, water will scatter and absorb light, which will cause serious weakening of underwater images and is not conducive to target tracking. Summary of the Invention

[0003] The present invention overcomes the deficiencies of the prior art and provides an underwater robot target tracking method and system based on machine vision.

[0004] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0005] In a first aspect of the present invention, an underwater robot target tracking method based on machine vision is provided, including the following steps:

[0006] Obtain the environmental feature data information in the target water area, set the imaging parameters of the imaging device according to the environmental feature data information in the target water area, and obtain the original image information collected by the underwater robot based on the imaging parameters of the imaging device;

[0007] Introduce the DCP algorithm and the genetic algorithm, perform feature processing on the original image information collected by the underwater robot according to the DCP algorithm and the genetic algorithm, and obtain the target tracking image information;

[0008] Construct an object recognition model based on a deep neural network, and identify the target tracking image information through the object recognition model to obtain the action trajectory information of the object within a preset time;

[0009] Construct the path information of the underwater robot within a preset time according to the action trajectory information of the object within a preset time, control the underwater robot according to the path information of the underwater robot within a preset time, and monitor the real-time operation status of each underwater robot.

[0010] Further, in this method, environmental feature data information in the target water area is obtained, and the imaging parameters of the imaging device are set according to the environmental feature data information in the target water area. The original image information collected by the underwater robot is obtained based on the imaging parameters of the imaging device, which specifically includes:

[0011] The image clarity feature data information under each water environment feature and imaging parameter is obtained through big data. A graph neural network is introduced, and the image clarity feature data information under each water environment feature and imaging parameter is input into the graph neural network;

[0012] The water environment feature is used as the first graph node of the graph neural network, the imaging parameter is used as the second graph node of the graph neural network, and the image clarity feature data information is used as the third graph node of the graph neural network. A topological structure graph is constructed based on the first graph node, the second graph node, and the third graph node;

[0013] The relevant adjacency matrix is obtained based on the topological structure graph, and a knowledge graph is constructed. The relevant adjacency matrix is input into the knowledge graph for storage, the environmental feature data information in the target water area is obtained, and the environmental feature data information in the target water area is imported into the knowledge graph for data matching;

[0014] Through data matching, the image clarity feature data information corresponding to each imaging parameter under the environmental feature data information in the current target water area is obtained, the imaging parameter corresponding to the maximum image clarity feature data information is obtained, and the imaging device is controlled to perform imaging acquisition based on the image clarity feature data information, and the original image information collected by the underwater robot is obtained.

[0015] Further, in this method, the DCP algorithm and the genetic algorithm are introduced. The original image information collected by the underwater robot is subjected to feature processing according to the DCP algorithm and the genetic algorithm to obtain the target tracking image information, which specifically includes:

[0016] The DCP algorithm is introduced. The image is divided into images of three RGB channels, the mean value of each channel image is calculated, the image with the maximum mean value and the image with the minimum mean value are obtained, and the mean difference between the image with the maximum mean value and the image with the minimum mean value is calculated;

[0017] Based on the mean value output, a depth-of-field image is obtained. When processing the first frame, the pixels with a brightness of the top 0.1% are counted on the depth-of-field image. At the same time, the mean values of the three RGB channels are calculated at the positions of the same pixels in the original image, and the mean values of the three RGB channels are used as the water body background color;

[0018] Initialize the depth-of-field image based on the DCP algorithm to obtain the transmission map, introduce the genetic algorithm, set the depth-of-field effect characteristic index of the transmission map, initialize the adjustment parameters of the transmission map, and adjust the transmission map based on the adjustment parameters of the transmission map to obtain the depth-of-field effect characteristics of the transmission map;

[0019] When the depth-of-field effect characteristic of the transmission map is not greater than the depth-of-field effect characteristic index of the transmission map, readjust the adjustment parameters of the transmission map until the depth-of-field effect characteristic of the transmission map is greater than the depth-of-field effect characteristic index of the transmission map, output the final transmission map, and splice and restore the image in combination with the final transmission map, the original image, and the water body background color to obtain the target tracking image information.

[0020] Furthermore, in this method, a target recognition model is constructed based on a deep neural network, specifically including:

[0021] Obtain target image data through big data, randomly select a preset number of target image data, construct a training set according to the preset number of target image data, obtain the type of the target, and divide the training set based on the type of the target to obtain a training subset for each target type;

[0022] Calculate the Jaccard similarity coefficient between the training subsets of different target types, calculate the Jaccard distance between the training subsets of different target types based on the Jaccard similarity coefficient, and determine whether the Jaccard distance is greater than the preset Jaccard distance threshold;

[0023] When the Jaccard distance is greater than the preset Jaccard distance threshold, use the training subset of the target type as the final training subset of the target type. When the Jaccard distance is not greater than the preset Jaccard distance threshold, reselect the sample data in the target type until the Jaccard distance is greater than the preset Jaccard distance threshold;

[0024] Construct a target recognition model based on a deep neural network, and input the final training subset of the target type into the target recognition model for training in sequence. When the convergence value of the loss function of the target recognition model reaches the preset value, output the target recognition model.

[0025] Furthermore, in this method, identify the target tracking image information through the target recognition model to obtain the action trajectory information of the target within a preset time, specifically including:

[0026] Continuously collect the target tracking image information at several time stamps through the underwater robot, and input the target tracking image information into the target recognition model for identification to determine whether there is a target in the target tracking image information;

[0027] When there is a target object in the target tracking image information, the position information of the target object at each timestamp is identified and obtained, and the action trajectory information of the target object within a preset time is constructed based on the position information of the target object at each timestamp.

[0028] Further, in this method, the path information of the underwater robot within a preset time is constructed based on the action trajectory information of the target object within a preset time, and the underwater robot is controlled according to the path information of the underwater robot within a preset time, specifically including:

[0029] Set a preset tracking distance threshold, obtain the position information of the target object at the current timestamp based on the action trajectory information of the target object within a preset time, and obtain the position information of the current underwater robot;

[0030] Calculate the Euclidean distance value between the position information of the target object at the current timestamp and the position information of the current underwater robot, and determine whether the Euclidean distance value between the position information of the target object at the current timestamp and the position information of the current underwater robot is greater than the tracking distance threshold;

[0031] When the Euclidean distance value between the position information of the target object at the current timestamp and the position information of the current underwater robot is greater than the tracking distance threshold, adjust the position information of the current underwater robot until the Euclidean distance value is not greater than the tracking distance threshold;

[0032] When the Euclidean distance value between the position information of the target object at the current timestamp and the position information of the current underwater robot is not greater than the tracking distance threshold, combine the timestamp to construct the path information of the underwater robot within a preset time, and control the underwater robot according to the path information of the underwater robot within a preset time.

[0033] The second aspect of the present invention provides an underwater robot target tracking system based on machine vision. The system includes a memory and a processor. The memory includes an underwater robot target tracking method program based on machine vision. When the underwater robot target tracking method program based on machine vision is executed by the processor, the following steps are implemented:

[0034] Obtain the environmental feature data information in the target water area, set the camera parameters of the camera device according to the environmental feature data information in the target water area, and obtain the original image information collected by the underwater robot based on the camera parameters of the camera device;

[0035] Introduce the DCP algorithm and the genetic algorithm, perform feature processing on the original image information collected by the underwater robot according to the DCP algorithm and the genetic algorithm, and obtain the target tracking image information;

[0036] Construct an object recognition model based on a deep neural network, and use the object recognition model to recognize the target tracking image information, so as to obtain the action trajectory information of the object within a preset time;

[0037] Construct the path information of the underwater robot within a preset time according to the action trajectory information of the object within a preset time, control the underwater robot according to the path information of the underwater robot within a preset time, and monitor the real-time operation status of each underwater robot.

[0038] The third aspect of the present invention provides a computer-readable storage medium, which includes a program for the method of underwater robot target tracking based on machine vision. When the program for the method of underwater robot target tracking based on machine vision is executed by a processor, the steps of any of the methods of underwater robot target tracking based on machine vision are implemented.

[0039] The present invention solves the defects in the background technology, and the present invention has the following beneficial effects:

[0040] The present invention obtains the environmental characteristic data information in the target water area, sets the shooting parameters of the camera device according to the environmental characteristic data information in the target water area, obtains the original image information collected by the underwater robot based on the shooting parameters of the camera device, and then introduces the DCP algorithm and the genetic algorithm. The original image information collected by the underwater robot is processed by the DCP algorithm and the genetic algorithm to obtain the target tracking image information. Thus, an object recognition model is constructed based on a deep neural network, and the target tracking image information is recognized by the object recognition model to obtain the action trajectory information of the object within a preset time. Finally, the path information of the underwater robot within a preset time is constructed according to the action trajectory information of the object within a preset time, the underwater robot is controlled according to the path information of the underwater robot within a preset time, and the real-time operation status of each underwater robot is monitored. The present invention can collect and process underwater images by integrating the environmental characteristics of the water environment, the DCP algorithm, and the genetic algorithm, enabling image enhancement of the original images, improving the tracking accuracy of the robot for objects, and achieving long-term tracking of underwater objects. Description of the Drawings

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0042] Figure 1Shows the overall method flowchart of the underwater robot target tracking method based on machine vision;

[0043] Figure 2 Shows the first method flowchart of the underwater robot target tracking method based on machine vision;

[0044] Figure 3 Shows the second method flowchart of the underwater robot target tracking method based on machine vision;

[0045] Figure 4 Shows the system block diagram of the underwater robot target tracking system based on machine vision. Detailed implementation manners

[0046] In order to more clearly understand the above objects, features and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation manners. It should be noted that, without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other.

[0047] In the following description, many specific details are set forth in order to fully understand the present invention. However, the present invention can also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0048] As Figure 1 shown, the first aspect of the present invention provides an underwater robot target tracking method based on machine vision, including the following steps:

[0049] S102: Obtain the environmental feature data information in the target water area, set the camera parameters of the camera device according to the environmental feature data information in the target water area, and obtain the original image information collected by the underwater robot based on the camera parameters of the camera device;

[0050] S104: Introduce the DCP algorithm and the genetic algorithm, perform feature processing on the original image information collected by the underwater robot according to the DCP algorithm and the genetic algorithm, and obtain the target tracking image information;

[0051] S106: Construct a target recognition model based on the deep neural network, and identify the target tracking image information through the target recognition model to obtain the action trajectory information of the target within a preset time;

[0052] S108: Construct the path information of the underwater robot within a preset time according to the action trajectory information of the target within a preset time, control the underwater robot according to the path information of the underwater robot within a preset time, and monitor the real-time operation status of each underwater robot.

[0053] It should be noted that the present invention can collect and process underwater images by integrating the environmental characteristics of the water environment, the DCP algorithm, and the genetic algorithm, enabling image enhancement of the original images, improving the tracking accuracy of the robot for the target object, and achieving long-term tracking of the underwater target object.

[0054] As Figure 2 shown, further, in this method, environmental characteristic data information in the target water area is obtained, and the shooting parameters of the imaging device are set according to the environmental characteristic data information in the target water area. Based on the shooting parameters of the imaging device, the original image information collected by the underwater robot is obtained, specifically including:

[0055] S202: Obtain the image sharpness characteristic data information under each water environment characteristic and shooting parameter through big data, introduce a graph neural network, and input the image sharpness characteristic data information under each water environment characteristic and shooting parameter into the graph neural network;

[0056] S204: Use the water environment characteristic as the first graph node of the graph neural network, the shooting parameter as the second graph node of the graph neural network, and the image sharpness characteristic data information as the third graph node of the graph neural network, and construct a topological structure diagram based on the first graph node, the second graph node, and the third graph node;

[0057] S206: Obtain the relevant adjacency matrix based on the topological structure diagram, construct a knowledge graph, input the relevant adjacency matrix into the knowledge graph for storage, obtain the environmental characteristic data information in the target water area, and import the environmental characteristic data information in the target water area into the knowledge graph for data matching;

[0058] S208: Through data matching, obtain the image sharpness characteristic data information corresponding to each shooting parameter under the environmental characteristic data information in the current target water area, obtain the shooting parameter corresponding to the maximum image sharpness characteristic data information, and control the imaging device to perform imaging acquisition based on the image sharpness characteristic data information to obtain the original image information collected by the underwater robot.

[0059] It should be noted that the water environment characteristics include data such as the light transmission ability in water, the turbidity of the water body, and the color of the water body. Since different shooting parameters will result in inconsistent image sharpness in different water body environments, this method can obtain the shooting parameter corresponding to the maximum image sharpness characteristic data information, enabling the underwater robot to use the optimal shooting parameter for imaging under the current water environment characteristics, thereby being able to obtain the optimal original image, reducing the data processing volume of the tracking image, and improving the response speed of the underwater robot during tracking.

[0060] Further, in this method, the DCP algorithm and the genetic algorithm are introduced. The original image information collected by the underwater robot is processed for features according to the DCP algorithm and the genetic algorithm to obtain the target tracking image information, specifically including:

[0061] Introduce the DCP algorithm, divide the image into images of three RGB channels, calculate the mean value of each channel image, obtain the image with the largest mean value and the image with the smallest mean value, and calculate the mean difference between the image with the largest mean value and the image with the smallest mean value;

[0062] Obtain the depth-of-field image based on the mean value output, and when processing the first frame, count the pixels with a brightness of the top 0.1% on the depth-of-field image. At the same time, calculate the mean values of the three RGB channels at the positions of the same pixels in the original image, and use the mean values of the three RGB channels as the water body background color;

[0063] Perform initialization processing on the depth-of-field image based on the DCP algorithm to obtain the transmission map, introduce the genetic algorithm, set the depth-of-field effect characteristic index of the transmission map, initialize the adjustment parameters of the transmission map, and adjust the transmission map based on the adjustment parameters of the transmission map to obtain the depth-of-field effect characteristics of the transmission map;

[0064] When the depth-of-field effect characteristic of the transmission map is not greater than the depth-of-field effect characteristic index of the transmission map, re-adjust the adjustment parameters of the transmission map until the depth-of-field effect characteristic of the transmission map is greater than the depth-of-field effect characteristic index of the transmission map, output the final transmission map, and splice and restore the image in combination with the final transmission map, the original image, and the water body background color to obtain the target tracking image information.

[0065] It should be noted that the DCP algorithm is a classic dehazing algorithm. In fact, in the images collected in the underwater environment, they are similar to the characteristics of fog, and there may be an over-compensation phenomenon in the classic DCP algorithm, resulting in unclear processed images; in this method, the depth-of-field map is used instead of the dark channel map. The essence of the traditional DCP algorithm using the dark channel is to obtain the depth-of-field information of the environment to estimate the atmospheric background color, and the transmission map is obtained through the depth-of-field effect of the dark channel image. In this paper, the depth-of-field map is obtained from the difference between the bright and dark channels, which reflects the depth-of-field effect of the underwater environment and can be used to estimate the water body background color instead of the dark channel image. The depth-of-field effect characteristics (image sharpness) of the transmission map are evaluated through AI technology or machine learning technology, and then optimized through the genetic algorithm, so that the depth-of-field effect characteristics of the transmission map are greater than the depth-of-field effect characteristic index of the transmission map, so as to adaptively adjust the processing effect of the target tracking image and improve the tracking accuracy of the underwater robot.

[0066] As Figure 3 shown, further, in this method, a target object recognition model is constructed based on a deep neural network, specifically including:

[0067] S302: Obtain the target object image data through big data, randomly select a preset number of target object image data, construct a training set based on the preset number of target object image data, obtain the type of the target object, divide the training set based on the type of the target object, and obtain the training subset of each target object type;

[0068] S304: Calculate the Jaccard similarity coefficient between the training subsets of different target object types, calculate the Jaccard distance between the training subsets of different target object types based on the Jaccard similarity coefficient, and determine whether the Jaccard distance is greater than the preset Jaccard distance threshold;

[0069] S306: When the Jaccard distance is greater than the preset Jaccard distance threshold, use the training subset of the target object type as the final training subset of the target object type. When the Jaccard distance is not greater than the preset Jaccard distance threshold, re-select the sample data in the target object type until the Jaccard distance is greater than the preset Jaccard distance threshold;

[0070] S308: Construct a target object recognition model based on a deep neural network, input the final training subset of the target object type into the target object recognition model for training in sequence. When the convergence value of the loss function of the target object recognition model reaches the preset value, output the target object recognition model.

[0071] It should be noted that the target objects include fish, shrimp, underwater plants, etc. Since there is one or more types of target objects in an image of the training set, when classifying the training set, there may be two target object categories in one image. At this time, when training the target object recognition model, the features of multiple target object types will cause certain interference to the target object recognition model. When the Jaccard distance is greater than the preset Jaccard distance threshold, it means that the discrimination degree between the training sets meets the predetermined requirements, that is, there is no sample data of two target object categories between the training sets. Through this method, the training data during the training of the target object recognition model can be further optimized, thereby improving the recognition accuracy of the target recognition model.

[0072] Furthermore, in this method, the target object recognition model is used to recognize the target tracking image information to obtain the action trajectory information of the target object within a preset time, specifically including:

[0073] Continuously collect the target tracking image information of several timestamps through an underwater robot, and input the target tracking image information into the target object recognition model for recognition to determine whether there is a target object in the target tracking image information;

[0074] When there is a target object in the target tracking image information, the position information of the target object at each timestamp is recognized and obtained, and the action trajectory information of the target object within a preset time is constructed based on the position information of the target object at each timestamp.

[0075] Further, in this method, the path information of the underwater robot within a preset time is constructed based on the action trajectory information of the target object within a preset time, and the underwater robot is controlled according to the path information of the underwater robot within a preset time, specifically including:

[0076] Set a preset tracking distance threshold, obtain the position information of the target object at the current timestamp based on the action trajectory information of the target object within a preset time, and obtain the position information of the current underwater robot;

[0077] Calculate the Euclidean distance value between the position information of the target object at the current timestamp and the position information of the current underwater robot, and determine whether the Euclidean distance value between the position information of the target object at the current timestamp and the position information of the current underwater robot is greater than the tracking distance threshold;

[0078] When the Euclidean distance value between the position information of the target object at the current timestamp and the position information of the current underwater robot is greater than the tracking distance threshold, adjust the position information of the current underwater robot until the Euclidean distance value is not greater than the tracking distance threshold;

[0079] When the Euclidean distance value between the position information of the target object at the current timestamp and the position information of the current underwater robot is not greater than the tracking distance threshold, combine the timestamp to construct the path information of the underwater robot within a preset time, and control the underwater robot according to the path information of the underwater robot within a preset time.

[0080] It should be noted that the target object can be continuously tracked by this method.

[0081] In addition, this method may further include the following steps:

[0082] Obtain the communication performance change characteristic data of the communication device configured on the underwater robot, construct a communication device performance prediction model based on a deep neural network, and input the communication performance change characteristic data of the communication device configured on the underwater robot into the communication device performance prediction model for training;

[0083] Through training, obtain the trained communication device performance prediction model, obtain the communication performance change characteristic data of the communication device within a preset time, input the communication performance change characteristic data of the communication device within a preset time into the communication performance change characteristic data of the communication device within a preset time for prediction, and obtain the communication performance data of each communication device configured on the underwater robot at the current timestamp;

[0084] Obtain the environmental characteristic data in the current water area, construct a retrieval label according to the environmental characteristic data in the current water area, and obtain the estimated communication performance characteristic data under the environmental characteristic data in the current water area through big data retrieval based on the retrieval label;

[0085] When the communication performance data of the communication devices configured on each underwater robot in the current timestamp is greater than the estimated communication performance characteristic data under the environmental characteristic data in the current water area, the corresponding underwater robot is used as the underwater robot recommended for work tasks.

[0086] It should be noted that after the communication device has been used for a certain number of years, there will be a certain performance decline. The communication performance change characteristic data includes communication delay, the amount of information transmitted during communication, information transmission rate, etc. Through this method, the work task allocation of the underwater robot can be made more reasonable.

[0087] In addition, the present invention may further include the following steps:

[0088] Obtain the estimated communication performance characteristic data under the environmental characteristic data in the current water area, set a communication performance characteristic threshold, and determine whether the estimated communication performance characteristic data under the environmental characteristic data in the current water area is greater than the communication performance characteristic threshold;

[0089] When the estimated communication performance characteristic data under the environmental characteristic data in the current water area is greater than the communication performance characteristic threshold, the corresponding area is used as the driving area that the underwater robot can track;

[0090] When the estimated communication performance characteristic data under the environmental characteristic data in the current water area is not greater than the communication performance characteristic threshold, the corresponding area is used as the driving area that the underwater robot cannot track;

[0091] Re-plan the path information of the underwater robot within a preset time according to the driving area that the underwater robot can track, and obtain the re-planned path information.

[0092] It should be noted that through this method, the path information of the underwater robot within a preset time can be re-planned according to the driving area that the underwater robot can track, so that when the remote control terminal controls the underwater robot, communication can be maintained, and the tracking of the underwater robot can be made more reasonable.

[0093] Such as Figure 4As shown in the figure, the second aspect of the present invention provides an underwater robot target tracking system 4 based on machine vision. The system 4 includes a memory 41 and a processor 42. The memory 41 includes a program for the underwater robot target tracking method based on machine vision. When the program for the underwater robot target tracking method based on machine vision is executed by the processor 42, the following steps are implemented:

[0094] Obtain the environmental feature data information in the target water area, set the camera parameters of the camera device according to the environmental feature data information in the target water area, and obtain the original image information collected by the underwater robot based on the camera parameters of the camera device;

[0095] Introduce the DCP algorithm and the genetic algorithm, perform feature processing on the original image information collected by the underwater robot according to the DCP algorithm and the genetic algorithm, and obtain the target tracking image information;

[0096] Build an object recognition model based on a deep neural network, and identify the target tracking image information through the object recognition model to obtain the action trajectory information of the object within a preset time;

[0097] Construct the path information of the underwater robot within a preset time according to the action trajectory information of the object within a preset time, control the underwater robot according to the path information of the underwater robot within a preset time, and monitor the real-time operation status of each underwater robot.

[0098] Further, in this system, obtaining the environmental feature data information in the target water area, setting the camera parameters of the camera device according to the environmental feature data information in the target water area, and obtaining the original image information collected by the underwater robot based on the camera parameters of the camera device specifically include:

[0099] Obtain the image clarity feature data information under each water environment feature and camera parameter through big data, introduce a graph neural network, and input the image clarity feature data information under each water environment feature and camera parameter into the graph neural network;

[0100] Take the water environment feature as the first graph node of the graph neural network, the camera parameter as the second graph node of the graph neural network, and the image clarity feature data information as the third graph node of the graph neural network, and construct a topological structure diagram based on the first graph node, the second graph node, and the third graph node;

[0101] Obtain the relevant adjacency matrix based on the topological structure diagram, construct a knowledge graph, input the relevant adjacency matrix into the knowledge graph for storage, obtain the environmental feature data information in the target water area, and import the environmental feature data information in the target water area into the knowledge graph for data matching;

[0102] Through data matching, obtain the image clarity feature data information corresponding to each camera parameter under the environmental feature data information in the current target water area, obtain the camera parameter corresponding to the maximum image clarity feature data information, and control the camera device to perform camera acquisition based on the image clarity feature data information to obtain the original image information collected by the underwater robot.

[0103] Furthermore, in this system, the DCP algorithm and the genetic algorithm are introduced. According to the DCP algorithm and the genetic algorithm, the original image information collected by the underwater robot is subjected to feature processing to obtain the target tracking image information, specifically including:

[0104] Introduce the DCP algorithm, divide the image into images of three RGB channels, calculate the mean value of each channel image, obtain the image with the maximum mean value and the image with the minimum mean value, and calculate the mean difference between the image with the maximum mean value and the image with the minimum mean value;

[0105] Based on the mean value output, obtain the depth of field image. When processing the first frame, count the pixels with a brightness of the top 0.1% on the depth of field image, and at the same time calculate the mean values of the three RGB channels at the positions of the same pixels in the original image, and use the mean values of the three RGB channels as the water body background color;

[0106] Based on the DCP algorithm, perform initialization processing on the depth of field image to obtain the transmission map, introduce the genetic algorithm, set the depth of field effect feature index of the transmission map, initialize the adjustment parameters of the transmission map, and adjust the transmission map based on the adjustment parameters of the transmission map to obtain the depth of field effect feature of the transmission map;

[0107] When the depth of field effect feature of the transmission map is not greater than the depth of field effect feature index of the transmission map, re-adjust the adjustment parameters of the transmission map until the depth of field effect feature of the transmission map is greater than the depth of field effect feature index of the transmission map, output the final transmission map, and combine the final transmission map, the original image, and the water body background color to splice and restore the image to obtain the target tracking image information.

[0108] Furthermore, in this system, a target object recognition model is constructed based on a deep neural network, specifically including:

[0109] Obtain the target object image data through big data, randomly select a preset number of target object image data, construct a training set according to the preset number of target object image data, obtain the type of the target object, and divide the training set based on the type of the target object to obtain the training subset of each target object type;

[0110] Calculate the Jaccard similarity coefficient between the training subsets of different target object types, calculate the Jaccard distance between the training subsets of different target object types based on the Jaccard similarity coefficient, and determine whether the Jaccard distance is greater than the preset Jaccard distance threshold;

[0111] When the Jaccard distance is greater than the preset Jaccard distance threshold, the training subset of the target object type is used as the final training subset of the target object type. When the Jaccard distance is not greater than the preset Jaccard distance threshold, the sample data of the target object type is reselected until the Jaccard distance is greater than the preset Jaccard distance threshold;

[0112] Based on the deep neural network, a target object recognition model is constructed. The final training subset of the target object type is sequentially input into the target object recognition model for training. When the convergence value of the loss function of the target object recognition model reaches the preset value, the target object recognition model is output.

[0113] Furthermore, in this system, the target object recognition model is used to recognize the target tracking image information to obtain the action trajectory information of the target object within the preset time, specifically including:

[0114] The underwater robot continuously collects the target tracking image information of several timestamps, and inputs the target tracking image information into the target object recognition model for recognition to determine whether there is a target object in the target tracking image information;

[0115] When there is a target object in the target tracking image information, the position information of the target object at each timestamp is recognized and obtained, and the action trajectory information of the target object within the preset time is constructed according to the position information of the target object at each timestamp.

[0116] Furthermore, in this system, the path information of the underwater robot within the preset time is constructed according to the action trajectory information of the target object within the preset time, and the underwater robot is controlled according to the path information of the underwater robot within the preset time, specifically including:

[0117] A preset tracking distance threshold is set, and the position information of the target object at the current timestamp is obtained according to the action trajectory information of the target object within the preset time, and the position information of the current underwater robot is obtained;

[0118] The Euclidean distance value between the position information of the target object at the current timestamp and the position information of the current underwater robot is calculated, and it is judged whether the Euclidean distance value between the position information of the target object at the current timestamp and the position information of the current underwater robot is greater than the tracking distance threshold;

[0119] When the Euclidean distance value between the position information of the target object at the current timestamp and the position information of the current underwater robot is greater than the tracking distance threshold, the position information of the current underwater robot is adjusted until the Euclidean distance value is not greater than the tracking distance threshold;

[0120] When the Euclidean distance value between the position information of the target at the current timestamp and the position information of the current underwater robot is not greater than the tracking distance threshold, combine the timestamps to construct the path information of the underwater robot within a preset time, and control the underwater robot according to the path information of the underwater robot within the preset time.

[0121] The third aspect of the present invention provides a computer-readable storage medium, which includes a program for the method of underwater robot target tracking based on machine vision. When the program for the method of underwater robot target tracking based on machine vision is executed by a processor, the steps of any of the methods of underwater robot target tracking based on machine vision are implemented.

[0122] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical, or other forms.

[0123] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units; they can be located in one place or distributed to multiple network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0124] In addition, in each embodiment of the present invention, the functional units can all be integrated in one processing unit, or each unit can be separately used as a unit, or two or more units can be integrated in one unit; the above-mentioned integrated units can be implemented in the form of hardware, or in the form of a combination of hardware and software functional units.

[0125] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above method embodiments; and the foregoing storage medium includes: mobile storage devices, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks, or optical disks, etc., which can store program codes.

[0126] Alternatively, if the above integrated units of the present invention are implemented in the form of software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present invention, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods of the various embodiments of the present invention. The foregoing storage medium includes: various media that can store program codes, such as removable storage devices, ROM, RAM, magnetic disks, or optical discs.

[0127] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A method for underwater robot target tracking based on machine vision, characterized in that: The following steps are involved: Acquire environmental characteristic data information in the target water area, set the camera parameters of the camera device according to the environmental characteristic data information in the target water area, and acquire the original image information collected by the underwater robot based on the camera parameters of the camera device; Introducing a DCP algorithm and a genetic algorithm, performing feature processing on the original image information collected by the underwater robot according to the DCP algorithm and the genetic algorithm, and obtaining target tracking image information; Building a target object recognition model based on a deep neural network, and identifying the target tracking image information through the target object recognition model to obtain the target object's movement trajectory information within a preset time; Constructing path information of the underwater robot within a preset time according to the movement trajectory information of the target object within a preset time, controlling the underwater robot according to the path information of the underwater robot within the preset time, and monitoring the real-time operation status of each underwater robot; Also includes: Acquire communication performance change characteristic data of the communication equipment configured on the underwater robot, and construct a communication equipment performance prediction model based on a deep neural network, and input the communication performance change characteristic data of the communication equipment configured on the underwater robot into the communication equipment performance prediction model for training; Through training, a communication device performance prediction model that has been trained is obtained, communication performance change characteristic data of the communication device within a preset time is obtained, the communication performance change characteristic data of the communication device within the preset time is input into the communication performance change characteristic data of the communication device within the preset time for prediction, and the communication performance data of the communication device configured on each underwater robot at the current timestamp is obtained; Acquire environmental characteristic data in the current water area, construct a search tag based on the environmental characteristic data in the current water area, and acquire estimated communication performance characteristic data under the environmental characteristic data in the current water area through big data retrieval based on the search tag; When the communication performance data of the communication device configured on each underwater robot at the current timestamp is greater than the estimated communication performance characteristic data under the environmental characteristic data in the current water area, the corresponding underwater robot is used as an underwater robot recommended to perform the work task; Also includes: Obtaining estimated communication performance characteristic data under the environmental characteristic data in the current water area, and setting a communication performance characteristic threshold, and determining whether the estimated communication performance characteristic data under the environmental characteristic data in the current water area is greater than the communication performance characteristic threshold; When the estimated communication performance characteristic data under the environmental characteristic data in the current water area is greater than the communication performance characteristic threshold, the corresponding area is used as a driving area that the underwater robot can track; When the estimated communication performance characteristic data under the environmental characteristic data in the current water area is not greater than the communication performance characteristic threshold, the corresponding area is regarded as a driving area that the underwater robot cannot track; The path information of the underwater robot within a preset time is replanned according to the driving area that the underwater robot can track, and the replanned path information is obtained.

2. The method for underwater robot target tracking based on machine vision according to claim 1, characterized in that: Acquiring environmental characteristic data information in the target water area, setting the camera parameters of the camera device according to the environmental characteristic data information in the target water area, and acquiring the original image information collected by the underwater robot based on the camera parameters of the camera device, specifically including: Obtaining image clarity feature data information of various water environment characteristics and camera parameters through big data, introducing a graph neural network, and inputting the image clarity feature data information of various water environment characteristics and camera parameters into the graph neural network; The water environment characteristics are used as the first graph node of the graph neural network, the camera parameters are used as the second graph node of the graph neural network, and the image clarity feature data information is used as the third graph node of the graph neural network. A topological structure graph is constructed based on the first graph node, the second graph node, and the third graph node; Based on the topological structure diagram, a relevant adjacency matrix is ​​obtained, and a knowledge graph is constructed, the relevant adjacency matrix is ​​input into the knowledge graph for storage, and environmental characteristic data information in the target water area is obtained, and the environmental characteristic data information in the target water area is imported into the knowledge graph for data matching; Through data matching, the image clarity feature data information corresponding to each camera parameter under the environmental feature data information in the current target water area is obtained, the camera parameters corresponding to the maximum image clarity feature data information are obtained, and the camera device is controlled to perform camera acquisition based on the image clarity feature data information to obtain the original image information collected by the underwater robot.

3. The method for underwater robot target tracking based on machine vision according to claim 1, characterized in that: The DCP algorithm and the genetic algorithm are introduced, and feature processing is performed on the original image information collected by the underwater robot according to the DCP algorithm and the genetic algorithm to obtain target tracking image information, specifically including: The DCP algorithm is introduced to divide the image into three channels of RGB. The mean of the images of each channel is calculated to obtain the image with the largest mean and the image with the smallest mean, and the mean difference between the image with the largest mean and the image with the smallest mean is calculated. A depth of field image is obtained based on the mean difference, and when processing the first frame, pixels whose brightness is in the top 0.1% are counted on the depth of field image, and the mean of the three RGB channels is calculated at the position of the same pixel in the original image, and the mean of the three RGB channels is used as the background color of the water body; Initializing the depth of field image based on the DCP algorithm to obtain a transmission map, introducing a genetic algorithm, setting a depth of field effect characteristic index of the transmission map, initializing a control parameter of the transmission map, adjusting the transmission map based on the control parameter of the transmission map, and obtaining a depth of field effect characteristic of the transmission map; When the depth of field effect feature of the transmission map is not greater than the depth of field effect feature index of the transmission map, the control parameters of the transmission map are readjusted until the depth of field effect feature of the transmission map is greater than the depth of field effect feature index of the transmission map, and the final transmission map is output. The image is spliced ​​and restored in combination with the final transmission map, the original image, and the background color of the water body to obtain target tracking image information.

4. The underwater robot target tracking method based on machine vision according to claim 1 is characterized in that: Build a target recognition model based on a deep neural network, including: Obtaining target object image data through big data, and randomly selecting a preset number of target object image data, constructing a training set according to the preset number of target object image data, obtaining the type of the target object, dividing the training set based on the type of the target object, and obtaining a training subset for each target object type; Calculating the Jaccard similarity coefficient between the training subsets of different target object types, calculating the Jaccard distance between the training subsets of different target object types based on the Jaccard similarity coefficient, and determining whether the Jaccard distance is greater than a preset Jaccard distance threshold; When the Jaccard distance is greater than a preset Jaccard distance threshold, the training subset of the target object type is used as the final training subset of the target object type; when the Jaccard distance is not greater than the preset Jaccard distance threshold, sample data in the target object type is reselected until the Jaccard distance is greater than the preset Jaccard distance threshold; An object recognition model is constructed based on a deep neural network, and the final training subset of the object type is sequentially input into the object recognition model for training. When the loss function of the object recognition model converges to a preset value, the object recognition model is output.

5. The underwater robot target tracking method based on machine vision according to claim 1, characterized in that: The target tracking image information is identified by the target object identification model to obtain the movement trajectory information of the target object within a preset time, specifically including: Continuously collecting target tracking image information of a plurality of time stamps by an underwater robot, and inputting the target tracking image information into the target object recognition model for recognition, and determining whether there is a target object in the target tracking image information; When there is a target object in the target tracking image information, the position information of the target object in each timestamp is identified and acquired, and the movement trajectory information of the target object within a preset time is constructed according to the position information of the target object in each timestamp.

6. The method for underwater robot target tracking based on machine vision according to claim 1, characterized in that: Constructing path information of the underwater robot within a preset time according to the movement trajectory information of the target object within a preset time, and controlling the underwater robot according to the path information of the underwater robot within the preset time, specifically includes: Preset a tracking distance threshold, and obtain the position information of the target object at the current timestamp according to the movement trajectory information of the target object within a preset time, and obtain the current position information of the underwater robot; Calculating the Euclidean distance between the position information of the target object at the current timestamp and the current position information of the underwater robot, and determining whether the Euclidean distance between the position information of the target object at the current timestamp and the current position information of the underwater robot is greater than the tracking distance threshold; When the Euclidean distance value between the position information of the target object at the current timestamp and the position information of the current underwater robot is greater than the tracking distance threshold, adjusting the position information of the current underwater robot until the Euclidean distance value is no greater than the tracking distance threshold; When the Euclidean distance value between the position information of the target object at the current timestamp and the current position information of the underwater robot is not greater than the tracking distance threshold, the path information of the underwater robot within the preset time is constructed in combination with the timestamp, and the underwater robot is controlled according to the path information of the underwater robot within the preset time.

7. An underwater robot target tracking system based on machine vision, characterized in that: The system includes a memory and a processor, wherein the memory includes a program of a method for underwater robot target tracking based on machine vision, and when the program of underwater robot target tracking based on machine vision is executed by the processor, the following steps are implemented: Acquire environmental characteristic data information in the target water area, set the camera parameters of the camera device according to the environmental characteristic data information in the target water area, and acquire the original image information collected by the underwater robot based on the camera parameters of the camera device; Introducing a DCP algorithm and a genetic algorithm, performing feature processing on the original image information collected by the underwater robot according to the DCP algorithm and the genetic algorithm, and obtaining target tracking image information; Building a target object recognition model based on a deep neural network, and identifying the target tracking image information through the target object recognition model to obtain the target object's movement trajectory information within a preset time; Constructing path information of the underwater robot within a preset time according to the movement trajectory information of the target object within a preset time, controlling the underwater robot according to the path information of the underwater robot within the preset time, and monitoring the real-time operation status of each underwater robot; Also includes: Acquire communication performance change characteristic data of the communication equipment configured on the underwater robot, and construct a communication equipment performance prediction model based on a deep neural network, and input the communication performance change characteristic data of the communication equipment configured on the underwater robot into the communication equipment performance prediction model for training; Through training, a communication device performance prediction model that has been trained is obtained, communication performance change characteristic data of the communication device within a preset time is obtained, the communication performance change characteristic data of the communication device within the preset time is input into the communication performance change characteristic data of the communication device within the preset time for prediction, and the communication performance data of the communication device configured on each underwater robot at the current timestamp is obtained; Acquire environmental characteristic data in the current water area, construct a search tag based on the environmental characteristic data in the current water area, and acquire estimated communication performance characteristic data under the environmental characteristic data in the current water area through big data retrieval based on the search tag; When the communication performance data of the communication device configured on each underwater robot at the current timestamp is greater than the estimated communication performance characteristic data under the environmental characteristic data in the current water area, the corresponding underwater robot is used as an underwater robot recommended to perform the work task; Also includes: Obtaining estimated communication performance characteristic data under the environmental characteristic data in the current water area, and setting a communication performance characteristic threshold, and determining whether the estimated communication performance characteristic data under the environmental characteristic data in the current water area is greater than the communication performance characteristic threshold; When the estimated communication performance characteristic data under the environmental characteristic data in the current water area is greater than the communication performance characteristic threshold, the corresponding area is used as a driving area that the underwater robot can track; When the estimated communication performance characteristic data under the environmental characteristic data in the current water area is not greater than the communication performance characteristic threshold, the corresponding area is regarded as a driving area that the underwater robot cannot track; The path information of the underwater robot within a preset time is replanned according to the driving area that the underwater robot can track, and the replanned path information is obtained.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a machine vision-based underwater robot target tracking method program. When the machine vision-based underwater robot target tracking method program is executed by a processor, the steps of the machine vision-based underwater robot target tracking method as described in any one of claims 1-6 are implemented.

Images