Target tracking methods, devices, electronic equipment and storage media for polar unmanned surface vessels

Abstract

The present invention provides a target tracking method, device, electronic device and storage medium for a polar unmanned boat, which relates to the field of target detection technology, including: obtaining distance information, unmanned boat driving information and image data to be processed; when the distance information is greater than a preset distance, obtaining the situation of the target to be detected; when the target to be detected is in a static situation, the first target feature to be detected is obtained by grayscale processing the image data to be processed and then extracting features; when the target to be detected is in a dynamic situation, the image data to be processed is input into a double-layer feature fusion model to obtain the second target feature to be detected; when the distance information is less than or equal to the preset distance, the image data to be processed is input into a YOLO convolutional network model to obtain the third target feature to be detected; and the position of the target to be detected is determined according to the first target feature to be detected, the second target feature to be detected or the third target feature to be detected. The present invention achieves improved target tracking accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a target tracking method, device, electronic equipment and storage medium for a polar unmanned boat. Background Art

[0002] Polar unmanned boats are unmanned vessels used to carry out exploration missions in polar regions. They can use a variety of different sensors and equipment to explore various natural environments in the polar regions, including glaciers, oceans, and mountains. Polar unmanned boat target tracking technology can help drivers detect and respond to potential safety issues in a timely manner and reduce the occurrence of accidents. Existing target tracking technology has a slow detection speed and cannot adapt to complex polar environments. It is easily affected by environmental interference, resulting in low tracking accuracy. Summary of the invention

[0003] The present invention aims to solve at least one of the above problems.

[0004] In order to solve the above problems, the present invention provides a target tracking method, device, electronic equipment and storage medium for a polar unmanned boat.

[0005] In a first aspect, the present invention provides a target tracking method for a polar unmanned boat, comprising:

[0006] Acquire distance information, unmanned boat driving information and image data to be processed, wherein the distance information is used to represent the distance between the unmanned boat and the target to be detected obtained by the sensor;

[0007] When the distance information is greater than a preset distance, the unmanned boat driving information and the image data to be processed are input into a pre-trained static and dynamic detection model to obtain a situation of the target to be detected, wherein the situation of the target to be detected includes a static situation of the target to be detected or a dynamic situation of the target to be detected;

[0008] When the state of the target to be detected is a static state of the target to be detected, a first feature of the target to be detected is obtained by performing feature extraction after grayscale processing on the image data to be processed;

[0009] When the state of the target to be detected is a dynamic state of the target to be detected, inputting the image data to be processed into a double-layer feature fusion model to obtain a second feature of the target to be detected;

[0010] When the distance information is less than or equal to the preset distance, inputting the image data to be processed into a YOLO convolutional network model to obtain a third target feature to be detected;

[0011] The position of the target to be detected is determined according to the first target feature to be detected, the second target feature to be detected, or the third target feature to be detected.

[0012] Optionally, the step of obtaining the first target feature to be detected by performing feature extraction after grayscale processing on the image data to be processed comprises:

[0013] Performing grayscale processing on the image data to be processed to obtain an initial grayscale image;

[0014] Obtaining grayscale image type information by matching the initial grayscale image with a preset grayscale image type database;

[0015] Performing grayscale enhancement on the initial grayscale image according to the grayscale image type information to obtain a target grayscale image;

[0016] The first target feature to be detected is obtained by performing feature extraction on the target grayscale image.

[0017] Optionally, the extracting features of the target grayscale image to obtain the first feature of the target to be detected includes:

[0018] Segmenting the target grayscale image to obtain a segmented grayscale image;

[0019] Performing feature extraction on the segmented grayscale image to obtain grayscale image features;

[0020] The first to-be-detected target feature is obtained by fusing the grayscale image features.

[0021] Optionally, the image data to be processed includes a plurality of sub-images taken continuously, and the step of inputting the image data to be processed into a double-layer feature fusion model to obtain the second target feature to be detected includes:

[0022] Obtaining corresponding fine-grained static features and fine-grained contour features according to each of the sub-images;

[0023] Obtaining the fine-grained fused feature by weighted averaging the fine-grained static feature and the fine-grained contour feature;

[0024] The second to-be-detected target feature is obtained by performing feature fusion according to the fine-grained fusion features corresponding to all the sub-images.

[0025] Optionally, the image data to be processed includes point cloud data, and the step of inputting the image data to be processed into a YOLO convolutional network model to obtain a third target feature to be detected includes:

[0026] Based on a clustering algorithm, a point cloud cluster is obtained according to the point cloud data;

[0027] Obtaining a point cloud feature matrix by performing feature extraction on the point cloud cluster;

[0028] Obtaining an RGB image by performing image processing on the multiple sub-images taken continuously;

[0029] The third target feature to be detected is obtained by performing multi-scale feature extraction and fusion on the point cloud feature matrix and the RGB image.

[0030] Optionally, the step of extracting and fusing the point cloud feature matrix and the RGB image to obtain the third target feature to be detected includes:

[0031] Performing multi-scale feature extraction on the point cloud feature matrix and the RGB image to obtain a point cloud feature map and a sub-image feature map;

[0032] The third target feature to be detected is obtained by cropping the point cloud feature map and the RGB image feature map and then performing matrix superposition and fusion.

[0033] Optionally, the determining the position of the target to be detected according to the first target feature to be detected or the second target feature to be detected or the third target feature to be detected further includes:

[0034] Obtaining target type information through the first target feature to be detected, the second target feature to be detected, or the third target feature to be detected;

[0035] Obtaining a target motion path and a target motion speed according to the image data to be processed and the target type information;

[0036] Based on a trajectory prediction algorithm, the target predicted position is obtained through the target motion path and the target motion speed.

[0037] In a second aspect, the present invention provides a target tracking device for a polar unmanned boat, comprising: a data acquisition module, used to acquire distance information, unmanned boat driving information and image data to be processed, wherein the distance information is used to represent the distance between the unmanned boat and the target to be detected obtained by a sensor;

[0038] A module for acquiring the situation of the target to be detected, for inputting the driving information of the unmanned boat and the image data to be processed into a pre-trained static and dynamic detection model to obtain the situation of the target to be detected when the distance information is greater than a preset distance, wherein the situation of the target to be detected includes a static situation of the target to be detected or a dynamic situation of the target to be detected;

[0039] A first target feature acquisition module for obtaining the first target feature to be detected, when the target is in a static state, by performing grayscale processing on the image data to be processed and then performing feature extraction to obtain the first target feature to be detected;

[0040] A second target feature acquisition module for obtaining a second target feature to be detected, for inputting the image data to be processed into a double-layer feature fusion model to obtain a second target feature to be detected when the target condition to be detected is a dynamic condition of the target to be detected;

[0041] A third target feature acquisition module to be detected, used for inputting the image data to be processed into a YOLO convolutional network model to obtain a third target feature to be detected when the distance information is less than or equal to the preset distance;

[0042] The module for acquiring the position of the target to be detected is used to determine the position of the target to be detected according to the first feature of the target to be detected, the second feature of the target to be detected, or the third feature of the target to be detected.

[0043] In a third aspect, the present invention provides an electronic device, including a memory and a processor;

[0044] The memory is used to store computer programs;

[0045] The processor is used to implement the target tracking method for the polar unmanned boat as described in the first aspect when executing the computer program.

[0046] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the target tracking method for a polar unmanned boat as described in the first aspect is implemented.

[0047] The target tracking method, device, electronic device and storage medium of the polar unmanned boat of the present invention have the beneficial effects of: using different target tracking methods by judging the distance between the unmanned boat and the target to be detected obtained by the sensor, when the distance information is greater than the preset distance, the target to be detected is far away, and further judging whether the target to be detected is static or dynamic through the pre-trained static and dynamic detection model, the static situation of the target to be detected or the dynamic situation of the target to be detected is obtained. When the situation of the target to be detected is the static situation of the target to be detected, the first feature of the target to be detected is obtained by grayscale processing of the image data to be processed and then extracting the feature. At this time, the target to be detected is a distant static object, which reduces unnecessary calculations and can improve the calculation rate. When the situation of the target to be detected is the dynamic situation of the target to be detected, the image data to be processed is input into the double-layer feature fusion model to obtain the second feature of the target to be detected. At this time, the target to be detected is a distant dynamic object. The double-layer feature fusion model can better capture the image details of the moving object, effectively improving the target tracking accuracy. When the distance information is less than or equal to the preset distance, the image data to be processed is input into the YOLO convolutional network model to obtain the third feature of the target to be detected. The position of the target to be detected is determined according to the first target feature to be detected, the second target feature to be detected, or the third target feature to be detected, and the image is processed differently for different situations, so that the target tracking method can better adapt to complex polar environment conditions, avoid environmental interference, and improve tracking accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic flow chart of a target tracking method for a polar unmanned boat according to an embodiment of the present invention;

[0049] Figure 2 This is a schematic structural diagram of a target tracking device for a polar unmanned boat according to an embodiment of the present invention;

[0050] Figure 3 The figure is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be interpreted as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not intended to limit the scope of protection of the present invention.

[0052] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0053] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to"; the term "based on" means "based at least in part on"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments"; the term "optionally" means "optional embodiments". The relevant definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc. mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0054] It should be noted that the modifications of "one" and "plurality" mentioned in the present invention are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0055] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only used for illustrative purposes, and are not used to limit the scope of these messages or information.

[0056] In view of the problems existing in the above-mentioned related technologies, this embodiment provides a target tracking method, device, electronic device and storage medium for a polar unmanned boat.

[0057] like Figure 1 As shown, an embodiment of the present invention provides a target tracking method for a polar unmanned boat, comprising:

[0058] Step 110, obtaining distance information, unmanned boat driving information and image data to be processed, wherein the distance information is used to represent the distance between the unmanned boat and the target to be detected obtained by the sensor.

[0059] Specifically, the distance information and the driving information of the unmanned boat are obtained by using sensors and other equipment. The sensors are arranged at the bow, stern, deck and both sides of the hull. The driving information of the unmanned boat includes the position of the unmanned boat, the direction of movement of the unmanned boat and the speed of the unmanned boat. The image data to be processed is obtained by using a laser radar and a camera installed at the bow of the unmanned boat. The image data to be processed is used to represent the image data of the target to be detected obtained by the equipment installed on the unmanned boat.

[0060] Step 121, when the distance information is greater than a preset distance, the unmanned boat driving information and the image data to be processed are input into a pre-trained static and dynamic detection model to obtain the situation of the target to be detected, wherein the situation of the target to be detected includes the static situation of the target to be detected or the dynamic situation of the target to be detected.

[0061] Specifically, historical unmanned boat driving information, historical image data to be processed, and historical target conditions to be detected are obtained; the historical unmanned boat driving information, the historical image data to be processed, and the historical target conditions to be detected are input into a neural network model for training until the model accuracy meets the preset model accuracy requirements, thereby obtaining the pre-trained motion and static detection model. The pre-trained motion and static detection model is used to represent a model for judging the motion and static state of a target to be detected through image data to be processed and unmanned boat driving information, wherein the neural network model may be a deep neural network model.

[0062] Step 1211, when the state of the target to be detected is a static state of the target to be detected, a first feature of the target to be detected is obtained by performing feature extraction after grayscale processing on the image data to be processed.

[0063] Specifically, grayscale processing is performed on the image data to be processed, and the color image data is converted into a grayscale image, and the grayscale image is used to represent an image containing only grayscale values. The image processing task can be simplified, and the brightness level of the pixel is represented by the grayscale value.

[0064] Step 1212, when the state of the target to be detected is a dynamic state of the target to be detected, inputting the image data to be processed into a double-layer feature fusion model to obtain a second feature of the target to be detected.

[0065] Specifically, the two-layer feature fusion model is a commonly used image and video feature extraction method that fuses features from multiple different modalities to extract richer feature representations. First, the feature data of the image data to be processed is extracted through a convolutional neural network, and then the extracted feature data is fused using the attention mechanism to obtain the second target feature to be detected.

[0066] Step 122: When the distance information is less than or equal to the preset distance, the image data to be processed is input into a YOLO convolutional network model to obtain a third target feature to be detected.

[0067] Specifically, the YOLO convolutional network model extracts feature data of the image data to be processed through convolution kernels of different sizes, then adds the output of each convolution kernel through the pooling layer and removes low-dimensional features to reduce the amount of calculation, and uses multiple fully connected layers of different sizes to connect the input and output.

[0068] Step 130: determining the position of the target to be detected according to the first target feature to be detected, the second target feature to be detected, or the third target feature to be detected.

[0069] Specifically, the target motion path and target motion speed are obtained through the acquired features, and then the trajectory prediction algorithm is used to predict the target position.

[0070] In this embodiment, different target tracking methods are adopted by judging the distance between the unmanned boat and the target to be detected obtained by the sensor. When the distance information is greater than the preset distance, the target to be detected is far away at this time. The pre-trained static and dynamic detection model is further used to judge whether the target to be detected is static or dynamic, and the static situation of the target to be detected or the dynamic situation of the target to be detected is obtained. When the situation of the target to be detected is the static situation of the target to be detected, the first target feature to be detected is obtained by grayscale processing of the image data to be processed and then extracting the features. At this time, the target to be detected is a distant static object, which reduces unnecessary calculations and can improve the calculation rate. When the situation of the target to be detected is the dynamic situation of the target to be detected, the image data to be processed is input into the double-layer feature fusion model to obtain the second target feature to be detected. At this time, the target to be detected is a distant dynamic object. The double-layer feature fusion model can better capture the image details of the moving object, effectively improving the target tracking accuracy. When the distance information is less than or equal to the preset distance, the image data to be processed is input into the YOLO convolutional network model to obtain the third target feature to be detected. The position of the target to be detected is determined according to the first target feature to be detected, the second target feature to be detected, or the third target feature to be detected, and the image is processed differently for different situations, so that the target tracking method can better adapt to complex polar environment conditions, avoid environmental interference, and improve tracking accuracy.

[0071] Optionally, the step of obtaining the first target feature to be detected by performing feature extraction after grayscale processing on the image data to be processed comprises:

[0072] Performing grayscale processing on the image data to be processed to obtain an initial grayscale image;

[0073] Obtaining grayscale image type information by matching the initial grayscale image with a preset grayscale image type database;

[0074] Performing grayscale enhancement on the initial grayscale image according to the grayscale image type information to obtain a target grayscale image;

[0075] The first target feature to be detected is obtained by performing feature extraction on the target grayscale image.

[0076] Optionally, the extracting features of the target grayscale image to obtain the first feature of the target to be detected includes:

[0077] Segmenting the target grayscale image to obtain a segmented grayscale image;

[0078] Performing feature extraction on the segmented grayscale image to obtain grayscale image features;

[0079] The first to-be-detected target feature is obtained by fusing the grayscale image features.

[0080] Specifically, the contour, texture and other information of the target to be detected can be obtained through the initial grayscale image, and the obtained relevant information is matched with the preset grayscale image type database to obtain the stored grayscale image type information, which includes animals, people, ships, ice floes, land, and buildings. Since the optimal grayscale processing control coefficients of different image types are different, the initial grayscale image is grayscale enhanced according to the grayscale image type information to obtain the target grayscale image. The target grayscale image is segmented to obtain a segmented grayscale image, and the segmented grayscale image is feature extracted to obtain grayscale image features, and the first target feature to be detected is obtained by fusing the grayscale image features.

[0081] In this optional embodiment, after grayscale processing of the image data to be processed, the grayscale image type is obtained by matching the preset grayscale image type database, and the grayscale of the image data to be processed is enhanced according to the grayscale image type, so as to facilitate subsequent image processing. Then, the processed image is segmented and features are extracted, and feature fusion is performed to obtain the first target feature to be detected, which not only improves the calculation efficiency, but also ensures the accuracy of target feature extraction.

[0082] Optionally, the image data to be processed includes a plurality of sub-images taken continuously, and the step of inputting the image data to be processed into a double-layer feature fusion model to obtain the second target feature to be detected includes:

[0083] Obtaining corresponding fine-grained static features and fine-grained contour features according to each of the sub-images;

[0084] Obtaining the fine-grained fused feature by weighted averaging the fine-grained static feature and the fine-grained contour feature;

[0085] The second to-be-detected target feature is obtained by performing feature fusion according to the fine-grained fusion features corresponding to all the sub-images.

[0086] Specifically, each sub-image is normalized and its background information is annotated, and fine-grained static features and fine-grained contour features are obtained through the processed sub-images. The fine-grained static features and fine-grained contour features obtained for each sub-image are fused to obtain fine-grained fused features, and then the second target feature to be detected is obtained through all the obtained fine-grained fused features.

[0087] In some more specific embodiments, the weight coefficients of weighted average fusion of the fine-grained static features and the fine-grained contour features are 0.6 and 0.4.

[0088] In this optional embodiment, by extracting fine-grained static features and fine-grained contour features of each sub-image, thereby extracting features of each sub-image, and then fusing the features of all the obtained sub-images, the target tracking accuracy can be effectively improved.

[0089] Optionally, the image data to be processed includes point cloud data, and the step of inputting the image data to be processed into a YOLO convolutional network model to obtain a third target feature to be detected includes:

[0090] Based on a clustering algorithm, a point cloud cluster is obtained according to the point cloud data;

[0091] Obtaining a point cloud feature matrix by performing feature extraction on the point cloud cluster;

[0092] Obtaining an RGB image by performing image processing on the multiple sub-images taken continuously;

[0093] The third target feature to be detected is obtained by performing multi-scale feature extraction and fusion on the point cloud feature matrix and the RGB image.

[0094] Optionally, the step of extracting and fusing the point cloud feature matrix and the RGB image to obtain the third target feature to be detected includes:

[0095] Performing multi-scale feature extraction on the point cloud feature matrix and the RGB image to obtain a point cloud feature map and a sub-image feature map;

[0096] The third target feature to be detected is obtained by cropping the point cloud feature map and the RGB image feature map and then performing matrix superposition and fusion.

[0097] Specifically, feature extraction is performed on the obtained point cloud cluster, and the obtained features are projected onto the RGB image in the form of a feature matrix, so as to obtain the effect of pixel-level fusion. At the same time, the intensity abnormal points are smoothed by Gaussian filtering, and then the intensity values ​​when there are no laser projection points on the target pixel points of the image are supplemented by expansion and corrosion operations. First, the formed point cloud feature matrix is ​​input into the multi-scale feature extraction network to perform feature extraction at different levels and scales. Subsequently, the extracted features are feature fused, and the fused large-scale, medium-scale, and small-scale features are detected using the YOLO detection end. Finally, the detection results are mapped to the point cloud feature matrix to obtain the corresponding point cloud cluster, thereby obtaining the third target feature to be detected, and the third target feature to be detected includes the final detection result including the real size of the target and the real position of the target in the body coordinate system.

[0098] In this optional embodiment, the advantages of RGB image data and three-dimensional point cloud data are utilized to construct a multi-source data fusion neural network for synchronous feature extraction, and the accuracy of the target recognition algorithm is improved by utilizing the stable depth features and intensity features of the three-dimensional point cloud in various harsh situations.

[0099] Optionally, the determining the position of the target to be detected according to the first target feature to be detected or the second target feature to be detected or the third target feature to be detected further includes:

[0100] Obtaining target type information through the first target feature to be detected, the second target feature to be detected, or the third target feature to be detected;

[0101] Obtaining a target motion path and a target motion speed according to the image data to be processed and the target type information;

[0102] Based on a trajectory prediction algorithm, the target predicted position is obtained through the target motion path and the target motion speed.

[0103] Specifically, the class of the target is determined according to the features of the target to be detected, the features are compared with the known target types, and the classification algorithm is used to determine the target category. The target motion path is obtained according to the image data to be processed and the target type information, and the target motion path is used to represent the position information sequence in the past period of time. The target motion speed is obtained by analyzing the position changes of the target at different time points through the target type information and the image data to be processed. Using the target motion path and the target motion speed, the algorithm will predict the future position of the target through a mathematical model, such as a linear prediction model, a Kalman filter, a particle filter, etc.

[0104] like Figure 2 As shown, an embodiment of the present invention provides a target tracking device for a polar unmanned boat, comprising:

[0105] The data acquisition module 10 is used to acquire distance information, unmanned boat driving information and image data to be processed, wherein the distance information is used to represent the distance between the unmanned boat and the target to be detected obtained by the sensor;

[0106] The module 20 for acquiring the situation of the target to be detected is used to input the driving information of the unmanned boat and the image data to be processed into a pre-trained static and dynamic detection model to obtain the situation of the target to be detected when the distance information is greater than a preset distance, wherein the situation of the target to be detected includes the static situation of the target to be detected or the dynamic situation of the target to be detected;

[0107] A first target feature acquisition module 30 is used to obtain the first target feature by performing feature extraction after grayscale processing on the image data to be processed when the target is in a static state;

[0108] A second target feature acquisition module 40 is used for inputting the image data to be processed into a double-layer feature fusion model to obtain a second target feature when the target is in a dynamic state.

[0109] A third target feature acquisition module 50 is used for inputting the image data to be processed into a YOLO convolutional network model to obtain a third target feature to be detected when the distance information is less than or equal to the preset distance;

[0110] The to-be-detected target position acquisition module 60 is used to determine the to-be-detected target position according to the first to-be-detected target feature, the second to-be-detected target feature, or the third to-be-detected target feature.

[0111] The target tracking device of the polar unmanned boat of this embodiment is used to implement the target tracking method of the polar unmanned boat as described above. Its advantages over the prior art are the same as the advantages of the target tracking method of the polar unmanned boat as described above over the prior art, and will not be repeated here.

[0112] Optionally, the first to-be-detected target feature acquisition module 30 is specifically used to: obtain an initial grayscale image after grayscale processing of the to-be-processed image data;

[0113] Obtaining grayscale image type information by matching the initial grayscale image with a preset grayscale image type database;

[0114] Performing grayscale enhancement on the initial grayscale image according to the grayscale image type information to obtain a target grayscale image;

[0115] The first target feature to be detected is obtained by performing feature extraction on the target grayscale image.

[0116] Optionally, the first to-be-detected target feature acquisition module 30 is specifically used to: segment the target grayscale image to obtain a segmented grayscale image;

[0117] Performing feature extraction on the segmented grayscale image to obtain grayscale image features;

[0118] The first to-be-detected target feature is obtained by fusing the grayscale image features.

[0119] Optionally, the second to-be-detected target feature acquisition module 40 is specifically used to: obtain corresponding fine-grained static features and fine-grained contour features according to each of the sub-images;

[0120] Obtaining the fine-grained fused feature by weighted averaging the fine-grained static feature and the fine-grained contour feature;

[0121] The second to-be-detected target feature is obtained by performing feature fusion according to the fine-grained fusion features corresponding to all the sub-images.

[0122] Optionally, the third to-be-detected target feature acquisition module 50 is specifically used to: obtain a point cloud cluster according to the point cloud data based on a clustering algorithm;

[0123] Obtaining a point cloud feature matrix by performing feature extraction on the point cloud cluster;

[0124] Obtaining an RGB image by performing image processing on the multiple sub-images taken continuously;

[0125] The third target feature to be detected is obtained by performing multi-scale feature extraction and fusion on the point cloud feature matrix and the RGB image.

[0126] Optionally, the third to-be-detected target feature acquisition module 50 is specifically used to: perform multi-scale feature extraction on the point cloud feature matrix and the RGB image to obtain a point cloud feature map and a sub-image feature map;

[0127] The third target feature to be detected is obtained by cropping the point cloud feature map and the RGB image feature map and then performing matrix superposition and fusion.

[0128] Optionally, the to-be-detected target position acquisition module 60 is specifically used to: obtain target type information through the first to-be-detected target feature or the second to-be-detected target feature or the third to-be-detected target feature;

[0129] Obtaining a target motion path and a target motion speed according to the image data to be processed and the target type information;

[0130] Based on a trajectory prediction algorithm, the target predicted position is obtained through the target motion path and the target motion speed.

[0131] like Figure 3 As shown, an electronic device 300 provided by an embodiment of the present invention includes a memory 310 and a processor 320; the memory 310 is used to store a computer program; the processor 320 is used to implement the target tracking method of the polar unmanned boat as described above when executing the computer program.

[0132] In other words, an electronic device 300 includes a memory 310 and a processor 320 coupled to the memory 310; the memory 310 is configured to store a computer program; and the processor 320 is configured to perform the following operations when executing the computer program:

[0133] Acquire distance information, unmanned boat driving information and image data to be processed, wherein the distance information is used to represent the distance between the unmanned boat and the target to be detected obtained by the sensor;

[0134] When the distance information is greater than a preset distance, the unmanned boat driving information and the image data to be processed are input into a pre-trained static and dynamic detection model to obtain a situation of the target to be detected, wherein the situation of the target to be detected includes a static situation of the target to be detected or a dynamic situation of the target to be detected;

[0135] When the state of the target to be detected is a static state of the target to be detected, a first feature of the target to be detected is obtained by performing feature extraction after grayscale processing on the image data to be processed;

[0136] When the state of the target to be detected is a dynamic state of the target to be detected, inputting the image data to be processed into a double-layer feature fusion model to obtain a second feature of the target to be detected;

[0137] When the distance information is less than or equal to the preset distance, inputting the image data to be processed into a YOLO convolutional network model to obtain a third target feature to be detected;

[0138] The position of the target to be detected is determined according to the first target feature to be detected, the second target feature to be detected, or the third target feature to be detected.

[0139] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the target tracking method for the polar unmanned boat as described above is implemented.

[0140] In other words, a non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor performs the following operations:

[0141] Acquire distance information, unmanned boat driving information and image data to be processed, wherein the distance information is used to represent the distance between the unmanned boat and the target to be detected obtained by the sensor;

[0142] When the distance information is greater than a preset distance, the unmanned boat driving information and the image data to be processed are input into a pre-trained static and dynamic detection model to obtain a situation of the target to be detected, wherein the situation of the target to be detected includes a static situation of the target to be detected or a dynamic situation of the target to be detected;

[0143] When the state of the target to be detected is a static state of the target to be detected, a first feature of the target to be detected is obtained by performing feature extraction after grayscale processing on the image data to be processed;

[0144] When the state of the target to be detected is a dynamic state of the target to be detected, inputting the image data to be processed into a double-layer feature fusion model to obtain a second feature of the target to be detected;

[0145] When the distance information is less than or equal to the preset distance, inputting the image data to be processed into a YOLO convolutional network model to obtain a third target feature to be detected;

[0146] The position of the target to be detected is determined according to the first target feature to be detected, the second target feature to be detected, or the third target feature to be detected.

[0147] An electronic device 300 that can be used as a server or client of the present invention will now be described, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device 300 is intended to represent various forms of digital electronic computer equipment, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 300 can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the present invention described and / or required herein.

[0148] The electronic device 300 includes a computing unit, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) or a computer program loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for the operation of the device can also be stored. The computing unit, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0149] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc. In the present application, the unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment of the present invention. In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0150] Although the present invention is disclosed as above, the protection scope of the present invention is not limited thereto. Those skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will fall within the protection scope of the present invention.

Claims

1. A target tracking method for a polar unmanned boat, characterized in that: include: Acquire distance information, unmanned boat driving information and image data to be processed, wherein the distance information is used to represent the distance between the unmanned boat and the target to be detected obtained by the sensor; When the distance information is greater than a preset distance, the unmanned boat driving information and the image data to be processed are input into a pre-trained static and dynamic detection model to obtain a situation of the target to be detected, wherein the situation of the target to be detected includes a static situation of the target to be detected or a dynamic situation of the target to be detected; When the state of the target to be detected is a static state of the target to be detected, a first feature of the target to be detected is obtained by performing feature extraction after grayscale processing on the image data to be processed; When the target to be detected is in a dynamic state, the image data to be processed is input into a double-layer feature fusion model to obtain a second target feature to be detected, and the image data to be processed includes a plurality of sub-images taken continuously, including: Obtaining corresponding fine-grained static features and fine-grained contour features according to each of the sub-images; Obtaining a fine-grained fusion feature by weighted averaging the fine-grained static feature and the fine-grained contour feature; Performing feature fusion according to the fine-grained fusion features corresponding to all the sub-images to obtain the second to-be-detected target feature; When the distance information is less than or equal to the preset distance, inputting the image data to be processed into a YOLO convolutional network model to obtain a third target feature to be detected; The position of the target to be detected is determined according to the first target feature to be detected, the second target feature to be detected, or the third target feature to be detected.

2. The target tracking method of the polar unmanned boat according to claim 1, characterized in that: The step of obtaining the first target feature to be detected by performing feature extraction after grayscale processing on the image data to be processed comprises: Performing grayscale processing on the image data to be processed to obtain an initial grayscale image; Obtaining grayscale image type information by matching the initial grayscale image with a preset grayscale image type database; Performing grayscale enhancement on the initial grayscale image according to the grayscale image type information to obtain a target grayscale image; The first target feature to be detected is obtained by performing feature extraction on the target grayscale image.

3. The target tracking method of the polar unmanned boat according to claim 2, characterized in that: The step of extracting features from the target grayscale image to obtain the first feature of the target to be detected includes: Segmenting the target grayscale image to obtain a segmented grayscale image; Performing feature extraction on the segmented grayscale image to obtain grayscale image features; The first to-be-detected target feature is obtained by fusing the grayscale image features.

4. The target tracking method of the polar unmanned boat according to claim 1, characterized in that: The image data to be processed includes point cloud data, and the step of inputting the image data to be processed into a YOLO convolutional network model to obtain a third target feature to be detected includes: Based on a clustering algorithm, a point cloud cluster is obtained according to the point cloud data; Obtaining a point cloud feature matrix by performing feature extraction on the point cloud cluster; Obtaining an RGB image by performing image processing on the multiple sub-images taken continuously; The third target feature to be detected is obtained by performing multi-scale feature extraction and fusion on the point cloud feature matrix and the RGB image.

5. The target tracking method of the polar unmanned boat according to claim 4, characterized in that: The step of performing multi-scale feature extraction and fusion on the point cloud feature matrix and the RGB image to obtain the third target feature to be detected includes: Performing multi-scale feature extraction on the point cloud feature matrix and the RGB image to obtain a point cloud feature map and a sub-image feature map respectively; The third target feature to be detected is obtained by cropping the point cloud feature map and the RGB image and then performing matrix superposition and fusion.

6. The target tracking method of the polar unmanned boat according to claim 1, characterized in that: The determining the position of the target to be detected according to the first target feature to be detected, the second target feature to be detected, or the third target feature to be detected further includes: Obtaining target type information through the first target feature to be detected, the second target feature to be detected, or the third target feature to be detected; Obtaining a target motion path and a target motion speed according to the image data to be processed and the target type information; Based on a trajectory prediction algorithm, the target predicted position is obtained through the target motion path and the target motion speed.

7. A target tracking device for a polar unmanned boat, characterized in that: include: A data acquisition module, used to acquire distance information, unmanned boat driving information and image data to be processed, wherein the distance information is used to represent the distance between the unmanned boat and the target to be detected obtained by the sensor; A module for acquiring the situation of the target to be detected, for inputting the driving information of the unmanned boat and the image data to be processed into a pre-trained static and dynamic detection model to obtain the situation of the target to be detected when the distance information is greater than a preset distance, wherein the situation of the target to be detected includes a static situation of the target to be detected or a dynamic situation of the target to be detected; A first target feature acquisition module for obtaining the first target feature to be detected, when the target is in a static state, by performing grayscale processing on the image data to be processed and then performing feature extraction to obtain the first target feature to be detected; The second target feature acquisition module is used for inputting the image data to be processed into a double-layer feature fusion model to obtain the second target feature to be detected when the target situation to be detected is a dynamic situation of the target to be detected, wherein the image data to be processed includes a plurality of sub-images taken continuously, including: Obtaining corresponding fine-grained static features and fine-grained contour features according to each of the sub-images; Obtaining a fine-grained fusion feature by weighted averaging the fine-grained static feature and the fine-grained contour feature; Performing feature fusion according to the fine-grained fusion features corresponding to all the sub-images to obtain the second to-be-detected target feature; A third target feature acquisition module to be detected, used for inputting the image data to be processed into a YOLO convolutional network model to obtain a third target feature to be detected when the distance information is less than or equal to the preset distance; The module for acquiring the position of the target to be detected is used to determine the position of the target to be detected according to the first feature of the target to be detected, the second feature of the target to be detected, or the third feature of the target to be detected.

8. An electronic device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to implement the target tracking method for a polar unmanned boat as described in any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the target tracking method for the polar unmanned boat as claimed in any one of claims 1 to 6 is implemented.

Images