A target recognition method suitable for polar unmanned boats

Abstract

The present invention provides a target recognition method suitable for polar unmanned boats, which relates to the technical field of target recognition. The method comprises acquiring image data and environmental data of a target area of ​​a polar unmanned boat, wherein the environmental data comprises meteorological data and obstacle data; extracting objective indicators of the image data, and determining recognition influencing parameters according to the meteorological data, obstacle data and objective indicators; segmenting the image data according to the recognition influencing parameters, generating at least one defective image and at least one normal image; determining the defect type of each defective image according to the recognition influencing parameters, wherein the defect type comprises an occlusion defect and a clarity defect; generating image data to be recognized according to an occlusion compensation image and / or a clarity compensation image, and all normal images, and generating a target recognition result according to the image data to be recognized by adopting a target recognition technology. The method of the present invention can improve the accuracy of target recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of target recognition, and in particular to a target recognition method suitable for a polar unmanned boat. Background Art

[0002] Polar unmanned boats refer to unmanned boats that perform scientific research, resource exploration, environmental monitoring, search and rescue and other tasks in polar environments. When they navigate autonomously in polar environments, they need to identify targets in the surrounding environment to ensure safe navigation and complete designated tasks.

[0003] In the existing technology, unmanned boats generally use image recognition algorithms to identify targets in the surrounding environment. However, due to the complex polar environment, there are harsh environments such as floating ice obstructions, overexposure, fog, etc., which have a great impact on image data, resulting in low accuracy of image recognition algorithms in identifying targets in the surrounding environment. Summary of the invention

[0004] The problem solved by the present invention is how to improve the accuracy of target recognition.

[0005] To solve the above problems, in a first aspect, the present invention provides a target recognition method applicable to a polar unmanned boat, comprising:

[0006] Acquire image data and environmental data of a polar unmanned boat target area, wherein the environmental data includes meteorological data and obstacle data;

[0007] extracting objective indicators of the image data, and determining identification influencing parameters according to the meteorological data, the obstacle data and the objective indicators;

[0008] Segmenting the image data according to the identification influencing parameter to generate at least one defect image and at least one normal image;

[0009] Determining the defect type of each of the defect images according to the identification influencing parameters, wherein the defect type includes an occlusion defect and a clarity defect;

[0010] For the occlusion defect, an image positioning technology is used to determine the spatial positioning of the occlusion defect, an image acquisition point is determined according to the spatial positioning and the obstacle data, and an occlusion compensation image is obtained according to the image acquisition point;

[0011] For the clarity defect, multi-source information data corresponding to the clarity defect space is acquired, and the multi-modal fusion technology is used to integrate the multi-source information data into a clarity compensation image, wherein the clarity defect space is a space in the target area corresponding to the clarity defect;

[0012] Image data to be identified are generated based on the occlusion compensation image and / or the clarity compensation image corresponding to each of the defective images, as well as all the normal images. Target recognition results are generated based on the image data to be identified using target recognition technology.

[0013] Optionally, determining the identification influencing parameter according to the meteorological data, the obstacle data and the objective indicator includes:

[0014] Segmenting the image data to obtain at least one sub-image data, wherein the size of each sub-image data is a first preset size;

[0015] Determining a first recognition influencing parameter of each of the sub-image data according to the objective indicator and a preset objective threshold;

[0016] According to the meteorological data and the obstacle data, an influence formula is used to determine a second identification influence parameter, wherein the influence formula includes:

[0017] ;

[0018] in, I for the second identification influencing parameter, , , , and are normalized coefficients, , , and are constants, V is the visibility data of the meteorological data, L is the light intensity data of the meteorological data, is the obstacle data, is the wind speed data of the meteorological data, The wind direction data of the meteorological data;

[0019] The first recognition influencing parameter and the second recognition influencing parameter are integrated to determine the recognition influencing parameter of each sub-image data.

[0020] Optionally, segmenting the image data according to the identification influencing parameter to generate at least one defect image and at least one normal image includes:

[0021] According to the identification influencing parameter and the corresponding sub-image data, extract the sub-image data whose identification influencing parameter is greater than or equal to a preset influencing threshold value to generate an influencing image; extract the sub-image data whose identification influencing parameter is less than the preset influencing threshold value to generate a non-influencing image;

[0022] Connecting adjacent influencing images to generate at least one total defect image, and segmenting the total defect image to generate at least one defect image, wherein the size of each defect image is a second preset size;

[0023] Adjacent unaffected images are connected to generate at least one normal image.

[0024] Optionally, the integrating the first recognition influencing parameter and the second recognition influencing parameter to determine the recognition influencing parameter of each sub-image data includes:

[0025] The identification influence parameter is generated by using a weighted formula according to the first identification influence parameter and the second identification influence parameter, wherein the weighted formula includes:

[0026] E = a + b*I ;

[0027] Wherein, E is the identification influencing parameter, a is the first weight, b is the second weight, H is the first identification influencing parameter, and I is the second identification influencing parameter.

[0028] Optionally, determining the defect type of each defect image according to the identification influencing parameter includes:

[0029] Performing grayscale processing on the defect image to obtain a defect grayscale image;

[0030] According to the defect grayscale image, a clarity score is obtained by using a clarity evaluation formula, wherein the clarity evaluation formula includes:

[0031] ;

[0032] in, is the clarity score of the i-th defect image, M is the total number of pixels of the i-th defect image, is the pixel point in the i-th defect image The grayscale value of the defect grayscale image, is the pixel point in the i-th defect image The grayscale value of the defect grayscale image after Gaussian processing;

[0033] The edge detection technology is used to extract edge points in the defect image, and an occlusion score is obtained according to the defect grayscale image, the edge points and an occlusion evaluation formula. The occlusion evaluation formula includes:

[0034] ;

[0035] in, is the occlusion score of the i-th defect image, To adjust the parameters, is the edge point in the i-th defect image The grayscale value of the defect grayscale image, is the edge point The average gray value of the neighboring area;

[0036] The defect type is determined according to the clarity score and the occlusion score.

[0037] Optionally, determining the defect type according to the clarity score and the occlusion score includes:

[0038] When the clarity score is greater than a preset multiple of the occlusion score, determining that the defect type is the clarity defect;

[0039] When the clarity score is less than the preset multiple of the occlusion score, the defect type is determined to be the occlusion defect.

[0040] Optionally, determining the image acquisition point according to the spatial positioning and the obstacle data includes:

[0041] Extracting three-dimensional coordinates of obstacles according to the spatial positioning and the obstacle data;

[0042] According to the coordinate range of the target area and the three-dimensional coordinates of the obstacle, a particle swarm optimization algorithm is used to obtain the image acquisition points.

[0043] Optionally, acquiring the occlusion compensation image according to the image acquisition points includes:

[0044] Acquire a compensation image according to the image acquisition points;

[0045] Respectively extracting first feature points of the compensation image and second feature points of the defect image corresponding to the compensation image;

[0046] Determine a compensation matching point pair using a feature matching algorithm according to the first feature point and the second feature point, wherein the compensation matching point pair includes the first feature point and the second feature point corresponding to each other;

[0047] Cropping the compensated image according to the first feature point in the compensated matching point pair to generate a matching image, and estimating a similarity transformation matrix using the compensated matching point pair;

[0048] According to the similarity transformation matrix, a similarity transformation is performed on the matching image to generate the occlusion compensation image.

[0049] Optionally, the multi-source information data includes RGB images and point cloud data, and the multi-modal fusion technology is used to integrate the multi-source information data into a clarity compensated image, including:

[0050] Based on the Point-Fusion network, image features of the RGB image and point cloud features of the point cloud data are respectively extracted;

[0051] The image features and the point cloud features are cropped, downsampled and superimposed in sequence to generate the clarity compensated image.

[0052] Optionally, the generating a target recognition result by using a target recognition technology according to the image data to be recognized includes:

[0053] Inputting the image data to be identified into a multi-scale feature extraction network to generate a first scale feature, a second scale feature, and a third scale feature, wherein the scales of the first scale feature, the second scale feature, and the third scale feature decrease in sequence;

[0054] Through the Darknet53 network, extracting first-scale depth features, second-scale depth features, and third-scale depth features of the first-scale features, the second-scale features, and the third-scale features respectively;

[0055] The first-scale depth feature, the second-scale depth feature, and the third-scale depth feature are input into a YOLO target detection network to generate the target recognition result.

[0056] In a second aspect, the present invention provides a target recognition device suitable for a polar unmanned boat, comprising:

[0057] An acquisition module, used to acquire image data and environmental data of a polar unmanned boat target area, wherein the environmental data includes meteorological data and obstacle data;

[0058] An influence module, used for extracting objective indicators of the image data, and determining identification influence parameters according to the meteorological data, the obstacle data and the objective indicators;

[0059] A segmentation module, used for segmenting the image data according to the identification influencing parameter to generate at least one defect image and at least one normal image;

[0060] A defect module, used to determine the defect type of each defect image according to the identification influencing parameter, wherein the defect type includes an occlusion defect and a clarity defect;

[0061] An occlusion module is used to determine the spatial location of the occlusion defect using image positioning technology, determine image acquisition points according to the spatial positioning and the obstacle data, and obtain an occlusion compensation image according to the image acquisition points;

[0062] A clarity module, for obtaining multi-source information data corresponding to a clarity defect space for the clarity defect, and integrating the multi-source information data into a clarity compensation image by using a multimodal fusion technology, wherein the clarity defect space is a space in the target area corresponding to the clarity defect;

[0063] The recognition module is used to generate image data to be recognized based on the occlusion compensation image and / or the clarity compensation image corresponding to each of the defective images, and all the normal images, and generate a target recognition result based on the image data to be recognized using target recognition technology.

[0064] In a third aspect, the present invention provides an electronic device, including a memory and a processor;

[0065] The memory is used to store computer programs;

[0066] The processor is used to implement the target recognition method applicable to polar unmanned boats as described in the first aspect when executing the computer program.

[0067] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the target recognition method applicable to a polar unmanned boat as described in the first aspect is implemented.

[0068] The beneficial effects of the target recognition method applicable to polar unmanned boats of the present invention are:

[0069] By acquiring the image data and environmental data of the target area of ​​the polar unmanned boat, it is convenient to determine the identification influence parameters of the environmental data on the image data for the target recognition task according to the objective indicators of the image data, as well as the meteorological data and obstacle data. Then, by segmenting the image, the problematic area and the non-problematic area, that is, the defect image and the normal image, are distinguished, which is helpful for the subsequent centralized processing of the problematic part. Then, according to the identification influence parameters, it is determined whether the defect type of each defect image is an occlusion defect or a clarity defect, so as to adopt targeted strategies to improve the defect part and eliminate the influence of the polar environment on the image quality, thereby improving the accuracy of target recognition. When the defect type is an occlusion defect, the spatial positioning obtained by the image positioning technology is combined with the obstacle data, so that the polar unmanned boat can re-plan the image acquisition position, thereby avoiding the obstruction and collecting the occlusion compensation image at the new image acquisition point. When the defect type is a clarity defect, the data of different information sources corresponding to the spatial position of the defect image, that is, multi-source information data, are collected, and then the data of different information sources are fused into a clarity compensation image through multimodal fusion technology. By making targeted compensation for the impact of different environmental factors on image data, not only can the image quality be improved, but also the polar unmanned boat can cope with various complex polar environments, improving adaptability and robustness. Then, the defective parts are compensated and repaired by occlusion compensation images and / or clarity compensation images to eliminate the impact of the polar environment on the image, and combined with all normal images, the image data to be identified that can feedback the most realistic target area can be obtained. Finally, the target recognition technology is used to perform target recognition and obtain high-precision target recognition results. In addition, since the present invention only compensates for defective images, it is not necessary to compensate the entire image data, which reduces unnecessary computing resource consumption and improves the overall efficiency of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 A schematic flow chart of a target recognition method applicable to a polar unmanned boat provided by an embodiment of the present invention;

[0071] Figure 2 A schematic diagram of the structure of a target recognition device applicable to a polar unmanned boat provided by an embodiment of the present invention;

[0072] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0073] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be interpreted as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not intended to limit the scope of protection of the present invention.

[0074] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0075] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to"; the term "based on" means "based at least in part on"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments"; the term "optionally" means "optional embodiments". The relevant definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc. mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0076] It should be noted that the modifications of "one" and "plurality" mentioned in the present invention are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0077] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only used for illustrative purposes, and are not used to limit the scope of these messages or information.

[0078] In view of the problems existing in the above-mentioned related technologies, this embodiment provides a target recognition method suitable for polar unmanned boats.

[0079] like Figure 1 As shown, an embodiment of the present invention provides a target recognition method applicable to a polar unmanned boat, comprising:

[0080] S1, acquiring image data and environmental data of a polar unmanned boat target area, wherein the environmental data includes meteorological data and obstacle data.

[0081] Specifically, firstly, image data and environmental data of the target area of ​​the polar unmanned boat are obtained to facilitate subsequent analysis of the image data and target recognition. The image data can be obtained by using a shooting device, and the environmental data can be obtained by using a sensor.

[0082] S2, extracting objective indicators of the image data, and determining identification influencing parameters according to the meteorological data, the obstacle data and the objective indicators.

[0083] Specifically, objective indicators include contrast, brightness, texture, color saturation, and noise level. The objective indicators of image data can reflect the quality of image data to a certain extent. Combined with the corresponding meteorological data and obstacle data, the adverse effects of environmental data on target recognition based on image data can be determined, that is, influencing parameters can be identified. By analyzing and identifying influencing parameters, it is convenient to compensate for the image data in the future to obtain image data that best reflects the real target area, thereby improving the accuracy of target recognition.

[0084] S3, segmenting the image data according to the identification influencing parameters to generate at least one defect image and at least one normal image.

[0085] Specifically, by identifying influencing parameters, the image data is divided into two types: defective images and normal images, so that only the defective parts, i.e., the defective images, can be compensated and repaired later, thereby reducing the amount of calculation and the recognition cost.

[0086] S4, determining the defect type of each of the defective images according to the identification influencing parameters, wherein the defect type includes an occlusion defect and a clarity defect.

[0087] Specifically, the impact of environmental factors on image recognition is mainly divided into occlusion impact and clarity impact. Occlusion impact refers to obstacle data in environmental data, such as floating ice obstacles, which cause occlusion to the target area, resulting in the inability to identify the entire target area during target recognition, making the image data have occlusion defects; clarity impact refers to the impact of meteorological data in environmental data on the recognition of image data. For example, high light intensity is likely to cause overexposure, low light intensity is likely to cause low image quality, high wind speed is likely to cause shaking of the shooting equipment, blurred shooting effects, and low visibility are likely to cause low image quality. Therefore, the impact of meteorological data on the recognition of image data is mainly in clarity and cannot reflect the real target area. Therefore, by determining the defect type of the defective image, it is convenient to take targeted measures to compensate for the image data in the future, which can improve the image quality and thus improve the accuracy of target recognition.

[0088] S5, when the defect type is the occlusion defect, the image positioning technology is used to determine the spatial positioning of the occlusion defect, the image acquisition points are determined according to the spatial positioning and the obstacle data, and the occlusion compensation image is obtained according to the image acquisition points.

[0089] Specifically, when the defect type is an occlusion defect, the image positioning technology is first used, for example, etc., to determine the spatial positioning of the obstacle that occludes the target area in the image data, that is, the spatial coordinates, and then based on the spatial positioning of the obstacle and the obstacle data outside the image data, that is, the spatial coordinates of the remaining obstacles, an optimization algorithm is used to determine a new image acquisition point, and at this image acquisition point, only the image of the target area occluded by the obstacle is collected, and it is not necessary to collect images of all the target areas, so as to reduce costs while obtaining occlusion compensation images.

[0090] S6. When the defect type is the clarity defect, multi-source information data corresponding to the clarity defect space is obtained, and multi-modal fusion technology is used to integrate the multi-source information data into a clarity compensation image, wherein the clarity defect space is the space in the target area corresponding to the clarity defect.

[0091] Specifically, multi-source information data refers to data from different information sources, such as infrared thermal imaging data, multi-spectral data, laser radar data, sonar data, and point cloud data, etc. When the defect type is a clarity defect, obtain the multi-source information data corresponding to the clarity defect space, that is, the spatial position of the clarity defect in the target area. For example, if the upper left corner of the image data is a defect image with a clarity defect, only the multi-source information data corresponding to the spatial position of the upper left corner is obtained. Then, the multi-modal fusion technology is used to fuse the data from different information sources into a cleanliness compensation image, which reduces costs and facilitates the subsequent compensation of image data.

[0092] S7, generating image data to be identified based on the occlusion compensation image and / or the clarity compensation image corresponding to each of the defective images, and all the normal images, and generating a target recognition result based on the image data to be identified by using target recognition technology.

[0093] Specifically, according to the defect situation of the image data, that is, only occlusion defect, only clarity defect, or both occlusion defect and clarity defect exist, the defective part is compensated by using occlusion compensation image and / or clarity compensation image, that is, only occlusion compensation image is used to compensate the defective part, only clarity compensation image is used to compensate the defective part, and both occlusion compensation image and clarity compensation image are used to compensate the defective part, and then spliced ​​with other normal images to obtain compensated image data to be identified, and finally, target recognition is performed on the image data to be identified through target recognition technology to obtain the final target recognition result.

[0094] In this embodiment, by acquiring the image data and environmental data of the target area of ​​the polar unmanned boat, it is convenient to determine the identification influence parameters of the environmental data on the image data for the target recognition task according to the objective indicators of the image data, as well as the meteorological data and the obstacle data, and then by segmenting the image, distinguishing the problematic area from the non-problematic area, that is, the defective image and the normal image, which is helpful for the subsequent centralized processing of the problematic part, and then determining whether the defect type of each defective image is an occlusion defect or a clarity defect according to the identification influence parameter, so as to adopt a targeted strategy to improve the defective part and eliminate the influence of the polar environment on the image, thereby improving the accuracy of target recognition. When the defect type is an occlusion defect, the spatial positioning obtained by the image positioning technology is combined with the obstacle data, so that the polar unmanned boat can re-plan the image acquisition position, thereby avoiding the obstruction and collecting the occlusion compensation image at the new image acquisition point. When the defect type is a clarity defect, the data of different information sources corresponding to the spatial position of the defective image, that is, multi-source information data, are collected, and then the data of different information sources are fused into a clarity compensation image through multimodal fusion technology. By making targeted compensation for the impact of different environmental factors on image data, not only can the image quality be improved, but also the polar unmanned boat can cope with various complex polar environments, improving adaptability and robustness. Then, the defective parts are compensated and repaired by occlusion compensation images and / or clarity compensation images to eliminate the impact of the polar environment on the image, and combined with all normal images, the image data to be identified that can feedback the most realistic target area can be obtained. Finally, the target recognition technology is used to perform target recognition and obtain high-precision target recognition results. In addition, since the present invention only compensates for defective images, it is not necessary to compensate the entire image data, which reduces unnecessary computing resource consumption, reduces costs, and improves the overall efficiency of the system.

[0095] Optionally, determining the identification influencing parameter according to the meteorological data, the obstacle data and the objective indicator includes:

[0096] Segmenting the image data to obtain at least one sub-image data, wherein the size of each sub-image data is a first preset size;

[0097] Determining a first recognition influencing parameter of each of the sub-image data according to the objective indicator and a preset objective threshold;

[0098] According to the meteorological data and the obstacle data, an influence formula is used to determine a second identification influence parameter, wherein the influence formula includes:

[0099] ;

[0100] in, I for the second identification influencing parameter, , , , and are normalized coefficients, , , and are constants, V is the visibility data of the meteorological data, L is the light intensity data of the meteorological data, is the obstacle data, is the wind speed data of the meteorological data, The wind direction data of the meteorological data;

[0101] The first recognition influencing parameter and the second recognition influencing parameter are integrated to determine the recognition influencing parameter of each sub-image data.

[0102] Specifically, the meteorological data includes visibility data, light intensity data, wind speed data and wind direction data. A first preset size is set according to actual conditions, and the image data is segmented. After obtaining at least one sub-image data, a preset objective threshold corresponding to the objective index is set, such as a contrast threshold, a brightness threshold, a texture threshold, a color saturation threshold and a noise level threshold. When the objective index exceeds or is less than the preset objective threshold, it can be said that the quality of the sub-image data is low. Therefore, by comparing each objective index of the sub-image data with the corresponding preset objective threshold, a preliminary analysis of the identification influencing parameter, i.e., the first identification influencing parameter, can be obtained. Exemplarily, when there are multiple objective indexes, a weighted method can be used to comprehensively compare all the indexes with the corresponding preset objective threshold to obtain the first identification influencing parameter. Then, the second identification influencing parameter is determined by using the meteorological data and obstacle data and the influencing formula, wherein the normalization coefficient range is [0,1]. Through the influencing formula, it can be analyzed that when the visibility data is low, the exponential function is close to 1, and the visibility data has a greater impact on the second identification parameter; when the visibility data increases, the exponential function decreases, and the visibility data has a smaller impact on the second identification parameter; when the light intensity data is enhanced, the image quality is improved, and the second identification influencing parameter is smaller; otherwise, the second identification influencing parameter is larger. For example, when the light intensity data exceeds the overexposure threshold or is lower than the low light threshold, the light intensity data is set to zero; the obstacle data The image quality is affected proportionally. The greater the obstacle, the greater the impact, and vice versa. The sine square function is used to represent the impact of wind speed. As the wind speed increases, the sine square function first increases and then decreases, indicating the negative impact of increased wind speed on image quality within a certain range. However, excessive wind speed may reduce the impact, such as blowing away fog. The cosine square function is used to represent the impact of wind direction. Since wind direction is a periodic variable, the cosine square function can well represent the impact of wind direction changes on image quality. Finally, the first recognition influencing parameter and the second recognition influencing parameter are integrated to determine the final recognition influencing parameter, which accurately expresses whether there is a problem with the sub-image data.

[0103] Optionally, the segmenting the image data according to the identification influencing parameter to generate at least one defect image and at least one normal image comprises:

[0104] According to the identification influencing parameter and the corresponding sub-image data, extract the sub-image data whose identification influencing parameter is greater than or equal to a preset influencing threshold value to generate an influencing image; extract the sub-image data whose identification influencing parameter is less than the preset influencing threshold value to generate a non-influencing image;

[0105] Connecting adjacent influencing images to generate at least one total defect image, and segmenting the total defect image to generate at least one defect image, wherein the size of each defect image is a second preset size;

[0106] Adjacent unaffected images are connected to generate at least one normal image.

[0107] Specifically, sub-image data whose identification influencing parameters are greater than or equal to a preset influencing threshold value are defined as influencing images, and sub-image data whose identification influencing parameters are less than a preset influencing threshold value are defined as normal images. Then, adjacent non-influencing images are connected to generate at least one normal image, wherein each group of adjacent connected non-influencing images is a normal image. When connecting adjacent influencing images to generate at least one total defect image, it is necessary to use the second preset size as the minimum unit range to divide the total defect image, so as to ensure that each defect image contains only one defect type, so as to facilitate subsequent accurate compensation, thereby improving accuracy.

[0108] Optionally, the integrating the first recognition influencing parameter and the second recognition influencing parameter to determine the recognition influencing parameter of each sub-image data includes:

[0109] The identification influence parameter is generated by using a weighted formula according to the first identification influence parameter and the second identification influence parameter, wherein the weighted formula includes:

[0110] E = a + b*I ;

[0111] Wherein, E is the identification influencing parameter, a is the first weight, b is the second weight, H is the first identification influencing parameter, and I is the second identification influencing parameter.

[0112] Specifically, by adopting a weighted formula, the importance of different factors can be reflected by adjusting the weights, so that the influence of each factor on the final result can be flexibly changed according to the actual situation, which facilitates the determination of the identification influence parameter by combining the advantages of the first identification influence parameter and the second identification influence parameter, thereby improving the accuracy of the identification influence parameter.

[0113] Optionally, determining the defect type of each defect image according to the identification influencing parameter includes:

[0114] Performing grayscale processing on the defect image to obtain a defect grayscale image;

[0115] According to the defect grayscale image, a clarity score is obtained by using a clarity evaluation formula, wherein the clarity evaluation formula includes:

[0116] ;

[0117] in, is the clarity score of the i-th defect image, M is the total number of pixels of the i-th defect image, is the pixel point in the i-th defect image The grayscale value of the defect grayscale image, is the pixel point in the i-th defect image The grayscale value of the defect grayscale image after Gaussian processing;

[0118] The edge detection technology is used to extract edge points in the defect image, and an occlusion score is obtained according to the defect grayscale image, the edge points and an occlusion evaluation formula. The occlusion evaluation formula includes:

[0119] ;

[0120] in, is the occlusion score of the i-th defect image, To adjust the parameters, is the edge point in the i-th defect image The grayscale value of the defect grayscale image, is the edge point The average gray value of the neighboring area;

[0121] The defect type is determined according to the clarity score and the occlusion score.

[0122] Specifically, through the clarity evaluation formula, the difference between the defect grayscale image and the defect grayscale image after blurring can be compared to evaluate clarity. The smaller the difference, the less difference there is between the image before and after blurring. In other words, the image before blurring is also blurry, so the clarity score is low. Conversely, the greater the difference, the higher the clarity score. The occlusion evaluation formula can be used to calculate the difference between the defect grayscale image of the edge point and the average grayscale value of its adjacent area, and use an exponential function to evaluate the occlusion. The pixel difference around the occluded area is usually large, so the occlusion score is higher. Finally, the clarity score and occlusion score are combined to determine the final defect type.

[0123] Optionally, determining the defect type according to the clarity score and the occlusion score includes:

[0124] When the clarity score is greater than a preset multiple of the occlusion score, determining that the defect type is the clarity defect;

[0125] When the clarity score is less than the preset multiple of the occlusion score, the defect type is determined to be the occlusion defect.

[0126] Specifically, the preset multiple is set according to the actual situation. Since the second preset size has been used as the minimum unit range to divide the total defect image, it is ensured that each defect image contains only one defect type. Therefore, when the clarity score is greater than the occlusion score of the preset multiple, that is, the difference between the clarity score and the occlusion score is more than the preset multiple, and the clarity score is much greater than the occlusion score, the defect type is determined to be a clarity defect; when the clarity score is less than the occlusion score of the preset multiple, that is, the clarity score is much less than the occlusion score, the defect type is determined to be an occlusion defect.

[0127] Exemplarily, if the clarity score is equal to a preset multiple of the occlusion score or the clarity score and the occlusion score do not differ by a preset multiple, that is, the difference between the clarity score and the occlusion score is small, the clarity score and the occlusion score are re-obtained.

[0128] Optionally, determining the image acquisition point according to the spatial positioning and the obstacle data includes:

[0129] Extracting three-dimensional coordinates of obstacles according to the spatial positioning and the obstacle data;

[0130] According to the coordinate range of the target area and the three-dimensional coordinates of the obstacle, a particle swarm optimization algorithm is used to obtain the image acquisition points.

[0131] Specifically, the particle swarm optimization algorithm is used to obtain the image acquisition points, and the target of the particle swarm optimization algorithm is set as the image acquisition points within the coordinate range of the target area, the obstacles are set as the three-dimensional coordinates of the obstacles, and the variables are set as the three-dimensional coordinates of the image acquisition points; then the model is constructed, a function is established to calculate whether the line of sight from a given shooting point to the target is blocked by obstacles, and an evaluation function is established to quantify the quality of the shooting location. Then a set of random shooting points is initialized, the particle swarm optimization algorithm is run, the shooting location is iteratively updated until the termination condition is met, and an evaluation function including obstacle occlusion, viewing angle, distance and other factors is constructed. Finally, ray tracing technology is used to determine whether the line of sight from the shooting point to the target intersects with the obstacle to obtain the final image acquisition point. Through the particle swarm optimization algorithm, factors such as obstacle occlusion, viewing angle, and distance can be considered to obtain the optimal image acquisition point.

[0132] Optionally, acquiring the occlusion compensation image according to the image acquisition points includes:

[0133] Acquire a compensation image according to the image acquisition points;

[0134] Respectively extracting first feature points of the compensation image and second feature points of the defect image corresponding to the compensation image;

[0135] Determine a compensation matching point pair using a feature matching algorithm according to the first feature point and the second feature point, wherein the compensation matching point pair includes the first feature point and the second feature point corresponding to each other;

[0136] Cropping the compensated image according to the first feature point in the compensated matching point pair to generate a matching image, and estimating a similarity transformation matrix using the compensated matching point pair;

[0137] According to the similarity transformation matrix, a similarity transformation is performed on the matching image to generate the occlusion compensation image.

[0138] Specifically, a feature detector such as SIFT, SURF, ORB, etc. can be used to respectively extract the first feature points of the compensation image and the second feature points of the defect image corresponding to the compensation image, and then a feature matching algorithm such as the BFMatcher algorithm or the FLANN algorithm is used to find the compensation matching point pairs of the defect image and the compensation image, that is, the feature points corresponding to each other between the defect image and the compensation image. Then, the compensation image is cropped according to the first feature points in the compensation matching point pairs in the compensation image to generate a matching image, and the compensation matching point pairs are used to estimate the similarity transformation matrix. The matching image is rotated, translated and scaled through the similarity transformation matrix to obtain an occlusion compensation image that matches the defect image, with the original image data ratio unchanged.

[0139] Optionally, the multi-source information data includes RGB images and point cloud data, and the multi-modal fusion technology is used to integrate the multi-source information data into a clarity compensated image, including:

[0140] Based on the Point-Fusion network, image features of the RGB image and point cloud features of the point cloud data are respectively extracted;

[0141] The image features and the point cloud features are cropped, downsampled and superimposed in sequence to generate the clarity compensated image.

[0142] Specifically, point cloud data is a common three-dimensional data representation. It consists of a large number of points, each of which has a clear position in three-dimensional space, usually expressed as (X, Y, Z) coordinates. In addition to basic geometric position information, points in a point cloud can also carry additional information, such as color or reflection intensity. RGB images are images in the form of three-channel matrices. Point-Fusion network is a network structure used to fuse RGB images and point cloud data for three-dimensional target detection. The network aims to combine data from two different sensors to improve the accuracy of detection. Through the Point-Fusion network, the image features of the RGB image and the point cloud features of the point cloud data can be extracted respectively, and the image features and point cloud features are sequentially cropped, downsampled and superimposed to fuse the image features and point cloud features to obtain a clarity compensated image.

[0143] Optionally, the generating a target recognition result by using a target recognition technology according to the image data to be recognized includes:

[0144] Inputting the image data to be identified into a multi-scale feature extraction network to generate a first scale feature, a second scale feature, and a third scale feature, wherein the scales of the first scale feature, the second scale feature, and the third scale feature decrease in sequence;

[0145] Through the Darknet53 network, extracting first-scale depth features, second-scale depth features, and third-scale depth features of the first-scale features, the second-scale features, and the third-scale features respectively;

[0146] The first-scale depth feature, the second-scale depth feature, and the third-scale depth feature are input into a YOLO target detection network to generate the target recognition result.

[0147] Specifically, the multi-scale feature extraction network is a network structure used to improve the performance of target detection algorithms on targets of different sizes, and can improve the expressiveness of target recognition. The multi-scale feature extraction network includes a CBL structure, a ResX structure, a Focus structure, and an SPP structure connected in sequence. The CBL structure consists of a convolutional layer (Conv), a batch normalization layer (BatchNormalization, BN), and a LeakyReLU activation function layer, which is used to extract features and perform nonlinear transformations. The ResX structure is composed of a CBL network layer and X residual components connected. The residual component is connected by two CBL structures, and the input data is subjected to matrix addition operations with the data processed by the two CBL structures. The ResX structure is used to alleviate the gradient vanishing problem in deep networks, so that the network can be trained deeper. The Focus structure is used to divide the input data into 4 slice matrices after slicing operations, and then re-splice them, which can improve the network's spatial resolution of the input data. The SPP structure inputs the input data into the CBL network structure, and then performs Maxpool pooling operations on the feature matrices of the input data, and then splices the pooled feature matrices. After the image data to be identified is input into the multi-scale feature extraction network, the multi-scale feature extraction network performs preliminary feature extraction through the CBL structure, and uses the ResX structure to deepen the network layer to maintain the richness of the features. The Focus structure is then used to improve the resolution, and finally the SPP structure is used to realize multi-scale feature extraction to generate the first scale feature, the second scale feature, and the third scale feature.

[0148] Darknet-53 is a deep convolutional neural network (CNN), which is the backbone network designed for the YOLO (You Only Look Once) target detection network. Darknet-53 can achieve a relatively fast speed while maintaining high accuracy. After extracting the first-scale features, second-scale features, and third-scale features, the first-scale features, second-scale features, and third-scale features are fused with the Darknet53 network to extract the first-scale depth features, second-scale depth features, and third-scale depth features of the first-scale features, second-scale features, and third-scale features; finally, the first-scale depth features, second-scale depth features, and third-scale depth features are input into the YOLO target detection network to generate accurate target recognition results.

[0149] like Figure 2 As shown, an embodiment of the present invention provides a target recognition device 200 suitable for a polar unmanned boat, comprising:

[0150] An acquisition module 210 is used to acquire image data and environmental data of a polar unmanned vehicle target area, wherein the environmental data includes meteorological data and obstacle data;

[0151] An influence module 220, configured to extract objective indicators of the image data, and determine identification influence parameters according to the meteorological data, the obstacle data and the objective indicators;

[0152] A segmentation module 230, configured to segment the image data according to the identification influencing parameter to generate at least one defect image and at least one normal image;

[0153] A defect module 240, configured to determine a defect type of each defect image according to the identification influencing parameter, wherein the defect type includes an occlusion defect and a clarity defect;

[0154] The occlusion module 250 is used to determine the spatial location of the occlusion defect using image positioning technology, determine the image acquisition point according to the spatial positioning and the obstacle data, and obtain the occlusion compensation image according to the image acquisition point;

[0155] A clarity module 260 is used to obtain multi-source information data corresponding to the clarity defect space for the clarity defect, and integrate the multi-source information data into a clarity compensation image by using a multimodal fusion technology, wherein the clarity defect space is a space in the target area corresponding to the clarity defect;

[0156] The recognition module 270 is used to generate image data to be recognized based on the occlusion compensation image and / or the clarity compensation image corresponding to each of the defective images, and all the normal images, and generate a target recognition result based on the image data to be recognized using target recognition technology.

[0157] like Figure 3 As shown, an electronic device 300 provided by an embodiment of the present invention includes a memory 310 and a processor 320; the memory 310 is used to store a computer program; the processor 320 is used to implement the target recognition method applicable to a polar unmanned boat as described above when executing the computer program.

[0158] In other words, an electronic device 300 includes a memory 310 and a processor 320 coupled to the memory 310; the memory 310 is configured to store a computer program; and the processor 320 is configured to perform the following operations when executing the computer program:

[0159] Acquire image data and environmental data of a polar unmanned boat target area, wherein the environmental data includes meteorological data and obstacle data;

[0160] extracting objective indicators of the image data, and determining identification influencing parameters according to the meteorological data, the obstacle data and the objective indicators;

[0161] Segmenting the image data according to the identification influencing parameter to generate at least one defect image and at least one normal image;

[0162] Determining the defect type of each of the defect images according to the identification influencing parameters, wherein the defect type includes an occlusion defect and a clarity defect;

[0163] For the occlusion defect, an image positioning technology is used to determine the spatial positioning of the occlusion defect, an image acquisition point is determined according to the spatial positioning and the obstacle data, and an occlusion compensation image is obtained according to the image acquisition point;

[0164] For the clarity defect, multi-source information data corresponding to the clarity defect space is acquired, and the multi-modal fusion technology is used to integrate the multi-source information data into a clarity compensation image;

[0165] Image data to be identified are generated based on the occlusion compensation image and / or the clarity compensation image corresponding to each of the defective images, as well as all the normal images. Target recognition results are generated based on the image data to be identified using target recognition technology.

[0166] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the target recognition method applicable to a polar unmanned boat as described above is implemented.

[0167] In other words, a non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor performs the following operations:

[0168] Acquire image data and environmental data of a polar unmanned boat target area, wherein the environmental data includes meteorological data and obstacle data;

[0169] extracting objective indicators of the image data, and determining identification influencing parameters according to the meteorological data, the obstacle data and the objective indicators;

[0170] Segmenting the image data according to the identification influencing parameter to generate at least one defect image and at least one normal image;

[0171] Determining the defect type of each of the defect images according to the identification influencing parameters, wherein the defect type includes an occlusion defect and a clarity defect;

[0172] For the occlusion defect, an image positioning technology is used to determine the spatial positioning of the occlusion defect, an image acquisition point is determined according to the spatial positioning and the obstacle data, and an occlusion compensation image is obtained according to the image acquisition point;

[0173] For the clarity defect, multi-source information data corresponding to the clarity defect space is acquired, and the multi-modal fusion technology is used to integrate the multi-source information data into a clarity compensation image;

[0174] Image data to be identified are generated based on the occlusion compensation image and / or the clarity compensation image corresponding to each of the defective images, as well as all the normal images. Target recognition results are generated based on the image data to be identified using target recognition technology.

[0175] An electronic device 300 that can be used as a server or client of the present invention will now be described, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device 300 is intended to represent various forms of digital electronic computer equipment, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 300 can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the present invention described and / or required herein.

[0176] The electronic device 300 includes a computing unit, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) or a computer program loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for the operation of the device can also be stored. The computing unit, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0177] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc. In the present application, the unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment of the present invention. In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0178] Although the present invention is disclosed as above, the protection scope of the present invention is not limited thereto. Those skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will fall within the protection scope of the present invention.

Claims

1. A target recognition method suitable for polar unmanned boats, characterized in that: include: Acquire image data and environmental data of a polar unmanned boat target area, wherein the environmental data includes meteorological data and obstacle data; extracting objective indicators of the image data, and determining identification influencing parameters according to the meteorological data, the obstacle data and the objective indicators; Segmenting the image data according to the identification influencing parameter to generate at least one defect image and at least one normal image; Determining the defect type of each of the defect images according to the identification influencing parameters, wherein the defect type includes an occlusion defect and a clarity defect; For the occlusion defect, an image positioning technology is used to determine the spatial positioning of the occlusion defect, an image acquisition point is determined according to the spatial positioning and the obstacle data, and an occlusion compensation image is obtained according to the image acquisition point; For the clarity defect, multi-source information data corresponding to the clarity defect space is acquired, and the multi-modal fusion technology is used to integrate the multi-source information data into a clarity compensation image, wherein the clarity defect space is a space in the target area corresponding to the clarity defect; Generate image data to be identified based on the occlusion compensation image and / or the clarity compensation image corresponding to each of the defective images, and all the normal images, and generate a target recognition result based on the image data to be identified using a target recognition technology; The step of determining the identification influencing parameters according to the meteorological data, the obstacle data and the objective indicator comprises: Segmenting the image data to obtain at least one sub-image data, wherein the size of each sub-image data is a first preset size; Determining a first recognition influencing parameter of each of the sub-image data according to the objective indicator and a preset objective threshold; According to the meteorological data and the obstacle data, an influence formula is used to determine a second identification influence parameter, wherein the influence formula includes: ； in, I for the second identification influencing parameter, , , , and are normalized coefficients, , , and are constants, V is the visibility data of the meteorological data, L is the light intensity data of the meteorological data, is the obstacle data, is the wind speed data of the meteorological data, The wind direction data of the meteorological data; Integrating the first recognition influencing parameter and the second recognition influencing parameter to determine the recognition influencing parameter of each sub-image data; The step of determining the defect type of each defect image according to the identification influencing parameter includes: Performing grayscale processing on the defect image to obtain a defect grayscale image; According to the defect grayscale image, a clarity score is obtained by using a clarity evaluation formula, wherein the clarity evaluation formula includes: ； in, is the clarity score of the i-th defect image, M is the total number of pixels of the i-th defect image, is the pixel point in the i-th defect image The grayscale value of the defect grayscale image, is the pixel point in the i-th defect image The grayscale value of the defect grayscale image after Gaussian processing; The edge detection technology is used to extract the edge points in the defect image, and the occlusion score is obtained according to the defect grayscale image, the edge points and the occlusion evaluation formula. The occlusion evaluation formula includes: ； in, is the occlusion score of the i-th defect image, To adjust the parameters, is the edge point in the i-th defect image The grayscale value of the defect grayscale image, is the edge point The average gray value of the neighboring area; The defect type is determined according to the clarity score and the occlusion score.

2. The target recognition method applicable to polar unmanned boats according to claim 1 is characterized in that: The step of segmenting the image data according to the identification influencing parameter to generate at least one defect image and at least one normal image comprises: According to the identification influencing parameter and the corresponding sub-image data, extract the sub-image data whose identification influencing parameter is greater than or equal to a preset influencing threshold value to generate an influencing image; extract the sub-image data whose identification influencing parameter is less than the preset influencing threshold value to generate a non-influencing image; Connecting adjacent influencing images to generate at least one total defect image, and segmenting the total defect image to generate at least one defect image, wherein the size of each defect image is a second preset size; Adjacent unaffected images are connected to generate at least one normal image.

3. The target recognition method applicable to polar unmanned boats according to claim 1 is characterized in that: The step of integrating the first recognition influencing parameter and the second recognition influencing parameter to determine the recognition influencing parameter of each sub-image data includes: The identification influence parameter is generated by using a weighted formula according to the first identification influence parameter and the second identification influence parameter, wherein the weighted formula includes: E ＝ a ＋ b*I ； Wherein, E is the identification influencing parameter, a is the first weight, b is the second weight, H is the first identification influencing parameter, and I is the second identification influencing parameter.

4. The target recognition method applicable to polar unmanned boats according to claim 1, characterized in that: The determining the defect type according to the clarity score and the occlusion score includes: When the clarity score is greater than a preset multiple of the occlusion score, determining that the defect type is the clarity defect; When the clarity score is less than the preset multiple of the occlusion score, the defect type is determined to be the occlusion defect.

5. The target recognition method applicable to polar unmanned boats according to claim 1, characterized in that: The step of determining the image acquisition point according to the spatial positioning and the obstacle data includes: Extracting three-dimensional coordinates of obstacles according to the spatial positioning and the obstacle data; According to the coordinate range of the target area and the three-dimensional coordinates of the obstacle, a particle swarm optimization algorithm is used to obtain the image acquisition points.

6. The target recognition method applicable to polar unmanned boats according to claim 1, characterized in that: The step of acquiring the occlusion compensation image according to the image acquisition points includes: Acquire a compensation image according to the image acquisition points; Respectively extracting a first feature point of the compensation image and a second feature point of the defect image corresponding to the compensation image; Determine a compensation matching point pair using a feature matching algorithm according to the first feature point and the second feature point, wherein the compensation matching point pair includes the first feature point and the second feature point corresponding to each other; Cropping the compensated image according to the first feature point in the compensated matching point pair to generate a matching image, and estimating a similarity transformation matrix using the compensated matching point pair; According to the similarity transformation matrix, a similarity transformation is performed on the matching image to generate the occlusion compensation image.

7. The target recognition method applicable to polar unmanned boats according to claim 1, characterized in that: The multi-source information data includes RGB images and point cloud data, and the multi-modal fusion technology is used to integrate the multi-source information data into a clarity compensated image, including: Based on the Point-Fusion network, image features of the RGB image and point cloud features of the point cloud data are respectively extracted; The image features and the point cloud features are cropped, downsampled and superimposed in sequence to generate the clarity compensated image.

8. The target recognition method applicable to polar unmanned boats according to claim 1, characterized in that: The step of using target recognition technology to generate a target recognition result based on the image data to be recognized includes: Inputting the image data to be identified into a multi-scale feature extraction network to generate a first scale feature, a second scale feature, and a third scale feature, wherein the scales of the first scale feature, the second scale feature, and the third scale feature decrease in sequence; Through the Darknet53 network, extracting first-scale depth features, second-scale depth features, and third-scale depth features of the first-scale features, the second-scale features, and the third-scale features respectively; The first-scale depth feature, the second-scale depth feature, and the third-scale depth feature are input into a YOLO target detection network to generate the target recognition result.

Images