Unmanned platform environment situation awareness self-evaluation system and method

Abstract

The invention relates to the technical field of unmanned system autonomous reconnaissance, and discloses an unmanned platform environment situation awareness self-evaluation system and method, and the method comprises the steps: obtaining specific scene situation data and public data, and carrying out the processing of the specific scene situation data; learning the processed data and public data to obtain a learning model, combining the learning model with a convolutional neural network, and performing fine tuning to obtain a picture quality prediction model; screening the picture quality prediction model to obtain a final picture quality prediction model; evaluating the plurality of situation pictures to be collected to obtain evaluation scores corresponding to the plurality of situation pictures, and storing the situation picture with the highest evaluation score in the database as the most valuable intelligence picture in the current position and the current environment if the condition is satisfied. Otherwise, re-collecting the situation pictures, and evaluating the re-collected situation pictures until the predetermined requirements are met; and carrying out transmission pre-processing on the situation picture with the highest evaluation score, and outputting the processed picture to a ground command center.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned system autonomous reconnaissance, and in particular to an unmanned platform environmental situation awareness self-assessment system and method. Background Art

[0002] Unmanned autonomous platforms are known for their low cost, flexibility, and the fact that they do not require deep involvement from operators. In recent years, unmanned platforms have increasingly demonstrated their significant advantages in target search and situational awareness in key defense and park guard missions. In the execution of unmanned platform tasks, situational awareness of the environment is crucial. Situational awareness usually involves the use of a variety of sensors (such as cameras, lidar, etc.) to collect environmental data and extract useful information through image processing and pattern recognition technology. In tasks such as patrolling or monitoring, unmanned platforms often need to obtain environmental information in real time and transmit it back in a timely manner. When important intelligence is detected, the platform usually takes pictures for further analysis and decision-making.

[0003] However, existing unmanned platforms often face a common problem when performing environmental situational awareness: among the large number of collected images, only a few actually carry valuable information. If the platform repeatedly transmits multiple photos or video streams of the same content, it will cause a waste of bandwidth and affect the efficiency and real-time performance of data transmission. In addition, handing the screening task over to the superior command system reduces the autonomy of the unmanned platform, making it less intelligent in the decision-making process, and also increases the data storage and processing burden of the command center.

[0004] Current unmanned platform situational awareness technologies usually focus on data collection and information transmission, while ignoring the self-assessment and optimization of image quality. The lack of an effective front-end environmental situational awareness self-assessment mechanism results in the unmanned platform being unable to autonomously select a series of situation images with better transmission quality, thus showing a certain degree of passivity in task execution. This situation not only increases the burden of data transmission, but may also affect the timeliness and accuracy of intelligence. If only a simple image quality evaluation algorithm is used to select the best quality image for transmission, the comprehensiveness of intelligence may be affected. Summary of the invention

[0005] The purpose of the present invention is to overcome the deficiencies of the prior art and to provide an unmanned platform environmental situation awareness self-assessment system and method, which can solve the problems in the above-mentioned prior art.

[0006] The technical solution of the present invention is a self-assessment method for environmental situation awareness of an unmanned platform, wherein the method comprises:

[0007] Obtaining specific scenario situation data and public data, and processing the specific scenario situation data;

[0008] The processed data and public data are learned using the twin network to obtain a learning model, which is then combined with the convolutional neural network. The combined model is fine-tuned and used as an image quality prediction model.

[0009] The picture quality prediction model is screened according to the Pearson linear correlation coefficient to obtain the final picture quality prediction model;

[0010] According to the final image quality prediction model, the multiple situation images to be collected are evaluated to obtain evaluation scores corresponding to each of the multiple situation images, and whether the multiple situation images meet the predetermined requirements is judged according to the evaluation scores. If they meet the requirements, the situation image with the highest evaluation score is stored in the database as the most valuable intelligence image in the current position and the current environment. Otherwise, the situation image is re-collected and the multiple situation images after re-collection are evaluated until the predetermined requirements are met;

[0011] The situation picture with the highest evaluation score is processed before transmission, and the processed situation picture with the highest evaluation score is output to the ground command center.

[0012] Preferably, processing the specific scene situation data includes:

[0013] The situation data of each specific scene is divided into multiple quality levels by multiple personnel, and the divided data is labeled with multiple quality scores by multiple personnel;

[0014] Perform data cleaning on the labeled data to remove abnormal quality scores;

[0015] The remaining quality scores of each specific scene situation data are averaged, and the average value is used as the annotation score of the corresponding specific scene situation data.

[0016] Preferably, the Pearson linear correlation coefficient is calculated by the following formula:

[0017]

[0018] Among them, PLCC represents the Pearson linear correlation coefficient, N represents the number of all situation pictures provided, and y i , They represent the annotation score and model prediction score of the i-th situation picture respectively, They represent the mean of the annotation scores and the mean of the prediction scores, respectively.

[0019] Preferably, judging whether the plurality of situation pictures meet predetermined requirements according to the evaluation scores includes:

[0020] When the evaluation scores of situation pictures greater than a predetermined proportion in the multiple situation pictures are less than a threshold, it is determined that the multiple situation pictures do not meet the predetermined requirement;

[0021] When the evaluation scores of situation pictures greater than a predetermined proportion in the multiple situation pictures are greater than or equal to a threshold, it is determined that the multiple situation pictures meet the predetermined requirement.

[0022] Preferably, the pre-transmission processing of the situation picture with the highest evaluation score includes:

[0023] The situation pictures with the highest evaluation scores are compressed and encrypted in turn.

[0024] The present invention also provides an unmanned platform environmental situation awareness self-assessment system, wherein the system comprises:

[0025] The data set collection module is used to obtain specific scenario situation data and public data, and process the specific scenario situation data;

[0026] The image quality assessment model training module is used to use the twin network to learn the processed data and public data to obtain a learning model, combine the learning model with the convolutional neural network, and fine-tune the combined model as an image quality prediction model;

[0027] A model screening module is used to screen the picture quality prediction model according to the Pearson linear correlation coefficient to obtain the final picture quality prediction model;

[0028] An image evaluation module is used to evaluate the multiple situation pictures to be collected according to the final picture quality prediction model, obtain the evaluation scores corresponding to the multiple situation pictures, and judge whether the multiple situation pictures meet the predetermined requirements according to the evaluation scores. If they meet the requirements, the situation picture with the highest evaluation score is stored in the database as the most valuable intelligence picture in the current position and the current environment. Otherwise, the situation pictures are collected again and the multiple situation pictures collected again are evaluated until they meet the predetermined requirements.

[0029] The situation image autonomous transmission module is used to process the situation image with the highest evaluation score before transmission, and output the processed situation image with the highest evaluation score to the ground command center.

[0030] Preferably, processing the specific scene situation data includes:

[0031] The situation data of each specific scene is divided into multiple quality levels by multiple personnel, and the divided data is labeled with multiple quality scores by multiple personnel;

[0032] Perform data cleaning on the labeled data to remove abnormal quality scores;

[0033] The remaining quality scores of each specific scene situation data are averaged, and the average value is used as the annotation score of the corresponding specific scene situation data.

[0034] Preferably, the Pearson linear correlation coefficient is calculated by the following formula:

[0035]

[0036] Among them, PLCC represents the Pearson linear correlation coefficient, N represents the number of all situation pictures provided, and y i , They represent the annotation score and model prediction score of the i-th situation picture respectively, They represent the mean of the annotation scores and the mean of the prediction scores, respectively.

[0037] Preferably, judging whether the plurality of situation pictures meet predetermined requirements according to the evaluation scores includes:

[0038] When the evaluation scores of situation pictures greater than a predetermined proportion in the multiple situation pictures are less than a threshold, it is determined that the multiple situation pictures do not meet the predetermined requirement;

[0039] When the evaluation scores of situation pictures greater than a predetermined proportion in the multiple situation pictures are greater than or equal to a threshold, it is determined that the multiple situation pictures meet the predetermined requirement.

[0040] Preferably, the pre-transmission processing of the situation picture with the highest evaluation score includes:

[0041] The situation pictures with the highest evaluation scores are compressed and encrypted in turn.

[0042] Through the above technical solution, RankIQA can be used to build multiple image quality prediction models, and PLCC (Pearson Linear Correlation Coefficient) screening models can be used to perceive and self-evaluate the environmental situation of the unmanned platform, and low-quality images can be identified and excluded before the image is submitted to the ground command center, requiring the unmanned platform to re-shoot and upload. This not only controls the number of low-quality images from the source, reduces their impact on the regional situation awareness and cognition of the entire system, but also significantly improves the overall clarity and sorting consistency of the situation image. Since low-quality images can be identified and excluded at the first time, it is beneficial for the unmanned platform to reduce the task re-planning caused by the need to re-shoot, improve the work efficiency of the unmanned platform, and at the same time, during the unmanned platform's execution of tasks, the system reduces the false alarm rate when performing regional situation awareness, effectively improving the overall efficiency of the system. Compared with traditional image evaluation methods, the present invention not only avoids repeated shooting caused by image quality problems and reduces energy consumption, but also simplifies the operation process through automated evaluation, reduces the burden of background command, and greatly improves the ability and efficiency of the unmanned platform to perform tasks in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The included drawings are used to provide a further understanding of the embodiments of the present invention, which constitute a part of the specification, are used to illustrate the embodiments of the present invention, and together with the text description, explain the principles of the present invention. Obviously, the drawings in the following description are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 A flow chart of a method for self-assessment of environmental situation awareness of an unmanned platform provided by an embodiment of the present invention;

[0045] Figure 2 A flowchart of an unmanned platform environmental situation awareness self-assessment method provided by another embodiment of the present invention. DETAILED DESCRIPTION

[0046] Below in conjunction with accompanying drawing, specific embodiment of the present invention is described in detail.In the following description, for explanation and not limiting purpose, set forth specific details, to help fully understand the present invention.But it is obvious to those skilled in the art that also can practice the present invention in other embodiment that breaks away from these specific details.

[0047] It should be noted that, in order to avoid obscuring the present invention due to unnecessary details, only the device structure and / or processing steps closely related to the solution according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0048] Figure 1 A flowchart of an unmanned platform environmental situation awareness self-assessment method provided in an embodiment of the present invention.

[0049] like Figure 1 As shown, the present invention provides an unmanned platform environmental situation awareness self-assessment method, wherein the method comprises:

[0050] Obtaining specific scenario situation data and public data, and processing the specific scenario situation data;

[0051] That is, a large number of labeled situation pictures (data sets to be labeled) are provided to the model for learning, among which the situation data of specific scenarios need to be processed (i.e., scored and labeled), and the public data are labeled data sets.

[0052] The processed data and public data are learned using the twin network to obtain a learning model, which is then combined with the convolutional neural network. The combined model is fine-tuned and used as an image quality prediction model (training situation image self-assessment model).

[0053] For example, the RankIQA model is used for learning to obtain a learning model.

[0054] The picture quality prediction model is screened according to the Pearson linear correlation coefficient to obtain the final picture quality prediction model;

[0055] That is, the Pearson linear correlation coefficient can be used to screen the image quality prediction model.

[0056] Through this step, one or more final picture quality prediction models can be obtained.

[0057] According to the final image quality prediction model, the multiple situation images to be collected are evaluated to obtain evaluation scores corresponding to each of the multiple situation images, and whether the multiple situation images meet the predetermined requirements is judged according to the evaluation scores. If they meet the requirements, the situation image with the highest evaluation score is stored in the database as the most valuable intelligence image in the current position and the current environment. Otherwise, the situation image is re-collected and the multiple situation images after re-collection are evaluated until the predetermined requirements are met;

[0058] The situation picture with the highest evaluation score is processed before transmission, and the processed situation picture with the highest evaluation score is output to the ground command center, that is, autonomous situation transmission.

[0059] Through the above technical solution, RankIQA can be used to build multiple image quality prediction models, and PLCC (Pearson Linear Correlation Coefficient) screening models can be used to perceive and self-evaluate the environmental situation of the unmanned platform, and low-quality images can be identified and excluded before the image is submitted to the ground command center, requiring the unmanned platform to re-shoot and upload. This not only controls the number of low-quality images from the source, reduces their impact on the regional situation awareness and cognition of the entire system, but also significantly improves the overall clarity and sorting consistency of the situation image. Since low-quality images can be identified and excluded at the first time, it is beneficial for the unmanned platform to reduce the task re-planning caused by the need to re-shoot, improve the work efficiency of the unmanned platform, and at the same time, during the unmanned platform's execution of tasks, the system reduces the false alarm rate when performing regional situation awareness, effectively improving the overall efficiency of the system. Compared with traditional image evaluation methods, the present invention not only avoids repeated shooting caused by image quality problems and reduces energy consumption, but also simplifies the operation process through automated evaluation, reduces the burden of background command, and greatly improves the ability and efficiency of the unmanned platform to perform tasks in complex environments.

[0060] According to an embodiment of the present invention, for data set acquisition, a batch of pictures can be prepared for model training. For example, picture preparation can include the following methods: 1) Use an unmanned platform to shoot a batch of situation data. The shooting scenes should be as complex and diverse as possible and include actual usage scenarios, and the image quality should be unbiased, including high-quality, medium-quality, and low-quality pictures (high-quality pictures such as normal light, accurate focus, and no compression; medium-quality pictures such as dim light, slightly blurred pictures, light compression, etc.; low-quality pictures such as poor light, overexposure, serious out-of-focus problems, and highly compressed pictures); 2) Use traditional image processing methods to expand the data set based on the existing data set, including pixelation, sharpening, Gaussian blur, motion blur, adding noise, adjusting saturation contrast, image compression, Gaussian blur, JPEG compression, adjusting contrast saturation, etc., so as to collect a batch of pictures of various qualities; 3) Collect annotated public data sets.

[0061] According to an embodiment of the present invention, processing the specific scene situation data includes:

[0062] The situation data of each specific scene is divided into multiple quality levels by multiple personnel, and the divided data is labeled with multiple quality scores by multiple personnel;

[0063] Perform data cleaning on the labeled data to remove abnormal quality scores;

[0064] The remaining quality scores of each specific scene situation data are averaged, and the average value is used as the annotation score of the corresponding specific scene situation data.

[0065] For example, for the standard determination of the situation picture data set: the quality of the situation picture can be divided into [0, x]; specifically, the quality of the situation picture to be labeled is divided into x levels as the standard for subsequent labeling.

[0066] For data labeling: manual labeling can be used to label the data set; specifically, the situation picture data set collected by the unmanned platform and obtained by simple image processing can be distributed to N people for labeling. The data set distribution can have overlaps, and each picture is repeatedly labeled by no less than 5 people, and the labeling results are finally collected;

[0067] For data cleaning: the abnormal data manually annotated in the situation picture dataset can be removed. Specifically, considering that the cognitive levels of different annotators are different, the annotation results of the same picture may be different. Therefore, the image quality score can be filtered by estimating the confidence interval, that is, the image score within the confidence interval is retained, and the image score outside the confidence interval is regarded as an outlier, and the annotation result (abnormal quality score) is eliminated.

[0068] For result calculation: the annotation scores of each image may be averaged; specifically, the average of the filtered data of each image may be taken as the annotation score of the situation image.

[0069] According to an embodiment of the present invention, using a twin network to learn the processed data and public data to obtain a learning model, combining the learning model with a convolutional neural network, and fine-tuning the combined model as a picture quality prediction model may include:

[0070] The RankIQA model (RankIQA: Learning from Rankings for No-reference Image Quality Assessment.) is adopted. The RankIQA model is based on the VGG16 network architecture and uses the Siamese network (Twin network is a kind of neural network structure, which consists of two or more identical networks. Twin networks are used to solve various similarity-related tasks, such as face recognition, speech recognition, target tracking and other problems. The basic idea of ​​the twin network is to input the input data into two or more identical neural networks at the same time. The two networks share the same weights and parameters during training. By learning the representation of the input data in the two networks, the twin network can calculate the similarity between the two input samples.) to learn the representation features of the order of pictures in the data set, and then migrate this feature representation method to the traditional CNN network (convolutional neural network), and after fine-tuning, it is used as the final picture quality prediction model.

[0071] As a result, many different models can be produced in the model training phase.

[0072] According to one embodiment of the present invention, the Pearson linear correlation coefficient is calculated by the following formula:

[0073]

[0074] Among them, PLCC represents the Pearson linear correlation coefficient, N represents the number of all situation pictures provided, and y i , They represent the annotation score and model prediction score of the i-th situation picture respectively, They represent the mean of the annotation scores and the mean of the prediction scores, respectively.

[0075] vector and The more similar, the closer the PLCC value is to 1, which means that the correlation between the image score predicted by the current evaluated model and the manually annotated score is higher, that is, the effect of the evaluated model is better. Finally, the model with the highest PLCC value is selected as the quality evaluation model of the situation pictures collected by the unmanned platform (the final picture quality prediction model).

[0076] That is, the model is evaluated by comparing the differences and correlations between the model's predicted values ​​and the labeled values.

[0077] According to an embodiment of the present invention, judging whether a plurality of situation pictures meet predetermined requirements according to the evaluation scores includes:

[0078] When the evaluation scores of situation pictures greater than a predetermined proportion in the multiple situation pictures are less than a threshold, it is determined that the multiple situation pictures do not meet the predetermined requirement;

[0079] When the evaluation scores of situation pictures greater than a predetermined proportion in the multiple situation pictures are greater than or equal to a threshold, it is determined that the multiple situation pictures meet the predetermined requirement.

[0080] According to an embodiment of the present invention, the pre-transmission processing of the situation picture with the highest evaluation score includes:

[0081] The situation pictures with the highest evaluation scores are compressed and encrypted in turn.

[0082] For example, you can first perform image compression, that is, compress the selected high-quality images (such as using JPEG compression) to reduce the amount of data required for transmission and reduce the risk of discovering unmanned platforms through traffic sniffing. Secondly, encrypt the compressed image data to ensure the security of data transmission. The encryption algorithm can use a symmetric encryption algorithm such as SM4 to ensure the absolute security of the data, or an identity-based encryption algorithm such as SM9 to quickly verify that the collected situation pictures are from a trusted unmanned platform to prevent information deception. Then you can establish a connection with the ground command center, and after establishing a connection through a wireless communication network (such as a satellite link, 4G / 5G network, etc.), upload the encrypted image data to the ground command center, and the ground command center decrypts the situation picture according to the pre-agreed secret key to restore the complete high-quality situation intelligence picture of the current location of the unmanned platform.

[0083] An embodiment of the present invention provides an unmanned platform environment situation awareness self-assessment system, wherein the system includes:

[0084] The data set collection module is used to obtain specific scenario situation data and public data, and process the specific scenario situation data;

[0085] The image quality assessment model training module is used to use the twin network to learn the processed data and public data to obtain a learning model, combine the learning model with the convolutional neural network, and fine-tune the combined model as an image quality prediction model;

[0086] A model screening module is used to screen the picture quality prediction model according to the Pearson linear correlation coefficient to obtain the final picture quality prediction model;

[0087] An image evaluation module is used to evaluate the multiple situation pictures to be collected according to the final picture quality prediction model, obtain the evaluation scores corresponding to the multiple situation pictures, and judge whether the multiple situation pictures meet the predetermined requirements according to the evaluation scores. If they meet the requirements, the situation picture with the highest evaluation score is stored in the database as the most valuable intelligence picture in the current position and the current environment. Otherwise, the situation pictures are collected again and the multiple situation pictures collected again are evaluated until they meet the predetermined requirements.

[0088] The situation image autonomous transmission module is used to process the situation image with the highest evaluation score before transmission, and output the processed situation image with the highest evaluation score to the ground command center.

[0089] Through the above technical solution, PLCC (Pearson Linear Correlation Coefficient) can be used to perceive and self-evaluate the environmental situation of the unmanned platform, and low-quality images can be identified and excluded before the image is submitted to the ground command center, requiring the unmanned platform to re-shoot and upload. This not only controls the number of low-quality images from the source, reduces their impact on the regional situation awareness and cognition of the entire system, but also significantly improves the overall clarity and sorting consistency of the situation image. Since low-quality images can be identified and excluded at the first time, it is beneficial for the unmanned platform to reduce the task re-planning caused by the need to re-shoot, improve the work efficiency of the unmanned platform, and at the same time, during the unmanned platform's execution of tasks, the system reduces the false alarm rate when performing regional situation awareness, effectively improving the overall efficiency of the system. Compared with the traditional image evaluation method, the present invention not only avoids repeated shooting due to image quality problems and reduces energy consumption, but also simplifies the operation process through automated evaluation, reduces the burden of background command, and greatly improves the ability and efficiency of the unmanned platform to perform tasks in complex environments.

[0090] According to an embodiment of the present invention, processing the specific scene situation data includes:

[0091] The situation data of each specific scene is divided into multiple quality levels by multiple personnel, and the divided data is labeled with multiple quality scores by multiple personnel;

[0092] Perform data cleaning on the labeled data to remove abnormal quality scores;

[0093] The remaining quality scores of each specific scene situation data are averaged, and the average value is used as the annotation score of the corresponding specific scene situation data.

[0094] According to one embodiment of the present invention, the Pearson linear correlation coefficient is calculated by the following formula:

[0095]

[0096] Among them, PLCC represents the Pearson linear correlation coefficient, N represents the number of all situation pictures provided, and y i , They represent the annotation score and model prediction score of the i-th situation picture respectively, They represent the mean of the annotation scores and the mean of the prediction scores, respectively.

[0097] According to an embodiment of the present invention, judging whether a plurality of situation pictures meet predetermined requirements according to the evaluation scores includes:

[0098] When the evaluation scores of situation pictures greater than a predetermined proportion in the multiple situation pictures are less than a threshold, it is determined that the multiple situation pictures do not meet the predetermined requirement;

[0099] When the evaluation scores of situation pictures greater than a predetermined proportion in the multiple situation pictures are greater than or equal to a threshold, it is determined that the multiple situation pictures meet the predetermined requirement.

[0100] According to an embodiment of the present invention, the pre-transmission processing of the situation picture with the highest evaluation score includes:

[0101] The situation pictures with the highest evaluation scores are compressed and encrypted in turn.

[0102] The following describes an unmanned platform environmental situation awareness self-assessment system and method according to the present invention with reference to examples. The system and method are used to perform quality assessment on collected images and select the best series of situation images for transmission during the execution of tasks by the unmanned platform.

[0103] 1. System architecture

[0104] The system of this embodiment includes the following main components:

[0105] Image acquisition module: A camera installed on an unmanned platform, used to collect situation images in the environment in real time.

[0106] Dataset collection module: collects the original images collected by the unmanned platform and performs preliminary processing on the images for dataset expansion, such as pixelation, sharpening, Gaussian blur, adding noise, adjusting saturation and contrast, image compression, Gaussian blur, JPEG compression, adjusting contrast and saturation, etc.

[0107] Image quality assessment model training module: trains one or more assessment models for assessing the overall quality and local detail quality of an image.

[0108] Model screening module: Many different models will be generated in the model training module. One or several models with high PLCC scores will be selected for use based on the PLCC evaluation index.

[0109] Image evaluation module: It evaluates the situation image according to the self-evaluation model obtained by the model screening module, and selects the best image for transmission based on the evaluation results.

[0110] Situation image autonomous transmission module: responsible for transmitting the selected images to the upper-level ground command and control center.

[0111] 2. Image Quality Assessment Method

[0112] Figure 2 It is a self-assessment method for environmental situation awareness of unmanned platforms, which is explained in detail through specific steps below.

[0113] (1) Data preparation: including data set preparation, situation picture data set quality level determination, data labeling, data cleaning, and labeling score calculation.

[0114] There are three methods for preparing data sets. The first is to use the camera installed on the unmanned platform to collect situation images in the environment in real time. In this process, the following matters need to be noted: the scenes should be as complex and diverse as possible, such as indoor, outdoor, densely built areas, forests, urban blocks and other scenes; the image quality should be distributed unbiasedly, including high-quality, medium-quality, and low-quality pictures (high-quality pictures such as normal light, accurate focus, no compression, etc.; medium-quality pictures such as dim light, slightly blurred pictures, light compression, etc.; low-quality pictures such as poor light, overexposure, serious out-of-focus problems, and highly compressed pictures). The second is to expand the data set based on the situation images collected in real time by the unmanned platform using traditional image processing methods, including pixelation, sharpening, Gaussian blur, motion blur, adding noise, adjusting saturation contrast, image compression, Gaussian blur, JPEG compression, adjusting contrast saturation, etc., so as to collect a batch of pictures of various qualities; the third is to use publicly labeled data sets.

[0115] In a specific example, the quality level of the situation picture dataset is determined as follows: the picture quality is divided into x levels, corresponding to different picture qualities. For example, X can be set to 5, that is, the quality evaluation level is [0,5], from 0 to 5, respectively, indicating that the situation picture is unavailable, the quality is very poor, the quality is poor, the quality is average, the quality is good, and the quality is good. The annotator annotates the situation picture quality according to the above levels.

[0116] In the specific example, manual annotation is used to annotate the data set. The situation picture data set collected by the unmanned platform and obtained by simple image processing is distributed to annotators for annotation. The data set distribution has overlaps. Each picture is repeatedly annotated by no less than 5 people, and the annotation results are finally collected.

[0117] In the process of data cleaning, considering that different annotators have different knowledge backgrounds and cognitive levels, they may give different scores to the same picture, so the quality scores of the annotated pictures can be filtered. The image quality scores are filtered by estimating the confidence interval, that is, the image scores within the confidence interval are retained, and those outside the confidence interval are regarded as outliers and the annotation results are removed. Assuming that the distribution of the quality scores of all situation pictures provided to the annotators follows a normal distribution, since the overall variance of the situation pictures is unknown and the sample size is not infinite, the confidence interval is estimated using the t distribution as follows:

[0118]

[0119] in, represents the sample mean; s represents the standard deviation of the sample; n represents the sample size; df represents the degrees of freedom, and its value is equal to the sample size minus 1, that is, df = n-1; t α / 2,df Represents the α / 2 quantile in the t-distribution with df degrees of freedom. After the confidence interval is calculated using the above formula, images with scores outside the interval are discarded.

[0120] In the process of calculating the annotation score, the cleaned data set is used using the confidence interval formula based on the t distribution to calculate the average value of the annotation results as the final annotation score of the image.

[0121] (2) Training model: In the present invention, a reference-free image quality assessment model RankIQA is used. The model uses VGG16 as the base network architecture and incorporates a Siamese network structure. The Siamese network is a special neural network design that contains two or more branches with the same network structure and shared weights. It is mainly used to process paired data, such as comparison or matching tasks. In the RankIQA model, the Siamese network is used to learn the relative quality ranking information in the image sequence, that is, by comparing the quality differences of paired input images, the feature representation that can reflect the quality of the image is learned.

[0122] Furthermore, the present invention uses an effective transfer learning strategy, that is, the image quality ranking features learned from the Siamese network are successfully transferred to the traditional convolutional neural network (CNN). By fine-tuning on the CNN, a deep learning model that can accurately predict image quality is finally constructed. This process not only fully utilizes the advantages of the Siamese network in learning subtle differences between images, but also combines the powerful feature extraction capabilities of CNN, thereby significantly improving the generalization performance and prediction accuracy of the model.

[0123] (3) Model screening: For multiple different models produced in the model training phase, one or several can be selected for use. This method evaluates the model by comparing the difference and correlation between the model prediction value and the labeled value. In the specific implementation, the linear correlation coefficient PLCC can be selected as the model evaluation indicator.

[0124] PLCC measures the similarity of each score vector to the average value. Its calculation formula is as follows:

[0125]

[0126] Where N represents the number of all situation pictures provided to the currently evaluated model, y i , are the manual annotation score and model prediction score of the i-th image, Represents the average of the manual annotation scores and the average of the prediction scores respectively. The vector and The more similar, the closer the PLCC value is to 1, which means that the correlation between the image score predicted by the current evaluated model and the manually annotated score is higher, that is, the better the effect of the evaluated model is, and the more accurately the quality score of the situation image can be given. Finally, the models with the top k PLCC values ​​are selected as the quality evaluation models of the situation images collected by the unmanned platform, where k represents the number of situation image self-evaluation models that need to be selected.

[0127] (4) Situation self-assessment and recollection: Use the quality assessment model obtained in step (3) to input the image to be judged into the model, and the model outputs the self-assessment score of the situation image quality. Comprehensively give all the environmental intelligence collected by the unmanned platform in the same situation environment, find the image with the highest quality self-assessment score, and store it in the database as the most valuable intelligence image in the current position and current environment. Set the threshold for the qualified situation image to 90, that is, in the situation self-assessment stage, the situation image quality score output by the model should be >90 points. If more than 70% of the collected situation images do not meet this threshold, send corresponding instructions to the unmanned platform to make the unmanned platform autonomously recollect the situation image of the current environment. At this time, the unmanned platform should carry out task planning. If the position, posture, etc. of the platform change, it should return to the posture when the image was collected, and then feed the collected situation intelligence image to the model for re-self-assessment.

[0128] (5) Autonomous situation transmission: compress the selected images (such as JPEG compression) to reduce the amount of data transmitted. Use the communication module to transmit the compressed images to the upper-level command system. Before transmission, use the encryption algorithm to encrypt the images. Option 1 is the SM4 symmetric encryption algorithm to ensure absolute data security. Option 2 is the SM9 identity-based encryption algorithm to quickly verify that the collected situation images are from a trusted unmanned platform to prevent information deception.

[0129] The present invention aims to solve the problems of inaccurate image quality assessment, high frequency of repeated shooting and low overall system efficiency in the existing unmanned platform situation data collection process, and achieves the purpose of the invention through the following technical points:

[0130] 1. Image quality assessment model based on deep learning: This paper proposes an image quality assessment method based on deep learning image quality assessment model. This method can build multiple twin assessment models and select the optimal image quality assessment model through the PLCC indicator to quantitatively evaluate the image. Compared with the traditional rule-based method, this method has higher accuracy and robustness and can operate stably under various environmental conditions.

[0131] 2. Real-time image quality feedback mechanism: During the image capture process of the unmanned platform, the image quality is evaluated in real time and fed back to the platform control system at the first time. If it is detected that the image quality does not meet the preset standard, the reshoot instruction is immediately triggered to ensure that each uploaded image meets the quality requirements. This mechanism significantly reduces the duplication of the entire image acquisition process caused by image quality problems, and improves the work efficiency of the unmanned platform through a fast closed loop of the situation picture acquisition sub-process.

[0132] 3. Optimized image selection algorithm: We have designed an efficient image selection algorithm that automatically selects the best quality images for transmission based on the PLCC evaluation results. The algorithm can comprehensively consider the overall clarity and local detail quality of the image to ensure that the images transmitted back to the command system have the highest availability. In addition, the algorithm can dynamically adjust the evaluation criteria according to different application scenarios to adapt to diverse mission requirements.

[0133] 4. Security considerations during the transmission of unmanned platform situation: During the autonomous transmission of situation, a lightweight and secure transmission workflow with the ground command and control center is designed and considered, and the use of image compression and data encryption is considered.

[0134] In general, by comparison with the prior art, the present invention has at least the following advantages: it can more objectively and accurately screen out high-quality images, which is better than the results of manual inspection. Traditional manual visual inspection methods have problems such as subjectivity and low efficiency, while the automated unmanned platform situation image quality assessment system of the present invention can more objectively and accurately screen out high-quality images; existing image assessment methods often perform quality inspections after the situation image is transmitted back, and once quality problems are found, multiple reshoots may be required, while the present invention can provide instant feedback when the unmanned platform is shooting, reducing unnecessary repeated shooting, unmanned platform task re-planning, and repeated work in the same environment. Therefore, the technology of the present invention not only improves image quality, reduces duplication of work, and enhances system performance, but also makes operation easier.

[0135] Features described and / or illustrated above for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or used in place of features in other embodiments.

[0136] It should be emphasized that the term "include / comprises" when used herein refers to the presence of features, integers, steps or components, but does not exclude the presence or addition of one or more other features, integers, steps, components or combinations thereof.

[0137] The above devices and methods of the present invention can be implemented by hardware, or by hardware combined with software. The present invention relates to such a computer-readable program, which, when executed by a logic component, enables the logic component to implement the above-mentioned devices or components, or enables the logic component to implement the above-mentioned various methods or steps. The present invention also relates to a storage medium for storing the above-mentioned program, such as a hard disk, a magnetic disk, an optical disk, a DVD, a flash memory, etc.

[0138] The many features and advantages of these embodiments are apparent from this detailed description, and thus the appended claims are intended to cover all such features and advantages of these embodiments that fall within their true spirit and scope. Furthermore, since numerous modifications and changes will readily occur to those skilled in the art, it is not intended that the embodiments of the invention be limited to the exact construction and operation illustrated and described, but rather all suitable modifications and equivalents falling within the scope thereof are intended to be covered.

[0139] Parts of the present invention that are not described in detail are well known to those skilled in the art.

Claims

1. A self-assessment method for environmental situation awareness of an unmanned platform, characterized in that: The method includes: Obtaining specific scenario situation data and public data, and processing the specific scenario situation data; The processed data and public data are learned using the twin network to obtain a learning model, which is then combined with the convolutional neural network. The combined model is fine-tuned and used as an image quality prediction model. The picture quality prediction model is screened according to the Pearson linear correlation coefficient to obtain the final picture quality prediction model; According to the final image quality prediction model, the multiple situation images to be collected are evaluated to obtain evaluation scores corresponding to each of the multiple situation images, and whether the multiple situation images meet the predetermined requirements is judged according to the evaluation scores. If they meet the requirements, the situation image with the highest evaluation score is stored in the database as the most valuable intelligence image in the current position and the current environment. Otherwise, the situation image is re-collected and the multiple situation images after re-collection are evaluated until the predetermined requirements are met; The situation picture with the highest evaluation score is processed before transmission, and the processed situation picture with the highest evaluation score is output to the ground command center.

2. The method according to claim 1, characterized in that Processing of situation data for specific scenarios includes: The situation data of each specific scene is divided into multiple quality levels by multiple personnel, and the divided data is labeled with multiple quality scores by multiple personnel; Perform data cleaning on the labeled data to remove abnormal quality scores; The remaining quality scores of each specific scene situation data are averaged, and the average value is used as the annotation score of the corresponding specific scene situation data.

3. The method according to claim 2, characterized in that The Pearson linear correlation coefficient was calculated by the following formula: Among them, PLCC represents the Pearson linear correlation coefficient, N represents the number of all situation pictures provided, and y i , They represent the annotation score and model prediction score of the i-th situation picture respectively, They represent the mean of the annotation scores and the mean of the prediction scores, respectively.

4. The method according to claim 3, characterized in that Judging whether multiple situation pictures meet predetermined requirements based on evaluation scores includes: When the evaluation scores of situation pictures greater than a predetermined proportion in the multiple situation pictures are less than a threshold, it is determined that the multiple situation pictures do not meet the predetermined requirement; When the evaluation scores of situation pictures greater than a predetermined proportion in the multiple situation pictures are greater than or equal to a threshold, it is determined that the multiple situation pictures meet the predetermined requirement.

5. The method according to claim 4, characterized in that The pre-transmission processing of the situation picture with the highest evaluation score includes: The situation pictures with the highest evaluation scores are compressed and encrypted in turn.

6. An unmanned platform environmental situation awareness self-assessment system, characterized in that: The system includes: The data set collection module is used to obtain specific scenario situation data and public data, and process the specific scenario situation data; The image quality assessment model training module is used to use the twin network to learn the processed data and public data to obtain a learning model, combine the learning model with the convolutional neural network, and fine-tune the combined model as an image quality prediction model; A model screening module is used to screen the picture quality prediction model according to the Pearson linear correlation coefficient to obtain the final picture quality prediction model; An image evaluation module is used to evaluate the multiple situation pictures to be collected according to the final picture quality prediction model, obtain the evaluation scores corresponding to the multiple situation pictures, and judge whether the multiple situation pictures meet the predetermined requirements according to the evaluation scores. If they meet the requirements, the situation picture with the highest evaluation score is stored in the database as the most valuable intelligence picture in the current position and the current environment. Otherwise, the situation pictures are collected again and the multiple situation pictures collected again are evaluated until they meet the predetermined requirements. The situation image autonomous transmission module is used to process the situation image with the highest evaluation score before transmission, and output the processed situation image with the highest evaluation score to the ground command center.

7. The system according to claim 6, characterized in that Processing of situation data for specific scenarios includes: The situation data of each specific scene is divided into multiple quality levels by multiple personnel, and the divided data is labeled with multiple quality scores by multiple personnel; Perform data cleaning on the labeled data to remove abnormal quality scores; The remaining quality scores of each specific scene situation data are averaged, and the average value is used as the annotation score of the corresponding specific scene situation data.

8. The system according to claim 7, characterized in that The Pearson linear correlation coefficient was calculated by the following formula: Among them, PLCC represents the Pearson linear correlation coefficient, N represents the number of all situation pictures provided, and y i , They represent the annotation score and model prediction score of the i-th situation picture respectively, They represent the mean of the annotation scores and the mean of the prediction scores, respectively.

9. The system according to claim 8, characterized in that Judging whether multiple situation pictures meet predetermined requirements based on evaluation scores includes: When the evaluation scores of situation pictures greater than a predetermined proportion in the multiple situation pictures are less than a threshold, it is determined that the multiple situation pictures do not meet the predetermined requirement; When the evaluation scores of situation pictures greater than a predetermined proportion in the multiple situation pictures are greater than or equal to a threshold, it is determined that the multiple situation pictures meet the predetermined requirement.

10. The system according to claim 9, characterized in that The pre-transmission processing of the situation picture with the highest evaluation score includes: The situation pictures with the highest evaluation scores are compressed and encrypted in turn.

Images