A data-driven optical imaging satellite discovery efficiency evaluation method and device

Abstract

The present invention relates to the intersection of optical image processing and artificial intelligence, and provides a data-driven optical imaging satellite discovery efficiency evaluation method and device, the method comprising: preprocessing imaging data of various imaging sensors to obtain target background image data; extracting feature differences from the target background image data; generating model training data based on feature differences in the target background image data and target detection algorithm label annotation results, generating discovery efficiency measurement models for various imaging sensors for targets of different sizes through BP neural network training; accessing real-time image data of various imaging sensors, and dynamically evaluating the discovery efficiency of imaging sensors for targets of different sizes based on the discovery efficiency measurement model. The present invention can accurately and quickly measure the discovery efficiency of various imaging sensors.

Description

Technical Field

[0001] The present invention relates to the intersection of optical image processing and artificial intelligence, and in particular to a data-driven optical imaging satellite discovery effectiveness evaluation method and device. Background Art

[0002] With the development of optical imaging sensor imaging technology, the image resolution and target visualization features have been greatly enhanced, making satellite remote sensing images more realistically reflect the real features of earth objects. Therefore, optical imaging satellites play an increasingly important role and are widely used in disaster warning, regional monitoring, regional mapping and other fields. When evaluating the detection performance of optical imaging satellites, the target detection probability index is usually used for evaluation. The traditional detection probability measurement method performs target detection on multiple samples and quantifies the target detection probability by statistically analyzing the target detection results. This method requires a large number of samples and has low calculation timeliness. If the number of samples is small, the target detection probability calculation will be inaccurate.

[0003] In order to solve the shortcomings of Monte Carlo calculation of target discovery probability, considering the influencing factors such as platform, sensor and environment, a quantitative analytical formula of target discovery probability is constructed through theoretical modeling, which greatly improves the timeliness of calculation. However, there is a large performance gap between the theoretical model and the actual application, and it is impossible to accurately evaluate and quantify the discovery capability of imaging satellites. Therefore, how to accurately and quickly measure the discovery efficiency of imaging satellites is a difficult problem that needs to be studied and overcome urgently. Summary of the invention

[0004] The present invention aims to provide a data-driven optical imaging satellite discovery efficiency evaluation method and device, so as to accurately and quickly measure the discovery efficiency of various imaging sensors.

[0005] The present invention provides a data-driven optical imaging satellite discovery efficiency evaluation method, comprising the following steps:

[0006] Step 1: Preprocess the imaging data of various imaging sensors to obtain target background image data;

[0007] Step 2: Extract feature differences from target background image data;

[0008] Step 3: Generate model training data based on the feature difference in the target background image data and the target detection algorithm label annotation results, and generate detection efficiency measurement models of various imaging sensors for targets of different sizes through BP neural network training;

[0009] Step 4: Access the real-time image data of various imaging sensors and dynamically evaluate the detection efficiency of imaging sensors for targets of different sizes based on the detection efficiency measurement model.

[0010] In one or more embodiments, the preprocessing described in step 1 includes: for imaging data of various types of imaging sensors, a surrounding background area of ​​9 times the target area is selected with the target as the center to divide the target background area, and the formed target background image data is classified and sorted according to the target size and time sequence.

[0011] In one or more embodiments, in step 2, for the target background image data obtained from the imaging data of the visible light imaging sensor, the visible light imaging feature difference is extracted; the extracted visible light imaging feature difference includes color feature difference, brightness feature difference and texture feature difference.

[0012] In only one step, in step 2, for the target background image data obtained by the imaging data of the visible light imaging sensor, the method for extracting the difference of the visible light imaging feature includes:

[0013] Color feature difference: Extract the color value of each pixel based on the target background image data obtained by the visible light imaging sensor imaging data; then quantify the color feature difference between the target area and the background area based on the extracted color value through the HIS space color histogram similarity calculation method based on the normalized color interval;

[0014] Brightness feature difference: Based on the target background image data obtained from the imaging data of the visible light imaging sensor, the brightness feature difference between the target area and the background area is quantified based on the grayscale histogram of the area;

[0015] Texture feature difference: In the target background image data obtained from the imaging data of the visible light imaging sensor, the texture feature difference between the target area and the background area is quantified based on the regional co-occurrence matrix.

[0016] In one or more embodiments, in step 2, for the target background image data obtained from the imaging data of the hyperspectral imaging sensor, the hyperspectral imaging feature difference is extracted; the extracted hyperspectral imaging feature difference includes spectral feature difference and spatial structure feature difference.

[0017] Furthermore, in step 2, for the target background image data obtained by the imaging data of the hyperspectral imaging sensor, the method of extracting the hyperspectral imaging feature difference includes:

[0018] Spectral feature difference: Based on the target background image data obtained by the hyperspectral imaging sensor imaging data, the spectral feature difference between the target area and the background area is quantified by calculating the spectral angle distance of the spectral curves of the target area and the background area;

[0019] Spatial structural feature difference: Based on the target background image data obtained from the imaging data of the hyperspectral imaging sensor, the spectral feature difference between the target area and the background area is quantified using the SSIM structural similarity algorithm.

[0020] In one or more embodiments, in step 2, infrared imaging feature differences are extracted for target background image data obtained from imaging data of the infrared imaging sensor; the extracted infrared imaging feature differences include temperature feature differences and texture feature differences.

[0021] Furthermore, in step 2, for the target background image data obtained by the imaging data of the infrared imaging sensor, the method of extracting the difference of the infrared imaging feature includes:

[0022] Temperature characteristic difference: A temperature characteristic difference calculation model is constructed based on the minimum resolution temperature difference, and the temperature difference is calculated through the model;

[0023] Texture feature difference: Based on the target background image data obtained from the infrared imaging sensor imaging data, the texture feature difference between the target area and the background area is quantified based on the regional co-occurrence matrix.

[0024] Further, step 3 includes:

[0025] The value ranges of the feature difference of target background image data of different target levels of different imaging sensors are equally divided into several intervals, and different feature difference interval combination categories for different imaging sensors are generated by permutation and combination. Then, the target background image data of different target levels of different imaging sensors are classified, and target detection is performed using typical target detection algorithms of different types of imaging sensors, and then the target detection probability under different feature difference interval combination categories for different imaging sensors is statistically calculated. Finally, the target detection probability corresponding to the data under each combination category is annotated as a label to form model training data.

[0026] According to the variable step sliding time window method, with the newly added data time period as the variable step, infrared samples of corresponding target size levels are selected in reverse order from the latest time data as training model data;

[0027] The model training data is divided into a training set and a test set, a BP neural network is constructed, and network parameters including target error, learning rate and maximum number of iterations are set. Finally, the training set and the test set are input into the constructed BP neural network training. After the training is completed, a discovery effectiveness measurement model is generated.

[0028] The present invention also provides a data-driven optical imaging satellite discovery effectiveness evaluation device, comprising:

[0029] An imaging sensor imaging data preprocessing module is used to preprocess the imaging data of various imaging sensors to obtain target background image data;

[0030] An imaging feature difference measurement module is used to extract feature differences from target background image data;

[0031] The discovery effectiveness measurement model training module is used to generate model training data based on the feature difference in the target background image data and the target detection algorithm label annotation results, and to generate the discovery effectiveness measurement model of various imaging sensors for targets of different sizes through BP neural network training;

[0032] The discovery probability calculation module is used to access the real-time image data of various imaging sensors and dynamically evaluate the discovery efficiency of imaging sensors for targets of different sizes based on the discovery efficiency measurement model.

[0033] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0034] 1. Based on the principle of optical imaging and target discovery, the present invention takes into account the influencing factors such as sensors, platforms and environment, analyzes and constructs a feature difference measurement model in multiple dimensions such as color, brightness, texture, spectrum, spatial structure, temperature, etc., making the quantification of discovery efficiency more scientific.

[0035] 2. The present invention uses the difference of sensor imaging features as the input of the discovery efficiency measurement model, and uses the BP neural network for offline training based on actual image data, so that the generated quantitative model is faster and closer to reality when calculating the discovery probability than the traditional method of Monte Carlo experiment. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 The figure is a flow chart of a data-driven optical imaging satellite discovery effectiveness evaluation method in an embodiment of the present invention.

[0037] Figure 2 It is a schematic diagram of demarcating the target background area in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0039] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] Example

[0041] like Figure 1 As shown, this embodiment proposes a data-driven optical imaging satellite discovery effectiveness evaluation method, comprising the following steps:

[0042] Step 1: The imaging sensor imaging data preprocessing module preprocesses the imaging data of various imaging sensors to obtain target background image data. In this embodiment, for the imaging data of various imaging sensors, historical image data including visible light imaging sensor image data, hyperspectral imaging sensor image data and infrared imaging sensor image data are usually used, such as Figure 2 As shown, a surrounding background area of ​​9 times the target area is selected with the target as the center to divide the target background area, and the formed target background image data is classified and sorted according to the target size and time sequence.

[0043] Step 2: The imaging feature difference measurement module extracts feature differences from the target background image data, and the extraction dimensions include color, brightness, texture, spectrum, spatial structure, temperature, etc. The specific method flow is as follows:

[0044] (1) Visible light imaging feature difference

[0045] (1.1) Color feature difference

[0046] According to the target background image data obtained by the visible light imaging sensor imaging data, the color value of each pixel is extracted; then the extracted color value is quantified based on the normalized color interval through the HIS (Hue, Saturation, Intensity) space color histogram similarity calculation method to quantify the color feature difference between the target area and the background area. The specific formula is:

[0047]

[0048] in, is the average color difference of the target background image data obtained from the imaging data of the visible light imaging sensor, is the background area number, For the goal and The color difference of the background area is calculated as follows:

[0049]

[0050] In the above formula, For the target area, is the first target background image data obtained based on the imaging data of the visible light imaging sensor. background area, is the HIS space color histogram similarity calculation function, is the normalized number of color interval pixels, is the number of color intervals into which the HIS space is divided, Is the color interval number.

[0051] (1.2) Brightness feature difference

[0052] According to the target background image data obtained from the imaging data of the visible light imaging sensor, the brightness feature difference between the target area and the background area is quantified based on the grayscale histogram of the area. The specific formula is:

[0053]

[0054] in, is the average brightness difference of the target background image data obtained by the visible light imaging sensor imaging data, For the goal and The brightness difference of the background area is as follows:

[0055]

[0056] In the above formula, Target area and Background area The grayscale histogram curve overlaps the horizontal and vertical coordinates. Target area and Background area The grayscale histogram curve is the area of ​​the union of the horizontal and vertical coordinates.

[0057] (1.3) Texture feature difference

[0058] In the target background image data obtained from the imaging data of the visible light imaging sensor, the texture feature difference between the target area and the background area is quantified based on the regional co-occurrence matrix. The specific formula is:

[0059]

[0060] in, is the average texture difference of the target background image data, For the goal and The texture difference of the background area is as follows:

[0061]

[0062] In the above formula, is the target region co-occurrence matrix Line The elements of the column, For the Background region co-occurrence matrix Line Elements of a column.

[0063] (2) Hyperspectral imaging feature differences

[0064] (2.1) Spectral feature difference

[0065] Based on the target background image data obtained from the imaging data of the hyperspectral imaging sensor, the spectral characteristic difference between the target area and the background area is quantified by calculating the spectral angle distance of the spectral curves of the target area and the background area. The specific formula is:

[0066]

[0067] in, is the average spectral difference of the target background image data obtained by the hyperspectral imaging sensor imaging data, For the goal and The spectral difference of the background area is as follows:

[0068]

[0069] In the above formula, is the transpose of a vector, is the vectorized representation of the average spectral curve of the target area, is the vectorized representation of the average spectral curve of the background area, specifically:

[0070]

[0071]

[0072] In the above formula, and are the number of rows and columns of pixels in the region, are the wavelengths of each spectral band, is the number of spectral segments, is the spectral value of each pixel in the target area, is the spectral value of each pixel in the background area, is the transpose of a vector.

[0073] (2.2) Difference of spatial structural characteristics

[0074] Based on the target background image data obtained from the imaging data of the hyperspectral imaging sensor, the SSIM (Structural Similarity) algorithm is used to quantify the difference in spectral characteristics between the target area and the background area. The specific formula is:

[0075]

[0076] in, is the spatial structural difference of the target background image data obtained by the hyperspectral imaging sensor imaging data, For the goal and The spatial structure difference of the background area is as follows:

[0077]

[0078] In the above formula, Target area With background area In the spectrum The number of pixels in the structural similarity interval, The sequence number of the spectrum segment.

[0079] (3) Difference of infrared imaging features

[0080] (3.1) Temperature characteristic difference

[0081] The temperature characteristic difference calculation model is constructed based on the minimum resolution temperature difference, specifically:

[0082]

[0083] in, is the temperature difference between the target and background image data obtained by the infrared imaging sensor imaging data, is the spatial angular frequency of the target in the image, is the horizontal field of view of the system, is the vertical field of view of the system, It is the time accumulation effect of the human eye. is the accumulation time of the human eye, generally about 0.2s~0.3s, F is the frame rate, is the infrared imager dwell time, is the equivalent noise bandwidth of the infrared sensor system, is the noise equivalent temperature difference of the system.

[0084] (3.2) Texture feature difference

[0085] Consistent with the calculation of texture feature difference in visible light imaging, the texture feature difference between the target area and the background area is quantified based on the co-occurrence matrix of the region according to the target background image data obtained from the imaging data of the infrared imaging sensor.

[0086] Step 3: The discovery effectiveness measurement model training module generates model training data based on the feature difference in the target background image data and the target detection algorithm label annotation results, and generates the discovery effectiveness measurement model of various imaging sensors for targets of different sizes through BP (Back Propagation) neural network training. The specific method flow is as follows:

[0087] (1) First, the imaging feature difference measurement module is called to calculate the feature difference of target background image data of different imaging sensors and different target levels output by the imaging sensor imaging data preprocessing module, and the feature difference value ranges of visible light, hyperspectral, and infrared imaging sensors are divided into , , intervals, and generate different feature difference interval combination categories for different imaging sensors through permutations and combinations; then classify the target background image data of different target levels of different imaging sensors, and use typical target detection algorithms of different types of imaging sensors to perform target detection, and then statistically calculate the target detection probability under different feature difference interval combination categories for different imaging sensors; finally, annotate the data under each combination category with the corresponding target detection probability as a label to form model training data.

[0088] (2) In order to ensure the efficiency of training, a variable step-size sliding time window is used, with the time period of newly added data as the variable step-size, and the training data is selected in reverse order from the latest data. indivual Visible light samples of target size classes, indivual Hyperspectral samples of target size classes and indivual Infrared samples of target size classes are used as training model data.

[0089] (3) Divide the model training data into a training set and a test set according to a certain ratio. For example, use 80% of the samples in the model training data as the training set and 20% of the samples in the model training data as the test set. Construct a BP neural network with two hidden layers and no more than 30 nodes. Set network parameters such as target error, learning rate, and maximum number of iterations. Finally, generate a discovery performance measurement model through training.

[0090] Step 4: The discovery probability calculation module accesses the real-time image data of various imaging sensors and dynamically evaluates the discovery efficiency of visible light, hyperspectral and infrared imaging sensors for targets of different sizes based on the discovery efficiency measurement model. The real-time image data also includes visible light imaging sensor image data, hyperspectral imaging sensor image data and infrared imaging sensor image data.

[0091] Correspondingly, this embodiment also provides a data-driven optical imaging satellite discovery effectiveness evaluation device, comprising:

[0092] An imaging sensor imaging data preprocessing module is used to preprocess the imaging data of various imaging sensors to obtain target background image data;

[0093] An imaging feature difference measurement module is used to extract feature differences from target background image data;

[0094] The discovery effectiveness measurement model training module is used to generate model training data based on the feature difference in the target background image data and the target detection algorithm label annotation results, and to generate the discovery effectiveness measurement model of various imaging sensors for targets of different sizes through BP neural network training;

[0095] The discovery probability calculation module is used to access the real-time image data of various imaging sensors and dynamically evaluate the discovery efficiency of imaging sensors for targets of different sizes based on the discovery efficiency measurement model.

[0096] The working methods of the above modules can refer to the description of the aforementioned data-driven optical imaging satellite discovery effectiveness evaluation method, which will not be repeated here.

[0097] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data-driven optical imaging satellite discovery effectiveness evaluation method, characterized in that: The steps include: Step 1: Preprocess the imaging data of various imaging sensors to obtain target background image data; Step 2: Extract feature differences from target background image data; Step 3: Generate model training data based on the feature difference in the target background image data and the target detection algorithm label annotation results, and generate detection efficiency measurement models of various imaging sensors for targets of different sizes through BP neural network training; Step 4: Access the real-time image data of various imaging sensors and dynamically evaluate the detection efficiency of imaging sensors for targets of different sizes based on the detection efficiency measurement model; In step 2, for the target background image data obtained by the imaging data of the visible light imaging sensor, the visible light imaging feature difference is extracted; the extracted visible light imaging feature difference includes color feature difference, brightness feature difference and texture feature difference; In step 2, for the target background image data obtained by the imaging data of the visible light imaging sensor, the method of extracting the difference of the visible light imaging feature includes: Color feature difference: Extract the color value of each pixel based on the target background image data obtained by the visible light imaging sensor imaging data; then quantify the color feature difference between the target area and the background area based on the extracted color value through the HIS space color histogram similarity calculation method based on the normalized color interval; Brightness feature difference: Based on the target background image data obtained from the imaging data of the visible light imaging sensor, the brightness feature difference between the target area and the background area is quantified based on the grayscale histogram of the area; Texture feature difference: In the target background image data obtained from the visible light imaging sensor imaging data, the texture feature difference between the target area and the background area is quantified based on the regional co-occurrence matrix; In step 2, for the target background image data obtained by the imaging data of the hyperspectral imaging sensor, the hyperspectral imaging feature difference is extracted; the extracted hyperspectral imaging feature difference includes the spectral feature difference and the spatial structure feature difference; In step 2, for the target background image data obtained by the imaging data of the hyperspectral imaging sensor, the method for extracting the hyperspectral imaging feature difference includes: Spectral feature difference: Based on the target background image data obtained by the hyperspectral imaging sensor imaging data, the spectral feature difference between the target area and the background area is quantified by calculating the spectral angle distance of the spectral curves of the target area and the background area; Spatial structural feature difference: Based on the target background image data obtained from the imaging data of the hyperspectral imaging sensor, the spectral feature difference between the target area and the background area is quantified using the SSIM structural similarity algorithm; In step 2, for the target background image data obtained by the imaging data of the infrared imaging sensor, the infrared imaging feature difference is extracted; the extracted infrared imaging feature difference, temperature feature difference and texture feature difference; In step 2, for the target background image data obtained by the imaging data of the infrared imaging sensor, the method of extracting the difference of the infrared imaging feature includes: Temperature characteristic difference: A temperature characteristic difference calculation model is constructed based on the minimum resolution temperature difference, and the temperature difference is calculated through the model; Texture feature difference: Based on the target background image data obtained from the infrared imaging sensor imaging data, the texture feature difference between the target area and the background area is quantified based on the regional co-occurrence matrix.

2. The data-driven optical imaging satellite discovery effectiveness evaluation method according to claim 1, characterized in that: The preprocessing described in step 1 includes: for imaging data of various imaging sensors, a surrounding background area of ​​9 times the target area is selected with the target as the center to divide the target background area, and the formed target background image data is classified and sorted according to the target size and time sequence.

3. The data-driven optical imaging satellite discovery effectiveness evaluation method according to claim 1, characterized in that: Step 3 includes: The value ranges of the feature difference of target background image data of different target levels of different imaging sensors are equally divided into several intervals, and different feature difference interval combination categories for different imaging sensors are generated by permutation and combination. Then, the target background image data of different target levels of different imaging sensors are classified, and target detection is performed using typical target detection algorithms of different types of imaging sensors, and then the target detection probability under different feature difference interval combination categories for different imaging sensors is statistically calculated. Finally, the target detection probability corresponding to the data under each combination category is annotated as a label to form model training data. According to the variable step sliding time window method, with the newly added data time period as the variable step, infrared samples of corresponding target size levels are selected in reverse order from the latest time data as training model data; The model training data is divided into a training set and a test set, a BP neural network is constructed, and network parameters including target error, learning rate and maximum number of iterations are set. Finally, the training set and the test set are input into the constructed BP neural network training. After the training is completed, a discovery effectiveness measurement model is generated.

4. A data-driven optical imaging satellite discovery efficiency evaluation device, used to implement the data-driven optical imaging satellite discovery efficiency evaluation method according to any one of claims 1 to 3, characterized in that: The device comprises: An imaging sensor imaging data preprocessing module is used to preprocess the imaging data of various imaging sensors to obtain target background image data; An imaging feature difference measurement module is used to extract feature differences from target background image data; The discovery effectiveness measurement model training module is used to generate model training data based on the feature difference in the target background image data and the target detection algorithm label annotation results, and to generate the discovery effectiveness measurement model of various imaging sensors for targets of different sizes through BP neural network training; The discovery probability calculation module is used to access the real-time image data of various imaging sensors and dynamically evaluate the discovery efficiency of imaging sensors for targets of different sizes based on the discovery efficiency measurement model.

Images