A multispectral sensing and target recognition method and device based on multi-element information fusion

Abstract

The application provides a multispectral sensing and target recognition method based on multi-element information fusion, which realizes multispectral measurement of a target, fuses information of different spectrums together through multi-element information fusion, obtains multispectral information of the target, and gives the spectral information a confidence degree, so as to realize multi-element information fusion, improve the accuracy and robustness of target recognition, and solve the problem of information fusion.

Description

Technical Field

[0001] This application relates to the field of optical detection and information fusion technology, specifically to a multispectral sensing and target recognition method and device based on multi-source information fusion. Background Technology

[0002] Traditional intensity detection techniques can only detect the radiation intensity of light. Therefore, when the target and background colors are similar or the ambient light is weak, traditional intensity detection techniques cannot meet the actual detection needs. Compared with traditional intensity detection techniques, multispectral polarization imaging detection technology has the advantage of obtaining object polarization and spectral information that traditional detection techniques cannot obtain. Moreover, different objects have significantly different polarization and spectral information, so multispectral polarization imaging detection technology is helpful in identifying targets hidden in the background. Summary of the Invention

[0003] To address the challenges of deep space exploration and weak target identification in projects, this application proposes a multispectral sensing and target identification method and device based on multi-source information fusion. This method effectively enhances the deep space exploration and feature recognition capabilities of targets, increases the time for aircraft to detect intercepted targets, and significantly improves the anti-interception capabilities of aircraft.

[0004] The technical solution adopted in this application is as follows:

[0005] A multispectral sensing and target recognition method based on multi-source information fusion, comprising the following steps:

[0006] Step 1: Use detectors with different spectral bands to acquire target spectral information in four spectral bands: ultraviolet, blue, red, infrared, and IR.

[0007] Step 2: Assign confidence levels to each target spectral information based on the signal-to-noise ratio of the target spectral information;

[0008] Step 3: By using the confidence levels of images in different spectral bands, pixel-level fusion of the spectral information of each target is performed to obtain a comprehensive grayscale image;

[0009] Step 4: Obtain the feature pixels of the composite grayscale image based on gradient analysis and identify the target point;

[0010] Step 5: Calculate the pixel coordinates and geometric coordinates of the target point to obtain the target point information;

[0011] Step 6: Restore the target point information to the spectral image, and obtain the overall confidence score based on the confidence curve.

[0012] Furthermore, in step 3, pixel-level fusion of the spectral information of each target is performed, including:

[0013] Four spectral bands—ultraviolet F1, blue F2, red F3, and infrared F4—were selected for ambiguity evidence fusion, and the calculation formula is as follows:

[0014]

[0015] Where F1 represents the spectral information of a pixel in the ultraviolet spectrum of the target point, and m1(F1) represents the confidence assessment value of whether it is a real target in the ultraviolet spectrum; F2 represents the spectral information of a pixel in the blue spectrum of the target point, and m2(F2) represents the confidence assessment value of whether it is a real target in the blue spectrum; F3 represents the spectral information of a pixel in the red spectrum of the target point, and m3(F3) represents the confidence assessment value of whether it is a real target in the red spectrum; F4 represents the spectral information of a pixel in the infrared spectrum of the target point, and m4(F4) represents the confidence assessment value of whether it is a real target in the infrared spectrum; F is the multispectral synthesis result of ultraviolet F1, blue F2, red F3, and infrared F4, which satisfies the intersection result in set theory, that is, F=F1∩F2∩F3∩F4, where ∩ is the intersection operation, thereby obtaining the comprehensive confidence M(F) of the target pixel-level fusion.

[0016] Furthermore, in step 6, based on the confidence curve, the overall confidence level of the result is obtained, including:

[0017] After restoring the target point information to the spectral image, a confidence curve is obtained based on the characteristics of the target point information, and the overall confidence score is calculated using the following formula:

[0018]

[0019] Where R is the signal-to-noise ratio of the target point in the ultraviolet F1, blue F2, red F3, and infrared F4 frequency bands, and c1, c2, τ1, τ2, μ, and σ are characteristic coefficients.

[0020] Furthermore, in step 1, for the target information, different spectral band sensors are used to acquire the spectral information corresponding to the target information.

[0021] Furthermore, in step 2, the image signal-to-noise ratio of each spectral information is calculated, and the confidence level of the images in different spectral bands is determined based on the signal-to-noise ratio at the same time, and then assigned to each image.

[0022] The formula for calculating the image signal-to-noise ratio (SNR) is as follows:

[0023] SNR = (mean gray level of the target imaging region - mean gray level of the background imaging region) / mean square error of gray level of the background region;

[0024] The signal-to-noise ratio (SNR) R of the target point in the ultraviolet, blue, red, and infrared light bands at the same time is calculated for the band image at the same time, thereby determining the confidence level of the images in different spectral bands.

[0025] Furthermore, the different spectral bands include ultraviolet light, blue light, red light, and infrared light, and the fusion method is to select four spectral bands—ultraviolet light, blue light, red light, and infrared light—for fusion.

[0026] Furthermore, in step 4, the gradient value of the composite grayscale image is calculated, and the larger value is taken as the candidate target. Since the candidate target contains only a small number of real targets, the real targets in the image are obtained by combining the motion features of the image before and after the time step.

[0027] The true target is a candidate target that moves continuously in the image without any abrupt changes.

[0028] Furthermore, in step 5, the pixel coordinates of the target point are obtained, and the pixel coordinates are mapped to spatial geometric coordinates through a mapping relationship to obtain target information.

[0029] Furthermore, the pixel coordinates are mapped to spatial geometric coordinates through mapping relationships, including:

[0030] When the pixels obtained in the image are [uv], the resulting spatial coordinates [xy] satisfy the following formula:

[0031]

[0032] Where a1, a2, a3, and a4 are camera parameters.

[0033] A multispectral sensing and target recognition device based on multi-source information fusion, the device comprising:

[0034] At least one processor; and

[0035] A memory communicatively connected to the at least one processor; wherein,

[0036] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the above-described method.

[0037] The following technical effects can be achieved through the embodiments of this application:

[0038] (1) Perform multispectral information fusion on the target to obtain target image features under different spectral conditions and realize multispectral sensing measurement;

[0039] (2) Perform multispectral pixel-level fusion on the information target to obtain the comprehensive grayscale image of the target, confirm the gradient characteristics of the target, and realize the recognition of targets with little information;

[0040] (3) Confidence analysis is performed based on spectral information to provide a confidence evaluation of the target and realize real-time online assessment of the credibility of the results, which serves as the basis for eliminating false targets. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0043] Figure 2 This is a schematic diagram showing the division of the target pixel into a 3×3 region. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0045] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0046] The method includes the following steps:

[0047] Step 1: For the target information, use sensors with different spectral bands to acquire the spectral information corresponding to the target information; the different spectral bands include ultraviolet light, blue light, red light, and infrared light;

[0048] Step 2: Calculate the image signal-to-noise ratio (SNR) for each spectral information, and determine the confidence level of images in different spectral bands based on the SNR R at the same time, assigning the value to each image. The specific calculation method for the image SNR R for each spectral information is as follows:

[0049] Signal-to-noise ratio R = (mean gray level of the target imaging region - mean gray level of the background imaging region) / mean square error of gray level of the background region

[0050] By analyzing the target points in the ultraviolet, blue, red, and infrared light frequency bands, the image signal-to-noise ratio (SNR) R corresponding to the same moment in each frequency band can be calculated. Then, based on formula (2), the overall confidence level of the images in different spectral bands is determined using the SNR R at the same moment. The formula for calculating the overall confidence level is as follows:

[0051]

[0052] Where R is the signal-to-noise ratio of the target point in the ultraviolet F1, blue F2, red F3, and infrared F4 frequency bands, and c1, c2, τ1, τ2, μ, and σ are characteristic coefficients;

[0053] Step 3: Perform pixel-level fusion based on the confidence of images from different spectral bands to obtain a composite grayscale image at the same time. The pixel-level fusion method is to select four spectral bands—ultraviolet, blue, red, and infrared—at the same time for fusion, and obtain the composite grayscale image of pixel-level fusion through the following calculation formula.

[0054]

[0055] Where F1 represents the spectral information of a pixel in the ultraviolet spectrum of the target point, and m1(F1) represents the confidence assessment value of whether it is a real target in the ultraviolet spectrum; F2 represents the spectral information of a pixel in the blue spectrum of the target point, and m2(F2) represents the confidence assessment value of whether it is a real target in the blue spectrum; F3 represents the spectral information of a pixel in the red spectrum of the target point, and m3(F3) represents the confidence assessment value of whether it is a real target in the red spectrum; F4 represents the spectral information of a pixel in the infrared spectrum of the target point, and m4(F4) represents the confidence assessment value of whether it is a real target in the infrared spectrum; F is the multispectral synthesis result of ultraviolet light F1, blue light F2, red light F3, and infrared light F4, which satisfies the intersection result in set theory, that is, F=F1∩F2∩F3∩F4, where ∩ is the intersection operation, thereby obtaining the comprehensive confidence M(F) of the target pixel-level fusion;

[0056] Step 4: Calculate the gradient value of the composite grayscale image, take the larger value as the candidate target, and combine it with the motion characteristics of the image before and after time, eliminate false targets with discontinuous motion states, and obtain the real target in the image.

[0057] In this step, the gradient value of each pixel within the scale space of the comprehensive grayscale image is calculated, and the larger value is taken as the candidate target. Since the candidate target contains only a small number of real targets, the motion characteristics of the image at different times are combined to obtain the real target in the image. The real target is the candidate target that moves continuously in the image without producing abrupt changes.

[0058] Figure 2This diagram illustrates the division of the target pixel into a 3×3 region. The calculation of the gradient value of the composite grayscale image is explained using this diagram, including:

[0059] Select a scale space, divide the target pixel into a 3×3 pixel grid, calculate the gray level of each grid, and denot it as U={u0,u1,u2,u3,u4,u5,u6,u7,u8};

[0060] Subtract the eight other elements u0, u1, u2, u3, u5, u6, u7, u8 from the element u4 in the middle position, and take the absolute value. This is denoted as Z = {|u4-u0|, |u4-u1|, |u4-u2|, |u4-u3|, |u4-u5|, |u4-u6|, |u4-u7|, |u4-u8|}.

[0061] The value of the largest element in vector Z is selected as the gradient value of this target pixel, that is:

[0062] T=max{|u4-u0|, |u4-u1|, |u4-u2|, |u4-u3|, |u4-u5|, |u4-u6|, |u4-u7|, |u4-u8|}.

[0063] Step 5: Obtain the pixel coordinates of the target point, and map the pixel coordinates to spatial geometric coordinates through the mapping relationship to obtain the target geometric coordinate information. Let the target pixel obtained in the image be [uv], then the spatial set coordinates [xy] are calculated by the following formula;

[0064]

[0065] Where a1, a2, a3, and a4 are camera parameters.

[0066] Step 6: Restore the target point information to the spectral image, and obtain the overall confidence score based on the confidence curve;

[0067] In this step, the mean R of the signal-to-noise ratios (SNR) for ultraviolet targets (R1), blue targets (R2), red targets (R3), and infrared targets (R4) is calculated. avr And obtain the overall confidence level of the results.

[0068] The specific process is as follows: After restoring the target point information to the spectral image, the confidence curve is obtained based on the characteristics of the target point information, and the overall confidence score is calculated. The calculation formula is as follows:

[0069]

[0070] Where R is the signal-to-noise ratio of the target point in the ultraviolet F1, blue F2, red F3, and infrared F4 frequency bands, and c1, c2, τ1, τ2, μ, and σ are characteristic coefficients.

[0071] The functions described above in this application can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload programmable logic devices (CPLDs), and so on.

[0072] Furthermore, although the operations are described in a specific order, this should be understood as requiring that such operations be performed in the specific order shown or in sequential order, or requiring that all illustrated operations be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.

[0073] Although the subject matter has been described using language specific to structural features and / or device logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A multispectral sensing and target recognition method based on multi-information fusion, characterized in that, The method comprises the following steps: Step 1, acquiring target spectral information of four spectral bands of ultraviolet light, blue light, red light and infrared light by using different spectral band detectors; Step 2, assigning a confidence level to each target spectral information according to the signal-to-noise ratio of the target spectral information; Step 3, performing pixel-level fusion of each target spectral information according to the confidence levels of different spectral band images to obtain a comprehensive gray-scale image; Step 4, obtaining feature pixels of the comprehensive gray-scale image based on gradient analysis to identify target points; Step 5, calculating pixel coordinates and geometric coordinates of the target points to obtain target point information; Step 6, restoring the target point information to a spectral image, and obtaining a result comprehensive confidence level according to a confidence level curve.

2. The method of claim 1, wherein, In step 3, the pixel-level fusion of each target spectral information comprises: Four spectral bands of ultraviolet light F1, blue light F2, red light F3 and infrared light F4 are selected for fuzzy evidence fusion, and the calculation formula is as follows: Wherein F1 represents spectral information of ultraviolet light of a certain pixel of the target point, m1(F1) represents a confidence level evaluation value of the ultraviolet spectral band whether it is a real target; F2 represents spectral information of blue light of a certain pixel of the target point, m2(F2) represents a confidence level evaluation value of the blue spectral band whether it is a real target; F3 represents spectral information of red light of a certain pixel of the target point, m3(F3) is a confidence level evaluation value of the red spectral band whether it is a real target; F4 represents spectral information of infrared light of a certain pixel of the target point, m4(F4) is a confidence level evaluation value of the infrared spectral band whether it is a real target; F is a multispectral synthesis result of ultraviolet light F1, blue light F2, red light F3 and infrared light F4, which satisfies the intersection result in set theory, that is, F=F1∩F2∩F3∩F4, wherein ∩ is an intersection operation, thereby obtaining a comprehensive confidence level M(F) of the target pixel-level fusion.

3. The method of claim 1, wherein, In step 6, the result comprehensive confidence level is obtained according to the confidence level curve, which comprises: After restoring the target point information to a spectral image, a confidence level curve is obtained according to the characteristics of the target point information, and a result comprehensive confidence level is calculated, and the calculation formula is as follows: Wherein R is a signal-to-noise ratio of the target point in the frequency bands of ultraviolet light F1, blue light F2, red light F3 and infrared light F4, and c1, c2, τ1, τ2, μ and σ are characteristic coefficients.

4. The method of claim 1, wherein, In step 1, different spectral band sensors are used to acquire each spectral information corresponding to the target information.

5. The method of claim 4, wherein, In step 2, the image signal-to-noise ratio of each spectral information is calculated, and the confidence levels of different spectral band images are determined according to the signal-to-noise ratio at the same time and are assigned to each image; The calculation formula of the image signal-to-noise ratio SNR is as follows: SNR=(point target imaging area gray mean value-background imaging area gray mean value) / background area imaging gray mean square deviation; the signal-to-noise ratio R of the frequency band image corresponding to the same time is calculated for the target point in each frequency band of ultraviolet light, blue light, red light and infrared light, and then the confidence levels of different spectral band images are determined.

6. The method of claim 3, wherein, The different spectral bands include ultraviolet light, blue light, red light and infrared light, and the fusion mode is to select four spectral bands of ultraviolet light, blue light, red light and infrared light for fusion.

7. The method of claim 4, wherein, In step 4, gradient values of the integrated gray image are calculated, and a larger value is taken as a candidate target, and since the candidate target only contains a small amount of real targets, real targets in the image are obtained in combination with motion features of the image at previous and subsequent moments; the real target is a candidate target that continuously moves in the image without a jump change.

8. The method of claim 5, wherein, In step 5, pixel coordinates of the target point are obtained, and the pixel coordinates are mapped to spatial geometric coordinates through a mapping relationship to obtain target information.

9. The method of claim 8, wherein, Mapping the pixel coordinates to the spatial geometric coordinates through the mapping relationship comprises: When the obtained pixel in the image is [uv], then the obtained spatial set coordinates [xy] satisfy the following formula: Wherein a1, a2, a3, a4 are camera parameters.

10. A multispectral sensing and target recognition device based on multi-information fusion, characterized in that, The device comprises: At least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method in any one of claims 1-9.

Images