Image reconstruction method and device in a multi-interference source scenario

Abstract

The present invention discloses a method and device for image reconstruction in a multi-interference source scenario, the method comprising the following steps: S1. For an input image, using multiple visual feature encoders for different interference scenarios to extract features and perform feature fusion; S2. An interference type classification model identifies interference factors based on the fused features; S3. Based on the identification results of the interference factors, a corresponding correction algorithm is used to correct the image; S4. Image reconstruction is performed based on the corrected image features to generate a final imaging result. The present invention obtains multiple data sources through multi-sensor fusion technology, combines deep learning algorithms to analyze and process the data, and achieves accurate identification and classification of different types of interference, thereby optimizing imaging and improving imaging quality and efficiency. The present invention has broad application prospects and is of positive significance for improving the level of imaging technology and promoting the development of related fields.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to an image reconstruction method and device in a multi-interference source scenario. Background Art

[0002] In today's world, with the rapid development of science and technology and the widespread adoption of its applications, imaging technology plays a vital role in various fields. However, in practical applications, many scenes often suffer from various types of interference, such as lighting changes and occlusions. These interference factors seriously affect imaging quality and efficiency, limiting the scope and effectiveness of imaging technology.

[0003] Traditional imaging methods often struggle to effectively address the challenges posed by multi-type interference scenarios. Existing imaging technologies are mostly based on single sensors or imaging modalities, which have limitations in image extraction and reconstruction in complex interference scenarios. Therefore, there is an urgent need for an image reconstruction method and device for scenarios with multiple interference sources that can fully leverage multimodal perception and intelligent data processing technologies to improve imaging robustness and reliability, thereby meeting the demand for high-quality imaging in practical applications. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to address the deficiencies of the above-mentioned existing technologies and provide an image reconstruction method and device in a multi-interference source scenario. Through multimodal perception and intelligent data processing technology, accurate identification and classification of interference can be achieved, thereby optimizing the imaging process, improving imaging quality and efficiency, providing more reliable and high-quality imaging solutions for various application scenarios, and promoting the development of imaging technology and the expansion of its application scope.

[0005] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:

[0006] An image reconstruction method in a multi-interference source scenario includes the following steps:

[0007] S1. For the input image, use multiple visual feature encoders for different interference scenes to extract features and perform feature fusion;

[0008] S2, the interference type classification model identifies interference factors based on the fused features;

[0009] S3, according to the identification result of the interference factor, using the corresponding correction algorithm to correct the image;

[0010] S4. Perform image reconstruction based on the corrected image features to generate the final imaging result.

[0011] To optimize the above technical solutions, specific measures taken also include:

[0012] The features extracted by S1 mentioned above include color, texture, and shape.

[0013] In S1 above, the visual feature encoder first uses the activation function to obtain the weight vector of the input image, multiplies it with the visual feature, and performs feature channel-level fusion;

[0014] Then, convolution and Sigmoid activation function are used to obtain the weight map, and the weight map is multiplied with the corresponding feature to obtain the spatial fusion feature.

[0015] Finally, the feature maps after channel fusion and spatial fusion are added together to obtain the final fused visual features.

[0016] YOLOv5 is used as a joint model of the visual feature encoder and the interference type classification model. The joint model includes an encoder, a bidirectional adaptive feature fusion module, a backbone network, a feature pyramid, and a detection head. The encoder and the bidirectional adaptive feature fusion module constitute the visual feature encoder, while the backbone network, feature pyramid, and the detection head constitute the interference type classification model.

[0017] The visual feature encoder uses a pre-trained model and is fine-tuned with a small learning rate during joint training. The first two layers of the YOLOv5 backbone network are removed, leaving only the last three layers. The model input is replaced with the adaptively fused visual features. The loss function used for constraints in the joint model training process is as follows:

[0018] Loss=λ1Loss L2 (X,Y)+λ2Loss char (X,Y)+λ2Loss edge (X,Y)

[0019] Where X and Y are the latent space features of the input, and λ1, λ2, λ2 are the coefficients of the corresponding loss;

[0020] Loss L2 (X,Y) is L2 loss, Loss char (X,Y) is Charbonnier Loss, Loss edge (X,Y) is EdgeLoss, the specific formula is:

[0021] Loss L2 (X,Y)=‖XY‖2

[0022]

[0023] Where Δ is the Laplace operator and ε is a very small constant.

[0024] The above-mentioned S2 includes the following sub-steps:

[0025] S201, using multiple sensors to acquire image feature data in a multi-interference source scenario to form a multimodal data set;

[0026] S202. Using a deep learning algorithm to train and optimize the multimodal data set to obtain an interference type classification model;

[0027] S203: Using an interference type classification model, perform interference scene recognition and analysis on the image feature data acquired in real time to determine the type and intensity of various interference factors.

[0028] The above S3 includes the following sub-steps:

[0029] According to the type and intensity of the interference factors, a corresponding correction algorithm is selected, which is divided into image deblurring, noise reduction, and contrast enhancement algorithms;

[0030] Determine the corresponding correction parameters according to the specific interference situation and correction algorithm;

[0031] Applying correction algorithms and parameters to image data to perform interference correction processing on the image data, including adjusting the pixel value, contrast, and color of the image;

[0032] Evaluate the corrected data and evaluate the correction effect by combining quantitative indicators and qualitative evaluation;

[0033] Based on the evaluation results, the correction algorithm and parameters are optimized and adjusted.

[0034] The above-mentioned S4 includes the following sub-steps:

[0035] S401, reconstructing and optimizing the target image using a corresponding imaging algorithm based on the corrected image features to achieve real-time imaging;

[0036] S402: Feedback adjustment is performed on the interference type classification model and the imaging algorithm according to the real-time imaging effect.

[0037] The above-mentioned S402 evaluates the image quality by performing structural similarity and peak signal-to-noise ratio on the reconstructed image.

[0038] An image reconstruction device in a multi-interference source scenario utilizes the above-mentioned image reconstruction method in a multi-interference source scenario.

[0039] The present invention has the following beneficial effects:

[0040] This method uses multi-sensor fusion technology to acquire multiple data sources and combines it with deep learning algorithms to analyze and process the data, accurately identifying and classifying different types of interference, thereby optimizing the imaging process and improving imaging quality and efficiency. This method has broad application prospects and can play an important role in security monitoring, medical imaging, aerospace, and other fields. It has positive significance for improving the level of imaging technology and promoting the development of related fields. Specific beneficial effects are as follows:

[0041] (1) Improve imaging quality: By accurately identifying and classifying multiple types of interference, the imaging process is optimized, the impact of interference on image quality is effectively reduced, and the clarity and accuracy of imaging are improved.

[0042] (2) Enhanced imaging stability: Real-time identification and analysis of interference scenarios, and taking corresponding measures to optimize the imaging process, so that the system can better adapt to changes in complex environments and improve imaging stability and reliability.

[0043] (3) Improve imaging efficiency: Through intelligent data processing and optimization algorithms, rapid response and processing of interference can be achieved, imaging time can be shortened, imaging efficiency can be improved, and application scenarios with high real-time requirements can be met.

[0044] (4) Expanding application areas: The present invention is applicable to imaging needs in various complex interference scenarios, including security monitoring, medical imaging, aerospace and other fields, providing more reliable and high-quality imaging solutions for these fields, and promoting the development of related technologies and the expansion of their application scope. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flow chart of the method of the present invention;

[0046] Figure 2 It is a schematic diagram of the joint model of visual feature extraction and image interference classification;

[0047] Figure 3 It is a schematic diagram of the image reconstruction device module in a multi-interference scenario. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0049] Although the steps in the present invention are arranged with numbers, they are not intended to limit the order of the steps. Unless the order of the steps is clearly stated or the execution of a step requires other steps as a basis, the relative order of the steps can be adjusted. It is understood that the term "and / or" used herein refers to and covers any and all possible combinations of one or more of the associated listed items.

[0050] like Figure 1-2 As shown, the image reconstruction method in a multi-interference source scenario of the present invention achieves accurate extraction and reconstruction of the target image by classifying and analyzing different types of interference. It is applicable to imaging requirements in various complex interference scenarios and has high practicality and application value. The method includes the following steps:

[0051] S1. For the input image, use multiple visual feature encoders for different interference scenes to extract features and perform feature fusion;

[0052] Feature extraction: Extract feature information from images, including color, texture, shape and other features.

[0053] Step S1 includes:

[0054] S101. Extract features such as texture, color, shape, etc. from the identified interference data.

[0055] S102: Use image processing technology or signal processing technology to perform feature extraction to obtain key feature information of interference factors.

[0056] For the input image, multiple feature encoders for different interference scenarios are used to extract features for subsequent target positioning and classification.

[0057] Since the input image may be obtained in different types of interference scenes, multi-visual feature fusion is required, and the network model adopts a bidirectional adaptive strategy.

[0058] The visual feature encoder first uses the activation function to obtain a weight vector, which is multiplied by the visual feature to perform feature channel-level fusion.

[0059] Then, convolution and Sigmoid activation function are used to obtain the weight map, and multiply it with the corresponding feature to obtain the spatial fusion feature.

[0060] Finally, the feature maps after channel fusion and spatial fusion are added together to obtain the final fused visual features.

[0061] Through bidirectional adaptive fusion, the noise information brought by direct fusion can be reduced and more accurate visual features can be obtained.

[0062] YOLOv5 is used as the basic target detection model for the joint visual feature encoder and interference type classification model. The joint model includes an encoder, a bidirectional adaptive feature fusion module, a backbone network, a feature pyramid, and a detection head. The encoder and the bidirectional adaptive feature fusion module constitute the visual feature encoder, while the backbone network, feature pyramid, and detection head constitute the interference type classification model.

[0063] The visual feature encoder uses a pre-trained model and is fine-tuned with a small learning rate during joint training. The first two layers of the backbone network are removed, leaving only the last three layers. The model input is replaced with the adaptively fused visual features. The loss function used for constraints in the joint model training process is as follows:

[0064] Loss=λ1Loss L2 (X,Y)+λ2Loss char (X,Y)+λ2Loss edge (X,Y)

[0065] Where X and Y are the latent space features of the input, and λ1, λ2, λ2 are the coefficients of the corresponding loss;

[0066] Loss L2 (X,Y) is L2 loss, Loss char (X,Y) is Charbonnier Loss, Loss edge (X,Y) is EdgeLoss, the specific formula is:

[0067] Loss L2 (X,Y)=‖XY‖2

[0068]

[0069] Where Δ is the Laplace operator and ε is a very small constant.

[0070] During model training, a smaller learning rate is used for fine-tuning.

[0071] S2, the interference type classification model identifies interference factors based on the extracted features;

[0072] Interference factor identification: Analyze the input image and identify various interference factors, such as lighting changes and occlusions.

[0073] Before performing interference correction, you first need to obtain the results of interference identification to determine the type of interference and the extent of its impact.

[0074] In order to enable the present invention to have the ability to quickly identify interference factors, the present invention has built a classification model based on deep learning.

[0075] First, a variety of sensors (such as optical, infrared, radar, etc.) are used to acquire image data of different modalities to form a multimodal dataset.

[0076] The collected data is then preprocessed, including denoising, filtering, and registration, to improve data quality and consistency.

[0077] The model is trained using labeled data sets to establish an interference factor classification model.

[0078] During the training process, the validation set is used to verify the currently established model, evaluate its classification performance and generalization ability, and ensure the accuracy and stability of the model.

[0079] Interference factors are identified through a pre-established interference factor classification model.

[0080] Therefore, step S2 includes the following sub-steps:

[0081] S201, Multi-sensor fusion: Utilize multiple sensors to acquire data of different modalities, including optical, infrared, radar, etc., to form a multi-modal data set.

[0082] S202. Establishing an interference classification model: Using deep learning algorithms to train and optimize the data set, establish an interference type classification model to achieve accurate identification and classification of interference.

[0083] S203. Real-time interference identification and analysis: Based on the classification model, interference scenarios are identified and analyzed on the real-time collected data to determine the impact and type of each type of interference.

[0084] S3. Correct the image using a corresponding correction method based on the identification result of the interference factor;

[0085] Interference Correction: Based on the type and intensity of interference factors, such as rain and fog intensity, blur, and light intensity, we apply corresponding correction algorithms to the image to reduce the impact of interference. Correction techniques eliminate or reduce the impact of interference on imaging results, improving image quality and accuracy.

[0086] According to the type of interference and the identification results, an appropriate correction algorithm is selected.

[0087] Different types of interference may require different correction strategies, such as image deblurring, noise reduction, contrast enhancement, etc.

[0088] According to the specific interference situation and the correction algorithm, the corresponding correction parameters are determined. These parameters can be pre-set or dynamically adjusted according to the actual situation.

[0089] Apply the correction algorithm and parameters to the original data to perform interference correction on the data.

[0090] This may involve adjusting the pixel values, contrast, color, etc. of the image to eliminate or reduce the effects of interference.

[0091] Evaluate the corrected data to check whether the correction effect meets expectations.

[0092] Evaluation can be carried out by combining quantitative indicators (such as image clarity, noise level, etc.) and qualitative evaluation (manual observation).

[0093] Based on the evaluation results, the correction algorithm and parameters are optimized and adjusted. This may involve further refining the correction strategy and adjusting parameters to improve the correction effect and image quality.

[0094] S4. Perform image reconstruction based on the corrected image features to generate the final imaging result.

[0095] Image reconstruction: Based on the corrected image features, an image reconstruction algorithm is used to generate the final imaging result.

[0096] Step S4 includes the following sub-steps:

[0097] S401, imaging algorithm optimization: Based on the interference identification results, a suitable imaging algorithm is used to reconstruct and optimize the target image to improve the imaging quality and accuracy.

[0098] S402, real-time feedback adjustment: Based on the real-time imaging effect, feedback adjustment is performed on the classification model and imaging algorithm to continuously optimize system performance and improve imaging efficiency and stability.

[0099] Specifically, based on the interference-corrected data, raw data for image reconstruction is prepared, which may be a multimodal dataset that has been preprocessed, interference removed, and corrected.

[0100] The system will automatically call the appropriate image reconstruction algorithm for the original data that needs to be reconstructed based on the interference detection results in step 2, specific imaging requirements, and data characteristics. In addition, the user can adjust the reconstruction parameters according to specific needs to optimize the imaging results.

[0101] After the image is reconstructed, the system evaluates the reconstructed image, including indicators such as clarity, accuracy, and noise level.

[0102] Two quantitative evaluation methods, structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR), are used to judge image quality.

[0103] According to the evaluation results, the transfer parameters of the joint loss function are adjusted, and the reconstruction algorithm is optimized and adjusted to improve the quality and effect of image reconstruction.

[0104] The peak signal-to-noise ratio (PSNR) formula is expressed as:

[0105]

[0106] Among them, MSE represents the mean square error between the current image and the reference image, which is expressed as

[0107]

[0108] X(i,j), Y(i,j) represent the pixel values ​​at the corresponding coordinates, H, W are the height and width of the image, and MaxValue represents the maximum value that the image pixel can take. For example, the MaxValue value of an 8-bit image is 2 8 -1=255.

[0109] The structural similarity (SSIM) formula is expressed as:

[0110]

[0111] where μ x ,σ x ,σ xy , represents the mean, standard deviation of image x and the covariance of image x, y.

[0112] Accordingly, the image reconstruction device module for implementing the above reconstruction method in a multi-interference scenario is as follows: Figure 3 As shown, the image restoration framework under the above-mentioned multi-type interference scenarios is integrated into an easy-to-deploy image reconstruction device that can be used in various visual systems, which can improve the imaging quality and imaging efficiency of images under interference.

[0113] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

[0114] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. An image reconstruction method in a multi-interference source scenario, characterized in that: The following steps are involved: S1. For the input image, use multiple visual feature encoders for different interference scenes to extract features and perform feature fusion; S2, the interference type classification model identifies interference factors based on the fused features; S3, according to the identification result of the interference factor, using the corresponding correction algorithm to correct the image; S4, performing image reconstruction based on the corrected image features to generate a final imaging result; In S1, the visual feature encoder first uses an activation function to obtain a weight vector of the input image, multiplies it by the visual feature, and performs feature channel-level fusion; Then, convolution and Sigmoid activation function are used to obtain the weight map, and the weight map is multiplied with the corresponding feature to obtain the spatial fusion feature. Finally, the feature maps after channel fusion and spatial fusion are added together to obtain the final fused visual features; YOLOv5 is used as a joint model of the visual feature encoder and the interference type classification model. The joint model includes an encoder, a bidirectional adaptive feature fusion module, a backbone network, a feature pyramid, and a detection head. The encoder and the bidirectional adaptive feature fusion module constitute the visual feature encoder, while the backbone network, feature pyramid, and the detection head constitute the interference type classification model. The visual feature encoder uses a pre-trained model and is fine-tuned with a small learning rate during joint training. The first two layers of the YOLOv5 backbone network are removed, leaving only the last three layers. The model input is replaced with the adaptively fused visual features. The loss function used for constraints during joint model training is as follows: ； Where X and Y are the latent space features of the input, is the coefficient corresponding to the loss; is the L2 loss, For Charbonnier Loss, It is Edge Loss, and the specific formula is: ； in is the Laplace operator, is a very small constant.

2. The image reconstruction method in a multi-interference source scenario according to claim 1, characterized in that: The features extracted by S1 include color, texture, and shape.

3. The image reconstruction method in a multi-interference source scenario according to claim 1, characterized in that: Step S2 includes the following sub-steps: S201, using multiple sensors to acquire image feature data in a multi-interference source scenario to form a multimodal data set; S202. Using a deep learning algorithm to train and optimize the multimodal data set to obtain an interference type classification model; S203: Using an interference type classification model, perform interference scene recognition and analysis on the image feature data acquired in real time to determine the type and intensity of various interference factors.

4. The image reconstruction method in a multi-interference source scenario according to claim 1, characterized in that: Step S3 includes the following sub-steps: According to the type and intensity of the interference factors, a corresponding correction algorithm is selected, which is divided into image deblurring, noise reduction, and contrast enhancement algorithms; Determine the corresponding correction parameters according to the specific interference situation and correction algorithm; Applying correction algorithms and parameters to image data to perform interference correction processing on the image data, including adjusting the pixel value, contrast, and color of the image; Evaluate the corrected data and evaluate the correction effect by combining quantitative indicators and qualitative evaluation; Based on the evaluation results, the correction algorithm and parameters are optimized and adjusted.

5. The image reconstruction method in a multi-interference source scenario according to claim 1, characterized in that: Step S4 includes the following sub-steps: S401, reconstructing and optimizing the target image using a corresponding imaging algorithm based on the corrected image features to achieve real-time imaging; S402: Feedback adjustment is performed on the interference type classification model and the imaging algorithm according to the real-time imaging effect.

6. The image reconstruction method in a multi-interference source scenario according to claim 5, characterized in that: The S402 evaluates the image quality by performing structural similarity and peak signal-to-noise ratio on the reconstructed image.

7. An image reconstruction device in a multi-interference source scenario, characterized in that: The device includes a processor and a memory, and the processor is used to implement the image reconstruction method in a multi-interference source scenario according to any one of claims 1 to 6 when executing a program stored in the memory.

Images