Image fusion target recognition method and device

Abstract

The embodiment of the present application provides an image fusion target recognition method, comprising the following steps: acquiring an initial infrared image through a scanning imaging mode; constructing a convolution kernel based on a polarization-dependent response characteristic of an infrared photoelectric detector; processing the initial infrared image by using the convolution kernel to obtain a fusion image with fused polarization information; and inputting the fusion image into a target recognition model for target detection and recognition. The image fusion target recognition method provided by the embodiment of the present application acquires an initial infrared image through a scanning imaging mode, and constructs a convolution kernel based on a polarization-dependent response to perform image processing, so that the accuracy and robustness of a target recognition task are improved. The embodiment of the present application further provides an image fusion target recognition device.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and in particular relates to an image fusion target recognition method. Background Technology

[0002] Infrared imaging is widely used in security monitoring, unmanned systems, night vision imaging, and complex environment perception. Existing infrared imaging systems typically only utilize light intensity information for imaging, neglecting the polarization characteristics of light. However, polarization information is of great value in enhancing target edges, distinguishing material properties, and suppressing interference from complex backgrounds.

[0003] Furthermore, traditional machine vision systems require the integration of photosensitive arrays, analog-to-digital converter systems, storage units, and processing units to complete sensing and cognitive tasks. These discrete architectures often lead to problems such as sensor data redundancy, high latency, and high energy consumption. Summary of the Invention

[0004] To address the problems existing in the aforementioned related technologies, this invention provides an image fusion target recognition method. It acquires an initial infrared image through scanning imaging and performs image processing by constructing a convolution kernel based on polarization-dependent response, thereby improving the accuracy and robustness of the target recognition task.

[0005] In a first aspect, embodiments of this application provide an image fusion target recognition method, comprising the following steps: The initial infrared image is acquired through scanning imaging. Convolution kernels are constructed based on the polarization-dependent response characteristics of infrared photodetectors; The initial infrared image is processed using the convolution kernel to obtain a fused image that incorporates polarization information; and The fused image is input into the target recognition model for target detection and recognition.

[0006] Furthermore, the acquisition of the initial infrared image via scanning imaging includes: Infrared laser light is modulated by a pattern mask and then irradiated onto the infrared laser detector; The scanning device is controlled to move along a preset path, and the photocurrent signal at the scanning position of the scanning device is collected synchronously; and Based on the spatial location information corresponding to the scanning path of the scanning device, the acquired photocurrent signal is spatially mapped and reconstructed to form a two-dimensional current distribution matrix in order to obtain the initial infrared image.

[0007] Further, the infrared laser, after being modulated by a patterned mask, illuminates the infrared laser detector, including: The infrared laser is collimated so that it is incident perpendicularly onto the mask surface and illuminates the infrared laser detector. The mask is a metal mask template on which the predefined pattern is etched to define the spatial distribution of the incident light, so as to cooperate with the image scanning device to achieve single-pixel scanning imaging.

[0008] Furthermore, the control scanning device moves along a preset path, synchronously acquiring the photocurrent signal at the scanning position of the scanning device, including: The scanning device is controlled to stay at each step position of the preset path for a preset duration, the preset duration including sampling time and command transmission and reception time; The movement of the scanning device's platform is controlled to match the size of the infrared laser spot, thereby completing the point-by-point scanning of the predefined pattern on the photosensitive surface of the infrared photodetector; and During each step position of the scanning device, the control data acquisition unit synchronously acquires the photocurrent signal generated by the infrared photodetector at that position.

[0009] Furthermore, the construction of the convolution kernel based on the polarization-dependent response characteristics of the detector includes: The infrared photodetector was subjected to polarization-dependent response testing to obtain its photoelectric response characteristics under different polarization state incident conditions; and Based on the difference in response intensity of the infrared photodetector under different polarization state incident conditions, a polarization response model is established, and a corresponding convolution kernel and the weight of the convolution kernel are generated based on the polarization response model.

[0010] Furthermore, the infrared photodetector undergoes polarization-dependent response testing to obtain its photoelectric response characteristics under different polarization state incident conditions, including... After collimation, the infrared laser light passes sequentially through a linear polarizer, a quarter-wave plate, and an objective lens, and is focused onto the surface of the infrared photodetector; and By adjusting the angle between the fast axis of the quarter-wave plate and the transmission direction of the linear polarizer, incident infrared light with different polarization states can be generated, and the current signal generated by the infrared photodetector under the corresponding polarization state can be measured simultaneously using a source meter.

[0011] Furthermore, the convolution kernel is a 3×3 convolution kernel, and the weight value of the convolution kernel is configured according to the photoelectric response intensity of the infrared photodetector under different polarization states.

[0012] Furthermore, the target recognition model is a target detection model based on deep learning.

[0013] Furthermore, the infrared photodetector is fabricated using molybdenum disulfide, topological insulator, or bismuth triselenide / bismuth bilayer superlattice material.

[0014] Secondly, embodiments of this application also provide an image fusion target recognition device, which performs target detection and recognition on the fused image by means of the method described in any of the preceding claims.

[0015] In the image fusion target recognition method provided in this application embodiment, an initial infrared image is first acquired through scanning imaging. Then, a convolutional kernel is constructed based on the polarization-dependent response characteristics of the infrared photodetector. Next, the initial infrared image is processed using the convolutional kernel to obtain a fused image with fused polarization information. Finally, the fused image is input into a target recognition model for target detection and recognition. The image fusion target recognition method provided in this application embodiment acquires an initial infrared image through scanning imaging and performs image processing based on a convolutional kernel constructed from the polarization-dependent response, thereby improving the accuracy and robustness of target recognition tasks in complex scenes. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0017] Figure 1 A schematic flowchart illustrating the image fusion target recognition method provided in this application embodiment; Figure 2 A schematic diagram of the structure of the scanning imaging system involved in the image fusion target recognition method provided in the embodiments of this application; Figure 3 A schematic diagram of the initial infrared image detected by the molybdenum disulfide photodetector involved in the image fusion target recognition method provided in the embodiments of this application; Figure 4 A schematic diagram of the optical path for polarization-dependent response testing of a molybdenum disulfide photodetector involved in the image fusion target recognition method provided in the embodiments of this application; Figure 5 A schematic diagram of a sharpened image related to the image fusion target recognition method provided in the embodiments of this application; Figure 6 This is a comparison chart of image recognition performance before and after preprocessing in the image fusion target recognition method provided in the embodiments of this application.

[0018] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0020] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0021] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. Those skilled in the art will be able to understand the specific meaning of the above terms in this application according to the specific circumstances.

[0022] See Figures 1 to 4 This application provides an image fusion target recognition method, including the following steps: S101: Acquire the initial infrared image through scanning imaging; S102: Constructing convolution kernels based on the polarization-dependent response characteristics of infrared photodetectors; S103: Process the initial infrared image using the convolution kernel to obtain a fused image with integrated polarization information; and S104: Input the fused image into the target recognition model for target detection and recognition.

[0023] The image fusion target recognition method provided in this application first acquires an initial infrared image through scanning imaging. This initial infrared image records the thermal radiation energy distribution of ground objects in the 8-14 micrometer thermal infrared band, but its resolution is usually low and affected by thermal diffusion effects. Then, a convolution kernel is designed based on the sensitivity of infrared photodetectors to the polarization state of incident light. Physically, different materials or geometric shapes of target surfaces will cause unique polarization modulation effects in reflected or emitted infrared light. This convolution kernel is constructed to extract and enhance these polarization-related micro-features, and its parameters are closely related to the calculation of the Stokes vector (used to describe light intensity, degree of linear polarization, and angle of linear polarization). Next, the initial infrared image is processed using this convolution kernel, which essentially performs deep fusion of the target's polarization attribute information, such as edge orientation and material texture details, with traditional thermal radiation intensity information at the feature level. This fusion effectively overcomes the limitations of a single infrared image in terms of low contrast and blurred details. The generated fused image simultaneously retains the target's thermal radiation characteristics and polarization characteristics, providing a richer information foundation for subsequent recognition.

[0024] After image fusion, the method proceeds to the target recognition stage, constructing a complete technical closed loop from feature fusion to intelligent discrimination. The generated fused image is input into a pre-trained target recognition model (e.g., a deep learning-based target detection network) for target detection and classification. This model typically employs a multi-layer convolutional neural network structure (e.g., improved CSPNet, VGG16, etc. backbone networks), capable of automatically learning deep features in the fused image. The model first performs hierarchical visual feature extraction on the input image, and then performs multi-scale feature fusion through a feature pyramid network or similar structure to take into account both global contextual information and local details of the target. During recognition, the model outputs the location information of the predicted bounding box and the category confidence score. The method also features a specially designed feedback optimization mechanism. When the recognition confidence score falls below a preset threshold, the system can re-trigger the feature extraction and fusion process or adjust the fusion strategy, forming a closed-loop system of acquisition, fusion, recognition, and feedback. This continuously improves the target recognition accuracy and the system's adaptability in complex scenarios (e.g., nighttime, fog, camouflage interference, etc.).

[0025] Therefore, in the image fusion target recognition method provided in this application embodiment, an initial infrared image is first acquired through scanning imaging. Then, a convolution kernel is constructed based on the polarization-dependent response characteristics of the infrared photodetector. Next, the initial infrared image is processed using the convolution kernel to obtain a fused image with fused polarization information. Finally, the fused image is input into a target recognition model for target detection and recognition. The image fusion target recognition method provided in this application embodiment acquires an initial infrared image through scanning imaging and performs image processing based on a convolution kernel constructed from the polarization-dependent response, thereby improving the accuracy and robustness of target recognition tasks in complex scenes. Furthermore, the image fusion target recognition method provided in this application fully utilizes the polarization physical characteristics of infrared photodetectors to acquire multi-dimensional optical information, significantly enhancing the image's feature representation capability and scene adaptability. By embedding polarization information into configurable convolution kernels, it achieves hardware and software synergy between image acquisition and information processing, improving the overall system efficiency. Moreover, by tightly coupling the computing unit and the sensing unit at the physical level, information processing can be performed directly at the data acquisition end, effectively reducing data redundancy, transmission delay, and system power consumption. As a front-end preprocessing module, the image fusion target recognition method provided in this application can significantly improve the recognition accuracy and robustness of subsequent detection models without changing the existing target recognition model structure. The overall system structure is clear, easy to integrate, and suitable for various infrared imaging and recognition application scenarios.

[0026] Furthermore, in some embodiments of this application, acquiring the initial infrared image via scanning imaging includes: Infrared laser light is modulated by a pattern mask and then irradiated onto the infrared laser detector; The scanning device is controlled to move along a preset path, and the photocurrent signal at the scanning position of the scanning device is collected synchronously; and Based on the spatial location information corresponding to the scanning path of the scanning device, the acquired photocurrent signal is spatially mapped and reconstructed to form a two-dimensional current distribution matrix in order to obtain the initial infrared image.

[0027] Specifically, the acquisition of the initial infrared image via scanning imaging includes the following steps: First, an infrared laser output from a coherent laser source in the infrared region is modulated by a metal mask with a predefined pattern and then perpendicularly irradiated onto the photosensitive surface of an infrared photodetector with polarization-dependent response characteristics. This mask can precisely define the spatial distribution of the incident light, improving the pattern recognition of the image. Second, a multi-axis stepper motor linear displacement stage with micron-level positioning accuracy is used as a scanning device. It is controlled to move point by point along a pre-planned matrix scanning path, stopping at each step position for a preset duration including sampling time and command transmission / reception time. The nanoampere-level photocurrent signal generated by the detector at the corresponding position is simultaneously acquired and recorded. High-precision displacement control can effectively avoid imaging pixel shift and ensure the stability of signal acquisition. Finally, based on the spatial coordinate information corresponding to the scanning path, the full photocurrent signal is spatially mapped and reconstructed to form a two-dimensional current distribution matrix matching the scanning pixel matrix, thereby acquiring the initial infrared image. This reconstruction method can maximize the preservation of the polarization physical characteristics of infrared light, laying a high-fidelity data foundation for subsequent convolution processing.

[0028] The two-dimensional current distribution matrix is ​​image data formed by arranging the photocurrent signals at each scanning position in spatial order. For example, a grayscale image is generated by arranging the photocurrent signals at 7569 scanning positions in a 78×78 matrix.

[0029] Furthermore, in some embodiments of this application, the infrared laser, after being modulated by a patterned mask, illuminates the infrared laser detector, including: The infrared laser is collimated so that it is incident perpendicularly onto the mask surface and illuminates the infrared laser detector. The mask is a metal mask template on which the predefined pattern is etched to define the spatial distribution of the incident light, so as to cooperate with the image scanning device to achieve single-pixel scanning imaging.

[0030] Specifically, in some embodiments of this application, the infrared laser, after being modulated by a patterned mask, illuminates the infrared laser detector. This process includes: first, collimating the laser output from a coherent laser source in the infrared region to eliminate beam divergence and ensure the laser is incident on the surface of the metal mask in a vertical orientation. The mask is a metal substrate with a preset pattern etched on its surface. The etched pattern can be flexibly customized according to imaging requirements, precisely limiting the spatial distribution range of the incident infrared light and avoiding stray light interference. The infrared laser modulated by the mask forms a beam shape matching the preset pattern and illuminates the photosensitive surface of an infrared photodetector with polarization-dependent response characteristics. This design can be used in conjunction with an image scanning device to achieve high-precision single-pixel scanning imaging, effectively improving the resolution and pattern reproduction of subsequent image reconstruction, and providing a reliable optical input basis for the accurate extraction of polarization information.

[0031] Furthermore, in some embodiments of this application, the control scanning device moves along a preset path, synchronously acquiring the photocurrent signal at the scanning position of the scanning device, including: The scanning device is controlled to stay at each step position of the preset path for a preset duration, the preset duration including sampling time and command transmission and reception time; The movement of the scanning device's platform is controlled to match the size of the infrared laser spot, thereby completing the point-by-point scanning of the predefined pattern on the photosensitive surface of the infrared photodetector; and During each step position of the scanning device, the control data acquisition unit synchronously acquires the photocurrent signal generated by the infrared photodetector at that position.

[0032] Specifically, in some embodiments of this application, the control scanning device moves along a preset path, synchronously acquiring photocurrent signals at the scanning positions of the scanning device. This includes controlling a high-precision scanning device, such as a multi-axis stepper motor linear displacement stage, to move along a pre-planned matrix-style preset path, pausing at each step position for a preset duration. This preset duration covers the photocurrent signal stabilization sampling time, as well as the device command transmission and reception and status calibration time, ensuring the integrity and accuracy of signal acquisition. Simultaneously, the movement step of the scanning device's carrying platform is precisely controlled to match the size of the infrared laser spot, ensuring that the spot can completely cover a single sampling area of ​​the detector's photosensitive surface, thereby completing the point-by-point scanning of the mask's predefined pattern on the detector's photosensitive surface. During the dwell time of the scanning device at each step position, data acquisition units such as a source meter are activated to synchronously acquire and record the nanoampere-level photocurrent signals generated by the infrared photodetector after laser irradiation at the corresponding position, providing high-fidelity raw data support for subsequent image reconstruction.

[0033] For example, the infrared polarization photodetector is made of materials such as molybdenum disulfide or topological insulators, and the scanning device is a multi-axis stepper motor linear displacement stage capable of precise positioning and movement, such as an XY dual-axis stepper motor displacement stage with a positioning accuracy of 2 micrometers; the preset scanning path is a pre-planned motion trajectory, such as a matrix scanning path with a step spacing of 0.5 mm, a coverage area of ​​4.35 cm × 4.35 cm, and a total of 87 × 87 pixels; the photocurrent signal is the electrical signal generated by the infrared photodetector after being illuminated, such as the nanoampere-level current signal generated by the infrared photodetector under 1064 nm laser irradiation.

[0034] Furthermore, in some embodiments of this application, the construction of the convolution kernel based on the polarization-dependent response characteristics of the detector includes: The infrared photodetector was subjected to polarization-dependent response testing to obtain its photoelectric response characteristics under different polarization state incident conditions; and Based on the difference in response intensity of the infrared photodetector under different polarization state incident conditions, a polarization response model is established, and a corresponding convolution kernel and the weight of the convolution kernel are generated based on the polarization response model.

[0035] In other words, the construction of the convolution kernel based on the polarization-dependent response characteristics of the detector specifically includes: First, conducting a polarization-dependent response test on the infrared photodetector. After collimation, the infrared laser light is sequentially focused onto the photosensitive surface of the detector through a linear polarizer, a quarter-wave plate, and an objective lens. By adjusting the angle between the fast axis of the quarter-wave plate and the transmission direction of the linear polarizer, incident infrared light with different polarization states, such as linear polarization, left-hand circular polarization, and right-hand circular polarization, is generated. Simultaneously, the current signal generated by the detector under each polarization state is measured and recorded using a source meter, thereby obtaining the complete photoelectric response characteristics of the detector. Subsequently, based on the difference in response intensity of the detector under different polarization state incident conditions, a precise polarization response model is established. Then, based on the response law of this model, a corresponding convolution kernel is designed and generated, and the response intensity parameters under different polarization states are converted into weight values ​​of the convolution kernel, enabling the convolution kernel to have the processing capability of fusing optical information in the polarization dimension.

[0036] Furthermore, in some embodiments of this application, the polarization-dependent response test of the infrared photodetector to obtain its photoelectric response characteristics under different polarization state incident conditions includes... After collimation, the infrared laser light passes sequentially through a linear polarizer, a quarter-wave plate, and an objective lens, and is focused onto the surface of the infrared photodetector; and By adjusting the angle between the fast axis of the quarter-wave plate and the transmission direction of the linear polarizer, incident infrared light with different polarization states can be generated, and the current signal generated by the infrared photodetector under the corresponding polarization state can be measured simultaneously using a source meter.

[0037] Specifically, the polarization-dependent response test of the infrared photodetector to obtain its photoelectric response characteristics under different polarization states of incidence includes: collimating the infrared laser output from a coherent laser source in the infrared region to eliminate beam divergence and form a parallel beam, then passing it sequentially through a linear polarizer, a quarter-wave plate, and a focusing objective lens, finally focusing it precisely onto the photosensitive surface of the infrared photodetector. By adjusting the angle between the fast axis of the quarter-wave plate and the transmission direction of the linear polarizer, incident infrared light with different polarization states, such as linear polarization, left-hand circular polarization, and right-hand circular polarization, can be generated. Simultaneously, a high-precision source meter is used to measure and record the nanoampere-level current signal generated by the infrared photodetector under the corresponding polarization state of incidence, thereby obtaining the photoelectric response characteristics of the detector in the entire polarization state range, providing accurate experimental data support for the subsequent establishment of a polarization response model.

[0038] It should be noted that when the angle between the fast axis of the quarter-wave plate and the transmission direction of the linear polarizer is 0 degrees, 45 degrees, 90 degrees, 135 degrees and 180 degrees, the corresponding polarization states of the incident light are linear polarization, left-hand circular polarization, linear polarization, right-hand circular polarization and linear polarization, respectively.

[0039] Furthermore, in some embodiments of this application, the convolution kernel is a 3×3 convolution kernel, and the weight value of the convolution kernel is configured according to the photoelectric response intensity of the infrared photodetector under different polarization states.

[0040] Specifically, the current signal intensity data collected by the detector under different polarization states is used as the core basis for assigning weights to the convolution kernel. Regions with high response intensity are assigned higher weights, while regions with low response intensity are assigned relatively lower weights. This weighting configuration allows the convolution kernel to directly reflect the polarization physical characteristics of the detector. In subsequent image processing, it can specifically enhance the optical information in the polarization dimension, improve the recognition of target features, and lay the algorithmic foundation for image enhancement and target recognition that fuse polarization information.

[0041] Of course, it is understood that the convolutional kernels involved in the embodiments of this application are not limited to a 3×3 size, and can be flexibly adjusted according to imaging resolution, target feature scale, and computing power requirements. For example, a 1×1 convolutional kernel is suitable for scenarios with channel dimension feature transformation and limited computing power, without changing the image resolution, while a 5×5 / 7×7 large-size convolutional kernel can cover a larger pixel neighborhood, enhancing the global polarization feature extraction of large-area targets. In short, variable-size convolutional kernels can be dynamically adjusted according to the polarization response of image regions, balancing accuracy and efficiency.

[0042] It should be noted that the embodiments of this application utilize convolution kernels to perform convolution operations on the original infrared image, which can achieve image sharpening, Gaussian filtering, edge extraction and other processing to obtain a processed image with fused polarization information.

[0043] Sharpening enhances image clarity by increasing edge and detail information, for example, making the edge gradients of the "peace dove" pattern more pronounced. Gaussian filtering suppresses Gaussian noise in the image, thus smoothing it, for example, reducing the noise signal intensity in the "peace dove" pattern. Edge extraction extracts the pattern's contour features, for example, revealing the contour structure of the "peace dove" pattern. The processed image is an image that has undergone at least one of the following operations: sharpening, Gaussian filtering, or edge extraction. Furthermore, in some embodiments of this application, the target recognition model is a deep learning-based target detection model.

[0044] Specifically, the target recognition model is a deep learning-based target detection model. This type of model relies on the feature extraction capabilities of deep neural networks to automatically mine deep target features from images fused with polarization information, eliminating the need for manually designed feature operators. By inputting a preprocessed image fused with polarization-dimensional optical information into the model, the advantages of polarization features and deep learning algorithms are fully combined, improving the accuracy and robustness of target detection in complex environments and adapting to diverse infrared sensing applications such as security monitoring, unmanned systems, and night vision imaging.

[0045] Furthermore, in some embodiments of this application, the infrared photodetector is fabricated using molybdenum disulfide, topological insulator, or bismuth triselenide / bismuth bilayer superlattice material.

[0046] All of the aforementioned materials possess excellent infrared band response characteristics and polarization-dependent response capabilities, enabling them to accurately capture the differences in photoelectric signals generated when infrared light of different polarization states is incident. Among them, molybdenum disulfide material has high photoelectric conversion efficiency, making it suitable for high-precision imaging scenarios; topological insulators have strong polarization response sensitivity, which can enhance the detection of weak polarization signals; and bismuth triselenide / bismuth bilayer superlattice material combines wide-band adaptability and stability, meeting the long-term detection requirements in complex environments and providing reliable hardware support for subsequent polarization response model establishment and convolution kernel weight configuration.

[0047] The image fusion target recognition method provided in this application will be further described below with reference to specific embodiments.

[0048] Example 1 Example 1: Image fusion target recognition method based on molybdenum disulfide infrared polarization photodetector. Preprocessing is performed using a sharpening operation, specifically including the following steps: S101: In such Figure 2In the scanning molding system shown, an infrared laser with a wavelength of 1064 nanometers is modulated by a mask with a "dove of peace" pattern and irradiates a molybdenum disulfide infrared polarization photodetector. The XY dual-axis stepper motor displacement stage is controlled to scan a pre-planned motion trajectory. The step spacing is set to 0.5 mm, the coverage area is 4.35 cm × 4.35 cm, and the dwell time at each step position is 0.5 seconds. Simultaneously, the photocurrent signal generated by the detector at each scanning position is collected and recorded using a Keithley source meter.

[0049] S102: Based on the spatial location information corresponding to the scanning path, the 7569 acquired photocurrent signals are spatially mapped and reconstructed to form an 87×87 pixel current distribution matrix, thereby obtaining the measured values ​​as shown below. Figure 3 The original infrared image shown, S103: As Figure 4 As shown, an infrared laser with a wavelength of 1064 nm is collimated and then passed sequentially through a linear polarizer, a quarter-wave plate, and an objective lens, and focused onto the surface of a molybdenum disulfide photodetector. By adjusting the angle between the fast axis of the quarter-wave plate and the transmission direction of the linear polarizer (from 0 degrees to 180 degrees, in 10-degree intervals), incident infrared light with different polarization states is generated, and the current signal generated by the infrared photodetector under the corresponding polarization state is measured simultaneously using a source meter.

[0050] S104: Based on the difference in response intensity of the infrared photodetector under different incident light polarization angles, a polarization response model is established, and a Laplacian sharpening operator is generated. The original infrared image is then convolved using a convolution kernel to achieve image sharpening, resulting in the following: Figure 5 The image shown is a fused image after processing and incorporating polarization information.

[0051] S105: The image after image processing is used as the preprocessing input for the target recognition task and input into the YOLOv8n model for target detection and recognition. This method can significantly improve the accuracy of target detection and the robustness of the system without modifying the existing model structure.

[0052] like Figure 6 As shown, in terms of detection performance, the images preprocessed using the above methods show significant improvements in all core metrics. Specifically, recall increased from 0.8705 in the original image to 0.8805, indicating a reduction in the false negative rate. mAP@0.5 (average precision at IoU threshold 0.5) improved from 0.9414 to 0.9492, demonstrating improved accuracy in target recognition. mAP@0.5:0.95 (average precision at multiple IoU thresholds) increased from 0.6246 to 0.8411, indicating enhanced robustness of the model under different matching criteria.

[0053] The above data differences demonstrate that by fusing polarization information in the preprocessing, the basic features of the infrared image are preserved, while the polarization dimension information is used to enhance the distinction between target edges and materials. Ultimately, without altering the model structure, the detection performance is improved.

[0054] Example 2 Embodiment 2 of the present invention is an image fusion target recognition method based on a bismuth triselenide / bismuth bilayer superlattice infrared polarization photodetector. The method utilizes Gaussian filtering for preprocessing and specifically includes the following steps: S201: An infrared laser with a wavelength of 1550 nm is modulated by a mask with a "butterfly" pattern and irradiates a bismuth triselenide / bismuth double-layer superlattice infrared polarization photodetector. The XY dual-axis stepper motor displacement stage is controlled to scan the pre-planned motion trajectory. The step spacing is set to 0.5 mm, the coverage area is 5 cm × 5 cm, and the dwell time at each step position is 0.5 seconds. The photocurrent signal generated by the detector at each scanning position is collected and recorded simultaneously using a Keithley source meter.

[0055] S202: Based on the spatial location information corresponding to the scanning path, the 10,000 acquired photocurrent signals are spatially mapped and reconstructed to form a current distribution matrix of 100×100 pixels, thereby obtaining the measured original infrared image.

[0056] S203: After collimation, an infrared laser with a wavelength of 1550 nm is passed sequentially through a linear polarizer, a quarter-wave plate, and an objective lens, and focused onto the surface of a bismuth selenide / bismuth bilayer superlattice photodetector. By adjusting the angle between the fast axis of the quarter-wave plate and the transmission direction of the linear polarizer (from 0 degrees to 180 degrees, in 10-degree intervals), incident infrared light with different polarization states is generated, and the current signal generated by the infrared photodetector under the corresponding polarization state is measured simultaneously using a source meter.

[0057] S204: Based on the difference in response intensity of the infrared photodetector under different incident light polarization angles, a polarization response model is established, and a Gaussian smoothing operator is generated. A convolution kernel is used to perform convolution operations on the original infrared image to achieve Gaussian filtering, resulting in a processed image with fused polarization information.

[0058] S205: The image after image processing is used as the preprocessing input for the target recognition task and input into the YOLOv5 model for target detection and recognition. This method can significantly improve the accuracy of target detection and the robustness of the system without modifying the existing model structure.

[0059] Furthermore, this application also provides an image fusion target recognition device, which performs target detection and recognition on the fused image using the method described above. This image fusion target recognition method refers to the above embodiments. Since the image fusion target recognition method adopts all the technical solutions of all the above embodiments, it at least has all the beneficial effects brought about by the technical solutions of the above embodiments, and will not be elaborated further here.

[0060] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image fusion target recognition method, characterized in that, Includes the following steps: The initial infrared image is acquired through scanning imaging. Convolution kernels are constructed based on the polarization-dependent response characteristics of infrared photodetectors; The initial infrared image is processed using the convolution kernel to obtain a fused image with integrated polarization information; and The fused image is input into the target recognition model for target detection and recognition.

2. The image fusion target recognition method according to claim 1, characterized in that, The acquisition of the initial infrared image via scanning imaging includes: Infrared laser light is modulated by a pattern mask and then irradiated onto the infrared laser detector; The scanning device is controlled to move along a preset path, and the photocurrent signal at the scanning position of the scanning device is collected synchronously; and Based on the spatial location information corresponding to the scanning path of the scanning device, the acquired photocurrent signal is spatially mapped and reconstructed to form a two-dimensional current distribution matrix in order to obtain the initial infrared image.

3. The image fusion target recognition method according to claim 2, characterized in that, The infrared laser, modulated by a patterned mask, illuminates the infrared laser detector, including: The infrared laser is collimated so that it is incident perpendicularly onto the mask surface and illuminates the infrared laser detector. The mask is a metal mask template on which the predefined pattern is etched to define the spatial distribution of the incident light, so as to cooperate with the image scanning device to achieve single-pixel scanning imaging.

4. The image fusion target recognition method according to claim 3, characterized in that, The control scanning device moves along a preset path, synchronously acquiring photocurrent signals at the scanning positions of the scanning device, including: The scanning device is controlled to stay at each step position of the preset path for a preset duration, the preset duration including sampling time and command transmission and reception time; The movement of the scanning device's platform is controlled to match the size of the infrared laser spot, thereby completing the point-by-point scanning of the predefined pattern on the photosensitive surface of the infrared photodetector; and During each step position of the scanning device, the control data acquisition unit synchronously acquires the photocurrent signal generated by the infrared photodetector at that position.

5. The image fusion target recognition method according to claim 1, characterized in that, The convolution kernel constructed based on the polarization-dependent response characteristics of the detector includes: The infrared photodetector was subjected to polarization-dependent response testing to obtain its photoelectric response characteristics under different polarization state incident conditions; and Based on the difference in response intensity of the infrared photodetector under different polarization state incident conditions, a polarization response model is established, and a corresponding convolution kernel and the weight of the convolution kernel are generated based on the polarization response model.

6. The image fusion target recognition method according to claim 5, characterized in that, The polarization-dependent response test of the infrared photodetector is performed to obtain its photoelectric response characteristics under different polarization state incident conditions, including... After being collimated, the infrared laser is passed sequentially through a linear polarizer, a quarter-wave plate, and an objective lens, and then focused onto the surface of the infrared photodetector. and By adjusting the angle between the fast axis of the quarter-wave plate and the transmission direction of the linear polarizer, incident infrared light with different polarization states can be generated, and the current signal generated by the infrared photodetector under the corresponding polarization state can be measured simultaneously using a source meter.

7. The image fusion target recognition method according to claim 6, characterized in that, The convolution kernel is a 3×3 convolution kernel, and the weight values ​​of the convolution kernel are configured according to the photoelectric response intensity of the infrared photodetector under different polarization states.

8. The image fusion target recognition method according to claim 1, characterized in that, The target recognition model is a deep learning-based target detection model.

9. The image fusion target recognition method according to claim 1, characterized in that, The infrared photodetector is made of molybdenum disulfide, topological insulator, or bismuth triselenide / bismuth bilayer superlattice material.

10. An image fusion target recognition device, characterized in that, The image fusion target recognition device performs target detection and recognition on the fused image by means of the method described in any one of claims 1 to 9.

Images