Multi-spectral fusion system and method based on large-view-field aspheric perspective group

Abstract

The invention provides a multispectral image fusion system and method based on a large-view-field aspheric perspective group, and belongs to the technical field of electrical equipment monitoring. The multispectral image fusion system comprises a multispectral image acquisition module, a data preprocessing module, an image fusion module, a data storage module and a display module; the multispectral image acquisition module comprises an ultraviolet image acquisition unit, an infrared image acquisition unit and a visible light image acquisition unit which are respectively used for acquiring ultraviolet, infrared and visible light image data of tested equipment, and the infrared image acquisition unit uses an optical system of a large-view-field aspheric perspective group as an infrared lens; the data preprocessing module receives image data acquired by the multispectral image acquisition module, preprocesses the image data and then transmits the image data to the image fusion module, the image fusion module is used for fusing the preprocessed multispectral image data to generate a fused image, and the data storage module is used for storing the fused image. The display module is used for displaying the fused image in real time.

Description

Technical Field

[0001] The present invention relates to the technical field of electrical equipment monitoring, and in particular to a multi-spectral fusion system and method based on a large-field-of-view aspheric perspective group. Background Art

[0002] In the power system, the safe operation of high-voltage electrical equipment is of vital importance. However, affected by the operating environment and bad weather, these equipment will produce discharge phenomena, resulting in reduced insulation performance, increasing the risk of equipment defects, and affecting the safe and stable operation of the power grid. At present, about 24.24% of the fault tripping of 10 kV and above transmission and transformation equipment is related to external insulation discharge. Therefore, it is particularly important to monitor and evaluate the external insulation discharge defect status of electrical equipment. Traditional maintenance methods rely on manual inspections and regular maintenance, which are inefficient and costly.

[0003] Research and application at home and abroad have shown that ultraviolet imaging technology has become an effective means of power system discharge detection. Foreign countries such as Russia, Israel and South Africa have widely used ultraviolet imagers for power system discharge detection and achieved good results, but the price is expensive. China has also begun to apply ultraviolet imaging technology since 2002, and has gradually explored its practical application in power equipment detection. However, the existing ultraviolet imagers still have some limitations, such as the lack of integration of infrared imaging technology, insufficient data management functions, and lack of integrated automation technology, network interaction, etc. These problems limit the application efficiency and accuracy of ultraviolet imaging technology in power systems, and further technical improvements and innovations are urgently needed. Summary of the invention

[0004] In order to solve the deficiencies in the prior art, the present invention provides a multi-spectral fusion system and method based on a large-field-of-view aspherical perspective group.

[0005] The present invention adopts the following technical solution.

[0006] A first aspect of the present invention provides a multi-spectral fusion system based on a large-field-of-view aspherical perspective group, the system comprising:

[0007] Multispectral image acquisition module, data preprocessing module, image fusion module, data storage module and display module;

[0008] The multispectral image acquisition module includes an ultraviolet image acquisition unit, an infrared image acquisition unit and a visible light image acquisition unit, which are respectively used to obtain ultraviolet, infrared and visible light image data of the device under test, wherein the infrared image acquisition unit uses an optical system of a large-field-of-view aspherical perspective group as an infrared lens; the data preprocessing module receives the image data acquired by the multispectral image acquisition module, preprocesses it and then transmits it to the image fusion module, the image fusion module is used to fuse the preprocessed multispectral image data to generate a fused image, the data storage module is used to store the fused image, and the display module is used to display the fused image in real time.

[0009] Optionally, the multispectral image acquisition module includes an ultraviolet image acquisition unit, a visible light image acquisition unit and an infrared image acquisition unit, which are integrated in the multispectral image acquisition module to simultaneously acquire ultraviolet, visible light and infrared image data of the device under test;

[0010] Among them, the ultraviolet image acquisition unit is used to collect ultraviolet light images. The ultraviolet image acquisition unit includes a spectroscope, an ultraviolet filter, an ultraviolet enhancement system, a lens and an ultraviolet sensor. When the light enters the multi-spectral image acquisition module, the ultraviolet light is separated by the spectroscope, and the ultraviolet light enters the ultraviolet enhancement system after further filtering by the ultraviolet filter. After being enhanced by the ultraviolet enhancement system, the ultraviolet light enters the ultraviolet sensor through the lens;

[0011] The visible light image acquisition unit is used to acquire visible light images. The visible light image acquisition unit includes a total reflector, a visible light lens, and a visible light high-speed camera module. The visible light separated by the spectroscope of the ultraviolet image acquisition unit enters the visible light image acquisition unit, and after being reflected by the total reflector, enters the visible light high-speed camera module through the visible light lens.

[0012] The infrared image acquisition unit is used to acquire infrared light images. The infrared image acquisition unit includes an infrared lens, a fiberscope and a detector. The thermal radiation passes through the infrared lens and the fiberscope in sequence and then enters the detector.

[0013] Optionally, the optical system used by the infrared lens in the infrared image acquisition unit is a large-field-of-view aspherical perspective group zoom system, specifically comprising:

[0014] The large-field aspherical perspective zoom system consists of 11 refractive lenses, with a full field of view of 140°, an F number of 2.8, and supports an object distance between 30cm and 100cm;

[0015] The optical system comprises eleven lenses and a bandpass filter, the first lens, the third lens, the fourth lens, the fifth lens and the second cemented lens group have negative optical power, the second lens, the first cemented lens group, the aperture stop, the tenth lens and the eleventh lens have positive optical power, wherein the first cemented lens group is composed of the sixth lens and the seventh lens, the second lens group is composed of the sixth lens with positive optical power and the seventh lens with positive optical power, and the second cemented lens group is composed of the eighth lens with positive optical power and the ninth lens with negative optical power.

[0016] Optionally, the data preprocessing module is used to preprocess the ultraviolet image, infrared image and visible light image acquired by the multispectral image acquisition module; wherein the preprocessing operation includes video decoding, and the image format transmitted to the data preprocessing module by the multispectral image acquisition module is a compressed digital video signal. The data preprocessing module restores the compressed digital video signal to a visual video image through video decoding, and provides readable data support for subsequent multispectral image fusion.

[0017] Optionally, the image fusion module includes a multispectral image fusion model, which includes a generator, a discriminator and a quality assessment unit;

[0018] Among them, the generator is used to generate a fused image, the discriminator is used to evaluate the quality of the fused image, the quality assessment unit is used to evaluate the quality of the source image, and the evaluation result is used to adjust the loss function of the generator, thereby controlling the adaptive fusion balance of infrared, ultraviolet and visible light features.

[0019] The second aspect of the present invention provides a multi-spectral fusion method of a large-field-of-view aspheric perspective group, based on the multi-spectral fusion system based on a large-field-of-view aspheric perspective group described in the first aspect of the present invention, comprising the following steps:

[0020] Step 1, collecting ultraviolet, infrared and visible light images of the device under test;

[0021] Step 2, preprocessing the ultraviolet, infrared and visible light images collected in step 1;

[0022] Step 3: Perform multispectral image fusion based on the multispectral image preprocessed in step 2 to obtain a fused image, and display and store the fused image in real time.

[0023] Optionally, in step 3, performing multispectral image fusion on the multispectral image specifically includes:

[0024] The preprocessed multispectral image is input into the multispectral image fusion model. The quality of the source image is first evaluated by the quality assessment unit, and the evaluation result is used to adjust the loss function of the generator, and then the generator generates a fused image;

[0025] The generator includes a strong attention unit, a scene texture unit and three branches; the three branches of the generator are used to extract the features of infrared, ultraviolet and visible light images respectively, obtain their feature vectors and input them into the strong attention unit; the strong attention unit performs feature enhancement based on the input feature vector, and inputs the enhanced feature map into the scene texture unit; the scene texture unit performs image fusion based on the enhanced feature map and outputs a fused image.

[0026] Optionally, the strong attention unit performs feature enhancement based on the input feature vector specifically including:

[0027] The feature vector input to the strong attention unit is Where H, W and C represent the height, width and number of channels of the feature map respectively. A set of transformed feature vectors A′ and A″ are obtained through a series of convolution operations and flattened into a two-dimensional matrix. The attention weight map M is calculated as shown in the following formula:

[0028]

[0029] Use the attention weight map M to weight the feature A′ and obtain the weighted feature map A α :

[0030] A α =M·A′

[0031] Finally, the weighted feature map A is adjusted by a learnable scale parameter β α The importance of is calculated and fused with the original feature A to obtain the output O of the strong attention unit:

[0032] O=β·A α +A

[0033] The output O of the strong attention unit is the enhanced feature map.

[0034] Optionally, the scene texture unit performs image fusion based on the enhanced feature map using a formula expressed as:

[0035]

[0036] Among them, F l represents the feature map of the current level l, F l-1 Represents the feature map of the previous level, Represents the connection operation of the feature map, Conv 3×3 Represents a 3x3 convolution operation.

[0037] Optionally, the evaluation function of the quality evaluation unit is:

[0038]

[0039] Among them, S(I) represents the information entropy of the input image, I vis ,I ir ,I er Represent visible light image and infrared and ultraviolet images respectively. S(I) can be expressed as follows:

[0040]

[0041] Among them, p(i) is the proportion of pixels with grayscale value i in the input image.

[0042] The beneficial effect of the present invention is that, compared with the prior art, since a single spectrum image cannot well locate the fault potential point of the equipment, the infrared image can only reflect the infrared radiation information of the target, that is, the characteristics of the equipment temperature, and is insensitive to the changes in the brightness detail information of the scene, while the ultraviolet image can only reflect the corona and surface partial discharge information of the target, and can find the equipment defects that cause the electric field abnormality, and the visible light is more sensitive to the changes in the brightness detail information of the scene, and the imaging clarity is high, and their functions form a complementary relationship. Therefore, a multi-spectral image fusion model is proposed, which fuses the infrared, ultraviolet, and visible light spectra, which can reflect both the infrared radiation information and the corona and surface partial discharge information, while taking into account the scene brightness detail information, and can obtain more detailed and accurate information at the same time. It brings great convenience to the safe operation and status maintenance of the power grid, and at the same time reduces the economic losses and safety problems caused by large-scale power outages of the power grid caused by personnel misoperation or negligence. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of the framework of a fixed miniaturized online discharge monitoring module system;

[0044] Figure 2 Schematic diagram of a multispectral image fusion method based on strong attention units and scene texture;

[0045] Figure 3 It is a schematic diagram of the business structure of instruments, conversion devices and mobile terminals. DETAILED DESCRIPTION

[0046] The present invention is further described below in conjunction with embodiments. It is necessary to point out that the following embodiments are only used to further illustrate the present invention and cannot be understood as limiting the scope of protection of the present invention. Some non-essential improvements and adjustments made by technical personnel in this field based on the above invention content still fall within the scope of protection of the present invention.

[0047] In Example 1, the present invention provides a multi-spectral fusion system based on a large-field-of-view aspherical perspective group, and the system specifically includes:

[0048] Multispectral image acquisition module, data preprocessing module, image fusion module, data storage module and display module;

[0049] The multispectral image acquisition module is used to obtain multispectral image data of the device under test, including ultraviolet, infrared and visible light image data; the data preprocessing module receives the image data collected by the multispectral image acquisition module, preprocesses it and then transmits it to the image fusion module, the image fusion module is used to fuse the preprocessed multispectral image data to generate a fused image, the data storage module is used to store the fused image, and the display module is used to display the fused image in real time.

[0050] In a further preferred but non-limiting embodiment, the multispectral image acquisition module includes an ultraviolet image acquisition unit, an infrared image acquisition unit and a visible light image acquisition unit, and the three are integrated in the multispectral image acquisition module to simultaneously acquire ultraviolet, visible light and infrared image data of the device under test;

[0051] like Figure 1 As shown, the ultraviolet image acquisition unit is used to collect ultraviolet images, and the ultraviolet image acquisition unit includes a spectroscope, an ultraviolet filter, an ultraviolet enhancement system, a lens, and an ultraviolet sensor; when the light enters the multi-spectral image acquisition unit, the ultraviolet light is separated by the spectroscope, and the ultraviolet light enters the ultraviolet enhancement system after further filtering by the ultraviolet filter. Since the application scenario of the present invention requires a more significant ultraviolet light intensity, the ultraviolet light is enhanced by the ultraviolet enhancement system to further improve the sensitivity and accuracy of the detection, so as to better perform the detection and analysis; after being enhanced by the ultraviolet enhancement system, the ultraviolet light enters the ultraviolet sensor through the lens;

[0052] The visible light image acquisition unit is used to acquire visible light images, and the visible light image acquisition unit includes a total reflector, a visible light lens, and a visible light high-speed camera module; the visible light separated by the spectroscope of the ultraviolet image acquisition unit enters the visible light image acquisition unit, and after being reflected by the total reflector, enters the visible light high-speed camera module through the visible light lens;

[0053] The infrared image acquisition unit is used to acquire infrared light images, and the infrared image acquisition unit includes an infrared lens, a fiberscope and a detector; thermal radiation enters the detector through the infrared lens and the fiberscope.

[0054] In a further preferred but non-limiting embodiment, the infrared lens in the infrared image acquisition unit applies a pre-designed large-field-of-view aspherical perspective group zoom system, specifically comprising:

[0055] The large-field aspherical perspective zoom system consists of 11 refractive lenses, with a full field of view of 140°, an F number of 2.8, and supports an object distance between 30cm and 100cm;

[0056] The optical system is composed of eleven lenses and a bandpass filter. The first lens, the third lens, the fourth lens, the fifth lens and the second cemented lens group have negative focal power, and the second lens, the first cemented lens group, the aperture stop (STO), the tenth lens and the eleventh lens have positive focal power. The first cemented lens group is composed of the sixth lens and the seventh lens, the second lens group is composed of the sixth lens (L6) with positive focal power and the seventh lens (L7) with positive focal power, and the second cemented lens group is composed of the eighth lens (L8) with positive focal power and the ninth lens (L9) with negative focal power;

[0057] The optical system has good imaging quality, and has the characteristics of small and dense diffuse spots and high relative illumination, which meets the given system design indicators. Since optical spherical lenses have always had aberration problems, multiple lenses are often required to form a high-performance optical system. However, while multiple lenses improve image quality, they also make the system thick and have a poor appearance. Therefore, it is proposed to use aspherical lenses to reduce the system size and reduce optical distortion.

[0058] In a further preferred but non-limiting embodiment, the data preprocessing module is used to preprocess the ultraviolet light image, infrared light image and visible light image acquired by the multispectral image acquisition module;

[0059] Among them, the preprocessing operation is video decoding, such as Figure 1 As shown, the image format transmitted by the multispectral image acquisition module to the data preprocessing module is a compressed digital video signal. The data preprocessing module restores the compressed digital video signal to a visual video image through video decoding and provides readable data support for subsequent multispectral image fusion.

[0060] Each frame of the video can be regarded as an image, so the subsequent processing can be understood as image fusion.

[0061] In one embodiment, illustratively, the data preprocessing module may use an ADC (digital-to-digital converter) to convert the image data collected by the multispectral image acquisition module into a digital signal, and output it in a 10-bit or 8-bit parallel manner.

[0062] In a further preferred but non-limiting embodiment, the image fusion module comprises a multispectral image fusion model, which includes a generator, a discriminator and a quality assessment unit;

[0063] Among them, the generator is used to generate a fused image, the discriminator is used to evaluate the quality of the fused image, the quality assessment unit is used to evaluate the quality of the source image, and the evaluation result is used to adjust the loss function of the generator to achieve an adaptive fusion balance of infrared, ultraviolet and visible light features.

[0064] In one embodiment, illustratively, DDR2 SDRAM is used as a data storage module to cache the obtained image data.

[0065] In one embodiment, exemplarily, a host computer is used as a display module for displaying the fused image in real time.

[0066] In a further preferred but non-limiting embodiment, the multi-spectral fusion system based on a large field of view aspheric perspective group described in Example 1 may also include a communication module, which includes a device component supporting multiple software communication protocols and a data conversion system, specifically:

[0067] Support multiple software communication protocols. By acquiring the key information required for power equipment status detection and diagnosis, a unified detection data format specification is formulated, covering UHF, high frequency, ultrasonic, transient ground voltage, infrared and other detection types, supporting structured, semi-structured, unstructured and other data types. In order to meet the development needs of horizontally expanded status detection, a standard communication protocol that meets the detection data access requirements of the test instrument is formulated. WiFi uses the TCP protocol, Bluetooth uses the classic Bluetooth SPP protocol, and USB uses the virtual serial port CDC protocol.

[0068] The data conversion system is assigned test plans and tasks by the service platform, and generates and saves test result reports.

[0069] Figure 3 In the data conversion system, the data conversion system is mainly composed of instruments, conversion devices and mobile terminal business structures. The mobile terminal is equipped with a matching SDK, receives the test plan and tasks distributed by the service platform, and connects with the conversion device for equipment matching. With the conversion device as the medium, the equipment test results completed by the test instrument are exchanged through the conversion device to realize data content.

[0070] The matching connection of devices uses scanning technology. The specific process is as follows: During the device discovery process of Wi-Fi Direct, mobile devices that support this function will scan all wireless channels to find other devices and store the collected information. After the scan is completed, the device enters the discovery phase, at which time the device will alternately search and listen on different channels. Two devices can only communicate on the same channel. If the devices that discover each other are not connected, they can perform a connection operation to form a one-to-one topology. If a device is already connected to other devices, it can still invite other devices to join, thus forming a one-to-many topology.

[0071] In this preferred embodiment, the beneficial effects of using the communication module include combining the ultraviolet imaging modular technology with a mobile phone App, connecting to the modular imager through network interaction, Bluetooth or WiFi communication, making the monitoring of external insulation discharge of electrical equipment more convenient and efficient. The equipment operation status can be checked anytime and anywhere, and the discharge situation can be monitored in real time to ensure the safe and stable operation of the power system. At the same time, the data transmission efficiency is improved, and the data collected by the imager can be quickly transmitted to the mobile phone App, which is convenient for real-time viewing of discharge images, data and analysis results, providing strong support for fault diagnosis and equipment maintenance.

[0072] Embodiment 2, a multi-spectral fusion method based on a large-field-of-view aspheric perspective group, based on the multi-spectral fusion system based on a large-field-of-view aspheric perspective group described in Embodiment 1, the method comprises the following steps:

[0073] Step 1, collecting ultraviolet, infrared and visible light images of the device under test;

[0074] Step 2, preprocessing the ultraviolet, infrared and visible light images collected in step 1;

[0075] Step 3: Perform multispectral image fusion based on the multispectral image preprocessed in step 2 to obtain a fused image, and display and store the fused image in real time.

[0076] In a further preferred but non-limiting embodiment, in step 3, performing multispectral image fusion on the multispectral image specifically includes:

[0077] The preprocessed multispectral image is input into the multispectral image fusion model. The quality of the source image is first evaluated by the quality assessment unit, and the evaluation result is used to adjust the loss function of the generator, and then the generator generates a fused image;

[0078] The generator includes a strong attention unit, a scene texture unit and three branches; the three branches of the generator are used to extract the features of infrared, ultraviolet and visible light images respectively, obtain their feature vectors and input them into the strong attention unit; the strong attention unit performs feature enhancement based on the input feature vector, and inputs the enhanced feature map into the scene texture unit; the scene texture unit performs image fusion based on the enhanced feature map and outputs a fused image.

[0079] In a further preferred but non-limiting embodiment, the strong attention unit and scene texture unit are capable of constraining differentiated feature representations from different modalities, achieving suppression of redundant features and compactness of complementary information, retaining texture and brightness information of the target area, and enhancing the description of detail information in the global scene.

[0080] In a further preferred but non-limiting embodiment, the strong attention unit aims to capture feature representations at different levels, and retain important features in infrared and ultraviolet images by paying attention to high pixel intensity areas in the image, while suppressing those unimportant low pixel intensity areas, which helps to retain key infrared image instance features during the image fusion process and improve the quality of the fused image;

[0081] The feature vector input to the strong attention unit is Where H, W and C represent the height, width and number of channels of the feature map respectively. A set of transformed feature vectors A′ and A″ are obtained through a series of convolution operations and flattened into a two-dimensional matrix. The attention weight map M is calculated as shown in the following formula:

[0082]

[0083] Use the attention weight map M to weight the feature A′ and obtain the weighted feature map A α :

[0084] A α =M·A′

[0085] Finally, the weighted feature map A is adjusted by a learnable scale parameter β α The importance of is calculated and fused with the original feature A to obtain the output O of the strong attention unit:

[0086] O=β·A α +A

[0087] The output O of the strong attention unit is the enhanced feature map.

[0088] In a further preferred but non-limiting embodiment, the scene texture unit has the characteristics of multi-level feature extraction, feature compression, and feature connection, and can perform feature extraction in a multi-level manner to recapture and refine texture features at different levels. A 3x3 convolution operation can be used to compress the number of channels of the feature map to half of the input to reduce the feature dimension. At the same time, the compressed feature map can be connected with the feature map of the previous layer to enhance the feature representation and eliminate noise.

[0089] The image fusion performed by the scene texture unit based on the enhanced feature map can be expressed as:

[0090]

[0091] Among them, F l represents the feature map of the current level l, F l-1 Represents the feature map of the previous level, Represents the connection operation of the feature map, Conv 3×3Represents a 3x3 convolution operation.

[0092] In a further preferred but non-limiting embodiment, the quality assessment unit balances the infrared, ultraviolet and visible light information to measure the quality of the source image to learn the fusion balance point and realize adaptive fusion based on strong attention unit and scene texture unit feature extraction;

[0093] Specifically, the result of the quality assessment unit is used to adjust the loss function of the generator to achieve an adaptive fusion balance of infrared, ultraviolet and visible light features. Through the evaluation of the quality assessment unit, the proportion of source image information retained during the fusion process can be determined, thereby guiding the generator to generate a higher quality fused image. The evaluation function of the quality assessment unit is as follows:

[0094]

[0095] Among them, S(I) represents the information entropy of the input image, I vis ,I ir ,I er Represent visible light image and infrared and ultraviolet images respectively. S(I) can be expressed as follows:

[0096]

[0097] Among them, p(i) is the proportion of pixels with grayscale value i in the input image.

[0098] In a further preferred but non-limiting embodiment, the discriminator can evaluate the quality of the fused images to determine whether they contain sufficient infrared, ultraviolet and visible light image information, help balance the feature fusion of visible light and infrared and ultraviolet images, and drive the generator to produce more balanced and realistic fused images by minimizing the loss of the discriminator.

[0099] In one embodiment, illustratively, a microprocessor composed of FPGA+DSP is used as the image fusion module.

[0100] In a preferred but non-limiting embodiment, the microprocessor composed of the FPGA+DSP can be used to identify, analyze, process, and extract the position and strength information of the heating, corona, and partial discharge of the infrared and ultraviolet imaging heads from the video signals output by the digital / analog data of the visible light camera, infrared, and ultraviolet imaging head sensors, and superimpose the position and strength information of the heating, corona, and partial discharge of the infrared and ultraviolet imaging heads, and digitally transmit the final fusion graphics to the embedded system for storage, analysis, and upload to the background defect detection system through the unified communication interface;

[0101] Among them, fever refers to abnormal fever caused by abnormality or fault of electrical equipment, corona and partial discharge refer to discharge in gas medium or insulating area of ​​electrical equipment due to abnormality or fault. The position strength information corresponds to the strength information of infrared and ultraviolet spectra, which can indirectly reflect the severity of fever, corona and partial discharge of electrical equipment.

[0102] In a further preferred but non-limiting embodiment, the method described in Example 2 may also include preferred data processing steps, specifically:

[0103] The input, output and transmitted data are standardized in a unified format. The specific implementation scheme described above further illustrates the purpose, technical scheme and technical effect of the present invention in detail. It should be understood that the above is only a specific implementation scheme of the present invention and is not intended to limit the scope of the present invention. Any equivalent changes and modifications made by any technician in the field without departing from the ideas and principles of the present invention should fall within the scope of protection of the present invention.

Claims

1. A multi-spectral fusion system based on a large field of view aspheric perspective group, characterized in that: The system comprises: Multispectral image acquisition module, data preprocessing module, image fusion module, data storage module and display module; The multispectral image acquisition module includes an ultraviolet image acquisition unit, an infrared image acquisition unit and a visible light image acquisition unit, which are respectively used to obtain ultraviolet, infrared and visible light image data of the device under test, wherein the infrared image acquisition unit uses an optical system of a large-field-of-view aspherical perspective group as an infrared lens; the data preprocessing module receives the image data acquired by the multispectral image acquisition module, preprocesses it and then transmits it to the image fusion module, the image fusion module is used to fuse the preprocessed multispectral image data to generate a fused image, the data storage module is used to store the fused image, and the display module is used to display the fused image in real time.

2. A multi-spectral fusion system based on a large field of view aspheric perspective group according to claim 1, characterized in that: The multispectral image acquisition module includes an ultraviolet image acquisition unit, a visible light image acquisition unit and an infrared image acquisition unit, which are integrated in the multispectral image acquisition module to simultaneously acquire ultraviolet, visible light and infrared image data of the device under test; Among them, the ultraviolet image acquisition unit is used to collect ultraviolet light images. The ultraviolet image acquisition unit includes a spectroscope, an ultraviolet filter, an ultraviolet enhancement system, a lens and an ultraviolet sensor. When the light enters the multi-spectral image acquisition module, the ultraviolet light is separated by the spectroscope, and the ultraviolet light enters the ultraviolet enhancement system after further filtering by the ultraviolet filter. After being enhanced by the ultraviolet enhancement system, the ultraviolet light enters the ultraviolet sensor through the lens; The visible light image acquisition unit is used to acquire visible light images. The visible light image acquisition unit includes a total reflector, a visible light lens, and a visible light high-speed camera module. The visible light separated by the spectroscope of the ultraviolet image acquisition unit enters the visible light image acquisition unit, and after being reflected by the total reflector, enters the visible light high-speed camera module through the visible light lens. The infrared image acquisition unit is used to acquire infrared light images. The infrared image acquisition unit includes an infrared lens, a fiberscope and a detector. The thermal radiation passes through the infrared lens and the fiberscope in sequence and then enters the detector.

3. The multi-spectral fusion system based on a large-field-of-view aspheric perspective group according to claim 2, characterized in that: The optical system used by the infrared lens in the infrared image acquisition unit is a large-field-of-view aspherical perspective group zoom system, specifically comprising: The large-field aspherical perspective zoom system consists of 11 refractive lenses, with a full field of view of 140°, an F number of 2.8, and supports an object distance between 30cm and 100cm; The optical system comprises eleven lenses and a bandpass filter, the first lens, the third lens, the fourth lens, the fifth lens and the second cemented lens group have negative optical power, the second lens, the first cemented lens group, the aperture stop, the tenth lens and the eleventh lens have positive optical power, wherein the first cemented lens group is composed of the sixth lens and the seventh lens, the second lens group is composed of the sixth lens with positive optical power and the seventh lens with positive optical power, and the second cemented lens group is composed of the eighth lens with positive optical power and the ninth lens with negative optical power.

4. The multi-spectral fusion system based on a large-field-of-view aspheric perspective group according to claim 1, characterized in that: The data preprocessing module is used to preprocess the ultraviolet image, infrared image and visible light image collected by the multispectral image acquisition module; wherein the preprocessing operation includes video decoding, and the image format transmitted to the data preprocessing module by the multispectral image acquisition module is a compressed digital video signal. The data preprocessing module restores the compressed digital video signal to a visual video image through video decoding, and provides readable data support for subsequent multispectral image fusion.

5. The multi-spectral fusion system based on a large-field-of-view aspheric perspective group according to claim 1, characterized in that: The image fusion module includes a multispectral image fusion model, which includes a generator, a discriminator and a quality assessment unit; Among them, the generator is used to generate a fused image, the discriminator is used to evaluate the quality of the fused image, the quality assessment unit is used to evaluate the quality of the source image, and the evaluation result is used to adjust the loss function of the generator, thereby controlling the adaptive fusion balance of infrared, ultraviolet and visible light features.

6. A multi-spectral fusion method of a large-field-of-view aspheric perspective group, based on a multi-spectral fusion system based on a large-field-of-view aspheric perspective group according to any one of claims 1 to 5, characterized in that: The steps include: Step 1, collecting ultraviolet, infrared and visible light images of the device under test; Step 2, preprocessing the ultraviolet, infrared and visible light images collected in step 1; Step 3: Perform multispectral image fusion based on the multispectral image preprocessed in step 2 to obtain a fused image, and display and store the fused image in real time.

7. The multi-spectral fusion method of a large-field-of-view aspheric perspective group according to claim 6, characterized in that: In step 3, performing multispectral image fusion on the multispectral image specifically includes: The preprocessed multispectral image is input into the multispectral image fusion model. The quality of the source image is first evaluated by the quality assessment unit, and the evaluation result is used to adjust the loss function of the generator, and then the generator generates a fused image; The generator includes a strong attention unit, a scene texture unit and three branches; the three branches of the generator are used to extract the features of infrared, ultraviolet and visible light images respectively, obtain their feature vectors and input them into the strong attention unit; the strong attention unit performs feature enhancement based on the input feature vector, and inputs the enhanced feature map into the scene texture unit; the scene texture unit performs image fusion based on the enhanced feature map and outputs a fused image.

8. The multi-spectral fusion method based on a large-field-of-view aspheric perspective group according to claim 7, characterized in that: The strong attention unit performs feature enhancement based on the input feature vector, specifically including: The feature vector input to the strong attention unit is Where H, W and C represent the height, width and number of channels of the feature map respectively. A set of transformed feature vectors A′ and A″ are obtained through a series of convolution operations and flattened into a two-dimensional matrix. The attention weight map M is calculated as shown in the following formula: Use the attention weight map M to weight the feature A′ and obtain the weighted feature map A α : A α =M·A′ Finally, the weighted feature map A is adjusted by a learnable scale parameter β α The importance of is calculated and fused with the original feature A to obtain the output O of the strong attention unit: O=β·A α +A The output O of the strong attention unit is the enhanced feature map.

9. The multi-spectral fusion method based on a large-field-of-view aspheric perspective group according to claim 7, characterized in that: The scene texture unit performs image fusion based on the enhanced feature map using the formula: Among them, F l represents the feature map of the current level l, F l-1 Represents the feature map of the previous level, Represents the connection operation of the feature map, Conv 3×3 Represents a 3x3 convolution operation.

10. The multi-spectral fusion method based on a large-field-of-view aspheric perspective group according to claim 7, characterized in that: The evaluation function of the quality evaluation unit is: Among them, S(I) represents the information entropy of the input image, I vis ,I ir ,I er Represent visible light image and infrared and ultraviolet images respectively. S(I) can be expressed as follows: Among them, p(i) is the proportion of pixels with grayscale value i in the input image.

Images