A monitoring system based on dual light fusion video technology
By comparing the dual-light fusion results with the single-light channel image in real time in the monitoring system, the problem of imaging deviation during long-term use of the equipment was solved, achieving efficient, clear imaging and stable operation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG SHENGXUAN ELECTRICAL POWER TECH
- Filing Date
- 2025-09-02
- Publication Date
- 2026-07-07
AI Technical Summary
Existing dual-light fusion equipment, during long-term use, may experience issues such as lens shift or position changes causing factory-corrected parameters to become inapplicable, affecting imaging accuracy, and lacking real-time monitoring and calibration mechanisms.
By setting up visible light capture module, infrared light capture module, dual-light fusion module and anomaly calibration module in the monitoring system, the fusion result is compared with the single light channel image in real time to generate early warning reminders and ensure stable operation of the equipment.
It improves imaging accuracy and equipment stability, ensuring clear imaging in different environments, reduces computational latency and generates timely warnings, thus guaranteeing the efficient operation of the monitoring system.
Smart Images
Figure CN121078298B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of surveillance technology, specifically a surveillance system based on dual-light fusion video technology. Background Technology
[0002] Dual-light fusion is a dual-channel system combining "low-light + thermal imager," integrating visible light and infrared light channels. It simultaneously utilizes infrared and low-light technologies to image at different wavelengths, synchronously detecting the target's two-dimensional geometric space and one-dimensional spectral information. Then, it uses certain image processing algorithms to analyze and process multi-band images, fully utilizing useful information from various channels to synthesize images. The visible light channel can display the target's real-time dynamics, functioning similarly to a camera, while the infrared light channel can display temperature measurement results, showing the on-site temperature difference and temperature values in the form of thermal images. This avoids the drawbacks of blurry images taken by a single thermal imaging device or personnel on-site, requiring further on-site inspection. It provides a one-stop solution, making the work more convenient and efficient.
[0003] Currently, in the market, dual-light fusion equipment for visible light and infrared thermal imaging images used in non-civilian industries such as security and drones all have their dual-light fusion correction algorithms corrected before the equipment leaves the factory. That is, the dual-light fusion correction of visible light and infrared thermal imaging images is completed during production. However, during subsequent use, the visible light lens, thermal imaging lens structure may shift or the position of the dual-light camera itself may shift as the usage time increases. This may cause the factory correction parameters to become inapplicable, resulting in interference such as ghosting in the finished image and affecting the imaging accuracy.
[0004] To address the aforementioned technical problems, this application proposes a solution. Summary of the Invention
[0005] This invention compares the fusion result with the individual optical channel images involved in the fusion after fusion imaging to obtain the imaging difference between the fusion result and the samples involved in the fusion. The fusion effect is verified based on the imaging difference, and an early warning is generated in a timely manner when imaging deviation occurs in the fusion effect, thereby ensuring the stable operation of the equipment. This invention solves the problem that the structure of the dual-light camera changes during long-term use of dual-light fusion equipment, causing the factory correction parameters to become inapplicable and affecting the imaging quality. Therefore, it proposes a monitoring system based on dual-light fusion video technology.
[0006] The objective of this invention can be achieved through the following technical solutions:
[0007] A monitoring system based on dual-light fusion video technology includes a visible light capture module, an infrared light capture module, a dual-light fusion module, an anomaly calibration module, and a monitoring interface analysis module. The visible light capture module captures visible light images through a visible light camera to generate visible light images.
[0008] The infrared light capture module captures infrared light images through an infrared camera, preprocesses the captured infrared light images using environmental parameters to improve the contrast of the infrared light images, and generates infrared images.
[0009] The dual-light fusion module decomposes the visible light image and the infrared image into frames, and then fuses the decomposed images one by one through time series and algorithm to obtain the corrected image.
[0010] After acquiring the corrected image, the monitoring interface analysis module imports the corrected image into the cloud for storage via the interface.
[0011] The anomaly calibration module can acquire the corrected image and compare and analyze it with the infrared image and the visible light image respectively to obtain the edge information and image size information of the corrected image. The anomaly calibration module obtains the edge repair degree based on the edge information and the fusion symmetry degree based on the image size information.
[0012] The anomaly calibration module performs a comprehensive analysis based on the degree of edge repair and the degree of fusion symmetry to obtain a fusion effect rating, and feeds the fusion effect rating back to the dual-light fusion module;
[0013] After obtaining the fusion effect rating, the dual-light fusion module provides feedback and reminders.
[0014] In a preferred embodiment of the present invention, when the visible light capture module acquires a visible light signal, it collects the signal through a visible light camera, automatically analyzes the brightness of the acquired visible light image, and obtains a fusion signal or an output signal based on whether the visible light brightness reaches a preset setting.
[0015] In a preferred embodiment of the present invention, the method by which the visible light capture module performs image brightness analysis on a visible light image is as follows:
[0016] The visible light capture module performs matrix sampling on the visible light image and performs grayscale conversion on each sampling point to obtain multiple single-channel grayscale images. The grayscale values of the pixels in each single-channel grayscale image are averaged, and the average value is used as the brightness value of the single-channel grayscale image. Then, the brightness values of each sample in the matrix sampling are averaged to obtain the overall brightness of the visible light image.
[0017] In a preferred embodiment of the present invention, the environmental parameter collected by the infrared light capture module is the ambient air temperature. After collecting the environmental parameter, the infrared light capture module records the collected ambient air temperature as the reference temperature. The infrared light capture module obtains the set amplitude threshold η and generates two sets of endpoint ratios through (1+η) and (1-η). The reference temperature is multiplied by the two sets of endpoint ratios respectively to obtain two sets of temperature endpoints. The temperature between the two sets of temperature endpoints is taken as the high contrast temperature, and the temperature outside the two sets of endpoint temperatures is taken as the normal temperature.
[0018] In a preferred embodiment of the present invention, when the infrared light capture module preprocesses the infrared image, it obtains the temperature values of different regions in the infrared image through a pre-calibrated temperature mapping relationship, marks the regions with high contrast temperatures, linearly stretches the gray values of the marked regions to increase the gray value range, and compresses the gray values of those not belonging to the marked regions to reduce the gray value range.
[0019] After changing the grayscale value, the infrared light capture module optimizes color mapping based on temperature, improves the contrast of areas with temperatures similar to the ambient air temperature, and enhances image clarity by utilizing high contrast. In areas with large temperature differences between the infrared light and the ambient air temperature, the image clarity is maintained even when the grayscale value range is compressed by utilizing the temperature difference of the infrared light itself.
[0020] In a preferred embodiment of the present invention, when the dual-light fusion module performs frame decomposition on the visible light image and the infrared image, it converts the video image into an independent single-frame static image and obtains the specific time information of the single-frame static image. Then, it fuses the single-frame static images of the infrared light image and the single-frame static images of the visible light image with the same time information.
[0021] The dual-light fusion module merges the fused static images to obtain a corrected image, and then merges them into a video image based on time information.
[0022] In a preferred embodiment of the present invention, the method by which the dual-light fusion module merges static images is as follows:
[0023] The visible light and infrared light images are subjected to perspective transformation based on camera parameters to eliminate image deviation. An attention mechanism is introduced to dynamically weight the infrared image, and features are extracted by an encoder. The fusion layer integrates the feature information to complete the fusion of the infrared and visible light images.
[0024] As a preferred embodiment of the present invention, the method for the abnormal calibration module to obtain the edge repair degree is as follows: the object edges of the corrected image, the visible light image, and the infrared light image are identified to obtain the number of edges in the three sets of images, and the number of edges is compared. If the number of edges in the corrected image is less than the number of edges in the visible light image or the infrared light image, the edge repair degree is determined to be insufficient; otherwise, the edge repair degree is determined to be up to standard.
[0025] The method for the abnormal calibration module to obtain image frame information is as follows: extract feature points in the corrected image, and extract the same feature points through the visible light image and the infrared light image. Locate multiple feature points in the corresponding images to obtain feature point coordinates. Compare the feature point coordinates in different images and count the number of feature points with the same coordinates. If the proportion of the number of feature points with the same coordinates in the total number of feature points is greater than a set value, the image frame is determined to be normal; otherwise, the image frame is determined to be abnormal.
[0026] The anomaly calibration module records image aberrations and normal image aberrations as the degree of fusion symmetry, and records insufficient repair and satisfactory repair as the degree of edge repair.
[0027] In a preferred embodiment of the present invention, the anomaly calibration module generates a fusion failure signal when it simultaneously detects both image format anomaly and insufficient repair level, generates a fusion normal signal when it simultaneously detects both image format anomaly and adequate repair level, and generates a fusion optimization signal otherwise.
[0028] Compared with the prior art, the beneficial effects of the present invention are:
[0029] 1. In this invention, during the operation of the video surveillance system, the operating effects of the visible light channel and the infrared light channel are collected synchronously, and the collected infrared light images and visible light images are preprocessed separately to improve the sample quality. Frame decomposition and image fusion are performed based on the time axis to obtain a dual-light fused image, thereby obtaining clear imaging in various environments and improving the monitoring effect.
[0030] 2. In this invention, when processing visible light, the imaging quality of visible light is judged to make a scientific decision on the fusion of infrared light, thereby reducing the local computation load and imaging delay. Infrared light is processed in regions, and grayscale stretching is performed according to different temperature ranges to improve the contrast of the approximate temperature range. This ensures that each temperature range of infrared imaging can obtain clear imaging edges, which improves the sample quality during dual-light fusion and guarantees the individual imaging clarity of a single light channel.
[0031] 3. In this invention, after fusion imaging, the fusion result is compared with the single optical channel image participating in the fusion, thereby obtaining the imaging difference between the fusion result and the sample participating in the fusion. The fusion effect is verified based on the imaging difference, and when the fusion effect shows imaging deviation, an early warning reminder is generated in a timely manner, thereby ensuring the stable operation of the equipment. Attached Figure Description
[0032] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.
[0033] Figure 1 This is a system block diagram of the present invention;
[0034] Figure 2 This is a system flowchart of the present invention. Detailed Implementation
[0035] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.
[0036] Example 1: Please refer to Figure 1 - Figure 2 As shown, a monitoring system based on dual-light fusion video technology includes a visible light capture module, an infrared light capture module, a dual-light fusion module, an anomaly calibration module, and a monitoring interface analysis module. The visible light capture module captures visible light images through a visible light camera to generate visible light images.
[0037] When acquiring visible signals, the visible light capture module uses a visible light camera to collect data and automatically analyzes the brightness of the captured visible light image. The method used by the visible light capture module to analyze the brightness of the visible light image is as follows:
[0038] The visible light capture module performs matrix sampling on the visible light image and performs grayscale conversion on each sampling point to obtain multiple single-channel grayscale images. The grayscale values of the pixels in each single-channel grayscale image are averaged, and the average value is used as the brightness value of the single-channel grayscale image. Then, the brightness values of each sample in the matrix sampling are averaged to obtain the overall brightness of the visible light image.
[0039] The visible light capture module judges the brightness of the visible light image. If the brightness of the visible light image reaches the preset brightness, it generates an image output signal. If the brightness of the visible light image does not reach the preset brightness, it generates an image fusion signal. After the visible light capture module generates an image output signal, it indicates that the image brightness is sufficient and the image details can be observed intuitively and effectively. In this case, the image is directly output to the monitoring interface analysis module. After generating an image fusion signal, it indicates that the image brightness is insufficient and the image details cannot be observed intuitively and effectively. It is necessary to fuse the infrared light image to improve the image clarity and recognition ability.
[0040] The infrared light capture module acquires infrared light images through an infrared camera, preprocesses the acquired infrared light images using environmental parameters to improve the contrast of the infrared light images, and generates infrared images. The environmental parameter acquired by the infrared light capture module is the ambient air temperature. After acquiring the environmental parameter, the infrared light capture module records the acquired ambient air temperature as the reference temperature. The infrared light capture module obtains the set amplitude threshold η and generates two sets of endpoint ratios through (1+η) and (1-η). The reference temperature is multiplied by the two sets of endpoint ratios respectively to obtain two sets of temperature endpoints. The temperature between the two sets of temperature endpoints is taken as the high contrast temperature, and the temperature outside the two sets of endpoint temperatures is taken as the normal temperature.
[0041] When the infrared light capture module preprocesses the infrared image, it obtains the temperature values of different areas in the infrared image through a pre-calibrated temperature mapping relationship, marks the areas with high contrast temperatures, linearly stretches the gray values of the marked areas to increase the gray value range, and compresses the gray values of areas that do not belong to the marked areas to reduce the gray value range.
[0042] After changing the grayscale value, the infrared light capture module optimizes color mapping based on temperature, improves the contrast of areas with temperatures similar to the ambient air temperature, and enhances image clarity by utilizing high contrast. In areas with large temperature differences between the infrared light and the ambient air temperature, the image clarity is maintained even when the grayscale value range is compressed by utilizing the temperature difference of the infrared light itself.
[0043] The dual-light fusion module decomposes the visible light image and the infrared image into frames. When the dual-light fusion module decomposes the visible light image and the infrared image into frames, it converts the video image into an independent single-frame static image and obtains the specific time information of the single-frame static image. It then fuses the single-frame static images of the infrared image and the single-frame static images of the visible light image with the same time information to obtain the corrected image.
[0044] The dual-light fusion module merges the fused static images to obtain a corrected image, and then merges them into a video image based on time information.
[0045] The method by which the dual-light fusion module merges static images is as follows:
[0046] The visible light and infrared light images are subjected to perspective transformation based on camera parameters to eliminate image deviation. An attention mechanism is introduced to dynamically weight the infrared image, and features are extracted by an encoder. The fusion layer integrates the feature information to complete the fusion of the infrared and visible light images.
[0047] After acquiring the corrected image, the monitoring interface analysis module imports the corrected image into the cloud for storage via the interface, making it convenient for managers to view or retrieve it.
[0048] Example 2: Please refer to Figure 1 - Figure 2 As shown, the anomaly calibration module can acquire the corrected image and compare and analyze it with the infrared image and the visible light image respectively to obtain the edge information and image size information of the corrected image. The anomaly calibration module obtains the edge repair degree based on the edge information and the fusion symmetry degree based on the image size information.
[0049] The abnormal calibration module obtains the edge repair level by identifying the object edges in the corrected image, visible light image, and infrared light image, obtaining the number of edges in the three sets of images. The number of edges is represented by the line length of the edges. The edge numbers are compared. If the number of edges in the corrected image is less than the number of edges in the visible light image or the infrared light image, the edge repair level is determined to be insufficient. If the number of edges in the corrected image is greater than or equal to the number of edges in the visible light image and the infrared light image, the edge repair level is determined to be up to standard.
[0050] The abnormal calibration module obtains image frame information by extracting feature points from the corrected image and extracting the same feature points from the visible light image and the infrared light image. Multiple feature points are located in the corresponding images to obtain the feature point coordinates (Xi, Yi), where i is the feature point number. The feature point coordinates in different images are compared, and the number of feature points with the same coordinates is counted. If the proportion of the number of feature points with the same coordinates in the total number of feature points is greater than a set value, the image frame is determined to be normal. If the proportion of the number of feature points with the same coordinates in the total number of feature points is not greater than a set value, the image frame is determined to be abnormal.
[0051] The anomaly calibration module records image aberration and normal image aberration as the degree of blending symmetry, and records insufficient repair and satisfactory repair as the degree of edge repair.
[0052] The anomaly calibration module performs a comprehensive analysis based on the degree of edge repair and the degree of fusion symmetry. When the anomaly calibration module simultaneously detects both image anomaly and insufficient repair, it generates a fusion failure signal. When it simultaneously detects both image anomaly and adequate repair, it generates a fusion normal signal. Otherwise, it generates a fusion optimization signal. The fusion failure signal, fusion normal signal, or fusion optimization signal are used as the fusion effect rating result, and the fusion effect rating is fed back to the dual-light fusion module.
[0053] After obtaining the fusion effect rating, the dual-light fusion module provides feedback and reminders, enabling managers to perform maintenance and repairs on the equipment after receiving fusion optimization signals or fusion failure signals. This allows them to adjust the dual-light fusion parameters of the equipment and ensure the imaging quality of the dual-light fusion monitoring system.
[0054] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to any specific implementation. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.
Claims
1. A monitoring system based on dual-light fusion video technology, characterized in that, It includes a visible light capture module, an infrared light capture module, a dual-light fusion module, an anomaly calibration module, and a monitoring interface analysis module. The visible light capture module captures visible light images through a visible light camera to generate visible light images. The infrared light capture module captures infrared light images through an infrared camera, preprocesses the captured infrared light images using environmental parameters to improve the contrast of the infrared light images, and generates infrared images. The dual-light fusion module decomposes the visible light image and the infrared image into frames, and then fuses the decomposed images one by one through time series and algorithm to obtain the corrected image. After acquiring the corrected image, the monitoring interface analysis module imports the corrected image into the cloud for storage via the interface. The anomaly calibration module can acquire the corrected image and compare and analyze it with the infrared image and the visible light image respectively to obtain the edge information and image size information of the corrected image. The anomaly calibration module obtains the edge repair degree based on the edge information and the fusion symmetry degree based on the image size information. The anomaly calibration module performs a comprehensive analysis based on the degree of edge repair and the degree of fusion symmetry to obtain a fusion effect rating, and feeds the fusion effect rating back to the dual-light fusion module; After obtaining the fusion effect rating, the dual-light fusion module provides feedback and reminders. When the dual-light fusion module decomposes the visible light image and the infrared image into frames, it transforms the video image into an independent single-frame static image and obtains the specific time information of the single-frame static image. Then, it fuses the infrared image single-frame static image and the visible light image single-frame static image with the same time information. The dual-light fusion module merges the fused static images to obtain a corrected image, and then merges them into a video image based on time information. The method for the anomaly calibration module to obtain the edge repair degree is as follows: the object edges in the corrected image, visible light image, and infrared image are identified to obtain the number of edges in the three sets of images, and the number of edges is compared. If the number of edges in the corrected image is less than the number of edges in the visible light image or the infrared image, the edge repair degree is determined to be insufficient; otherwise, the edge repair degree is determined to be up to standard. The method for the abnormal calibration module to obtain image frame information is as follows: extract feature points in the corrected image, and extract the same feature points through the visible light image and infrared image, locate multiple feature points in the corresponding images to obtain feature point coordinates, compare the feature point coordinates in different images, count the number of feature points with the same coordinates, if the proportion of the number of feature points with the same coordinates in the total number of feature points is greater than a set value, then the image frame is determined to be normal; otherwise, the image frame is determined to be abnormal. The anomaly calibration module records image aberration and normal image aberration as the degree of fusion symmetry, and records insufficient repair and qualified repair as the degree of edge repair. When the anomaly calibration module simultaneously detects both an abnormal image size and insufficient repair level, it generates a fusion failure signal; when it simultaneously detects both a normal image size and adequate repair level, it generates a fusion normal signal; otherwise, it generates a fusion optimization signal.
2. The monitoring system based on dual-light fusion video technology according to claim 1, characterized in that, When acquiring visible signals, the visible light capture module uses a visible light camera to collect the data and automatically analyzes the brightness of the captured visible light image. Based on whether the visible light brightness reaches a preset setting, it obtains a fused signal or an output signal.
3. A monitoring system based on dual-light fusion video technology according to claim 2, characterized in that, The method by which the visible light capture module performs image brightness analysis on a visible light image is as follows: The visible light capture module performs matrix sampling on the visible light image and performs grayscale conversion on each sampling point to obtain multiple single-channel grayscale images. The grayscale values of the pixels in each single-channel grayscale image are averaged, and the average value is used as the brightness value of the single-channel grayscale image. Then, the brightness values of each sample in the matrix sampling are averaged to obtain the overall brightness of the visible light image.
4. A monitoring system based on dual-light fusion video technology according to claim 1, characterized in that, The environmental parameter collected by the infrared light capture module is the ambient air temperature. After collecting the environmental parameter, the infrared light capture module records the collected ambient air temperature as the reference temperature. The infrared light capture module obtains the set amplitude threshold η and generates two sets of endpoint ratios through (1+η) and (1-η). The reference temperature is multiplied by the two sets of endpoint ratios respectively to obtain two sets of temperature endpoints. The temperature between the two sets of temperature endpoints is taken as the high contrast temperature, and the temperature outside the two sets of endpoint temperatures is taken as the normal temperature.
5. A monitoring system based on dual-light fusion video technology according to claim 1, characterized in that, When the infrared light capture module preprocesses the infrared light image, it obtains the temperature values of different areas in the infrared light image through a pre-calibrated temperature mapping relationship, marks the areas with high contrast temperatures, linearly stretches the gray values of the marked areas to increase the gray value range, and compresses the gray values of areas that do not belong to the marked areas to reduce the gray value range. After changing the grayscale value, the infrared light capture module optimizes color mapping based on temperature.
6. A monitoring system based on dual-light fusion video technology according to claim 1, characterized in that, The method by which the dual-light fusion module fuses static images is as follows: The visible light and infrared images are subjected to perspective transformation based on camera parameters to eliminate image deviation. An attention mechanism is introduced to dynamically weight the infrared image, and features are extracted by an encoder. The fusion layer integrates the feature information to complete the fusion of the infrared and visible light images.