An image fusion method, device, equipment and storage medium

Abstract

Embodiments of the present application provide an image fusion method, device and equipment and storage medium, relating to the technical field of image processing, the method comprising: obtaining a target video frame from monitoring videos of multiple monitoring devices, determining a monitoring video overlap area of the multiple monitoring devices based on the target video frame; extracting a gray overlap image and a chroma overlap image corresponding to each monitoring device from the target video frame according to the monitoring video overlap area; performing pixel transformation fusion on the gray overlap image of each monitoring device to obtain a fused gray image; performing chroma calibration and color noise elimination on the chroma overlap image corresponding to each monitoring device to obtain a calibrated image, and fusing each calibrated image to obtain a fused chroma image; fusing the fused gray image and the fused chroma image to obtain a target fused image, and generating a target monitoring image based on the target fused image and the target video frame. The present application can increase the monitoring field of view and ensure the definition of the monitoring video.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to an image fusion method, apparatus, device, and storage medium. Background Technology

[0002] With the explosive growth of the surveillance industry, various scenarios and backend algorithms are demanding increasingly higher field of view for images, such as using a wide-angle fixed-focus lens combined with a zoom pan-tilt unit to achieve real-time detection, capture, or tracking. However, traditional wide-angle lenses suffer from severe distortion and cannot meet the needs of a wider field of view. Therefore, how to achieve clear surveillance with a large field of view has become an urgent problem to be solved. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide an image fusion method, apparatus, device and storage medium to solve the problem of unclear monitoring under a large field of view.

[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides an image fusion method, comprising: Obtain a target video frame from the monitoring videos of multiple monitoring devices, and determine the overlapping area of ​​the monitoring videos of the multiple monitoring devices based on the target video frame; Extract the grayscale image and chroma image corresponding to each monitoring device from the target video frame; extract the corresponding grayscale overlapping image from the grayscale image of each monitoring device according to the overlapping area of ​​the monitoring video; and extract the corresponding chroma overlapping image from the chroma image of each monitoring device. The grayscale images of each of the monitoring devices that overlap are transformed and fused to obtain a fused grayscale image. A calibrated image is obtained by performing chromaticity calibration and color noise reduction on the chromaticity coincidence image corresponding to each monitoring device, and then the calibrated images are fused to obtain a fused chromaticity image. The fused grayscale image and the fused chroma image are fused to obtain a target fused image, and a target monitoring image is generated based on the target fused image and the target video frame.

[0005] In an optional implementation, the monitoring equipment includes a first monitoring device and a second monitoring device, and the step of determining the overlapping area of ​​the monitoring videos of the multiple monitoring devices based on the target video frame includes: The monitoring video range of the first monitoring device is determined based on the first target video frame of the first monitoring device. The monitoring video range of the second monitoring device is determined based on the second target video frame of the second monitoring device; the first target video frame and the second target video frame are video frames at the same time. The overlapping area of ​​the monitoring videos is determined based on the monitoring video range of the first monitoring device and the monitoring video range of the second monitoring device.

[0006] In an optional implementation, the step of generating a target surveillance image based on the target fused image and the target video frame includes: An initial monitoring image is generated based on the first target video frame and the second target video frame; The target monitoring image is obtained by replacing the overlapping area of ​​the monitoring video in the initial monitoring image with the target fused image.

[0007] In an optional implementation, the step of performing pixel transformation and fusion on the grayscale overlapping images of each of the monitoring devices to obtain a fused grayscale image includes: Calculate the weight of each pixel in the grayscale overlay image of each of the monitoring devices; The transformed pixel value of each pixel is calculated based on the weight of each pixel and the original pixel value; A fused grayscale value is calculated based on the transformed pixel value and the original pixel value, and the fused grayscale image is generated based on the fused grayscale value.

[0008] In an optional implementation, the step of performing chromaticity calibration and noise reduction on the chromaticity coincidence image corresponding to each of the monitoring devices to obtain a calibrated image includes: A reference image is selected from multiple chromaticity-matched images corresponding to multiple monitoring devices; Calculate the mean value of the reference U component corresponding to the reference image and the mean value of the U component of other chromaticity-matching images; the other chromaticity-matching images are any chromaticity-matching images other than the reference image; A correction value is calculated based on the mean of the reference U component and the mean of the U component of other chromaticity-overlapping images. A chromaticity calibration map is obtained by performing chromaticity calibration on the chromaticity-overlapping images based on the correction value. The calibration image is obtained by performing color noise removal on the chromaticity calibration map.

[0009] In an optional implementation, the step of performing color noise removal on the chromaticity calibration map to obtain the calibration image includes: Obtain the U-component gradient field of each chromaticity calibration image, and generate a target gradient field based on the U-component gradient field and the activation function; The target gradient field is used to replace the U component gradient field in the chromaticity calibration map to generate the calibration U component corresponding to the chromaticity calibration map; The calibration image is generated based on the calibration U component.

[0010] In an optional implementation, the step of generating the calibration image based on the calibration U component includes: A fused image is generated based on the calibrated U component and the weight of each pixel; The calibrated image is obtained by performing local mean calibration on the fused image.

[0011] In a second aspect, the present invention provides an image fusion apparatus, comprising: The video acquisition module is used to acquire target video frames from the monitoring videos of multiple monitoring devices, and determine the overlapping area of ​​the monitoring videos of the multiple monitoring devices based on the target video frames; The image extraction module is used to extract the grayscale image and chroma image corresponding to each monitoring device from the target video frame, extract the corresponding grayscale overlapping image from the grayscale image of each monitoring device according to the overlapping area of ​​the monitoring video, and extract the corresponding chroma overlapping image from the chroma image of each monitoring device. The pixel transformation module is used to perform pixel transformation and fusion on the grayscale overlapping images of each of the monitoring devices to obtain a fused grayscale image. The color noise reduction module is used to perform color calibration and color noise reduction on the color overlap image corresponding to each of the monitoring devices to obtain a calibration image, and to fuse each calibration image to obtain a fused color image; The image fusion module is used to fuse the fused grayscale image and the fused chroma image to obtain a target fused image, and to generate a target monitoring image based on the target fused image and the target video frame.

[0012] Thirdly, the present invention provides an electronic device including a processor and a memory, the memory storing machine-executable instructions executable by the processor, the processor executing the machine-executable instructions to implement the image fusion method described in the first aspect.

[0013] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image fusion method described in the first aspect.

[0014] This invention provides an image fusion method, apparatus, device, and storage medium that acquires target video frames from the surveillance videos of multiple monitoring devices. Then, based on the target video frames, it determines the overlapping areas of the surveillance videos from each monitoring device. Finally, it fuses the grayscale and chroma images corresponding to the overlapping areas of each monitoring device's video feed, thereby merging the surveillance images from multiple devices into a single image. This solves the problem of insufficient field of view in existing market solutions for surveillance cameras, while avoiding the issues of severe image distortion caused by wide-angle lenses that cannot be corrected or result in loss of field of view after correction.

[0015] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A block diagram of an electronic device provided by an embodiment of the present invention is shown; Figure 2 A flowchart illustrating an image fusion method provided by an embodiment of the present invention is shown; Figure 3 This illustration shows a surveillance video diagram provided by an embodiment of the present invention; Figure 4 A functional block diagram of an image fusion device provided by an embodiment of the present invention is shown.

[0018] icon: 100 - Electronic device; 110 - Memory; 120 - Processor; 130 - Communication module; 400 - Image fusion device; 410 - Video acquisition module; 420 - Image extraction module; 430 - Pixel transformation module; 440 - Color noise reduction module; 450 - Image fusion module. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0020] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0021] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0022] Please refer to Figure 1 , Figure 1 This is a block diagram of an electronic device 100 provided in this embodiment. The electronic device 100 includes a memory 110, a processor 120, and a communication module 130. The memory 110, processor 120, and communication module 130 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0023] The memory 110 is used to store programs or data. The memory 110 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0024] The processor 120 is used to read / write data or programs stored in the memory 110 and to perform corresponding functions.

[0025] The communication module 130 is used to establish a communication connection between the electronic device 100 and other communication terminals through the network, and to send and receive data through the network.

[0026] It should be understood that, Figure 1 The structure shown is only a schematic diagram of the electronic device 100. The electronic device 100 may also include components that are larger than... Figure 1The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0027] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating an image fusion method provided in this embodiment. The method includes: S210. Obtain a target video frame from the monitoring videos of multiple monitoring devices, and determine the overlapping area of ​​the monitoring videos of the multiple monitoring devices based on the target video frame.

[0028] Please refer to Figure 3 , Figure 3 This is a schematic diagram of a surveillance video provided in this embodiment.

[0029] The number of monitoring devices should be at least two. The two video feeds corresponding to the two monitoring devices are usually on the same horizontal plane and there is a certain angle between them. When the number of monitoring devices is greater than two, each pair of adjacent monitoring devices can be used as a group of monitoring devices to ensure that each group of monitoring devices has an overlapping monitoring area in order to achieve subsequent image fusion.

[0030] like Figure 3 As shown in the figure, the gray area represents the video overlap area. The overlap area between the two videos is 10% of the video content. Here, w is the video width, h is the video height, and a is the horizontal width of the overlapping part.

[0031] The target video frame can be a video image captured by two monitoring devices at the same time. Since a single monitoring device cannot achieve a wide field of view, and two or more monitoring devices capture images separately during monitoring, directly stitching together the monitoring images of multiple monitoring devices may lead to problems such as video overlap, brightness, and pixel inconsistency, resulting in a less clear or realistic video after stitching.

[0032] Therefore, image fusion algorithms can be used to achieve unified output of surveillance videos from multiple monitoring devices. First, images captured by multiple monitoring devices at the same time can be acquired, i.e., target video frames. Based on these target video frames, the overlapping areas of the surveillance videos from the multiple monitoring devices can be determined. Figure 3 The gray area shown indicates that the number of monitoring devices can be set to two, and the number of monitoring devices can also be increased according to the actual situation.

[0033] S220. Extract the grayscale image and chroma image corresponding to each monitoring device from the target video frame. Extract the corresponding grayscale overlapping image from the grayscale image of each monitoring device according to the overlapping area of ​​the monitoring video. Extract the corresponding chroma overlapping image from the chroma image of each monitoring device.

[0034] First, the grayscale and chroma maps of each monitoring device are determined based on the target video frame. Both the grayscale and chroma maps are extracted from the original monitoring images of the monitoring devices. Then, based on the overlapping areas of the monitoring videos, the grayscale and chroma maps corresponding to each monitoring device are extracted or cropped, retaining only the images corresponding to the overlapping areas of the monitoring videos, which are defined as grayscale overlapping images and chroma overlapping images, respectively.

[0035] S230. Perform pixel transformation and fusion on the grayscale overlapping images of each of the monitoring devices to obtain a fused grayscale image.

[0036] To achieve seamless fusion and transition of monitoring images from different monitoring devices, similarity transformation fusion can be performed on the grayscale overlapping images from different monitoring devices. For example, a linear pixel weighted transformation algorithm can be used to fuse the grayscale overlapping images from two monitoring devices. The principle of this algorithm is to calculate the weight of each pixel based on the distance from the image to the edge, then multiply the pixel values ​​of the original image by the corresponding pixel weights to obtain new pixel value images, and finally add the two images together to obtain the grayscale image of the fused part.

[0037] S240. Perform colorimetric calibration and color noise reduction on the colorimetric overlap image corresponding to each of the monitoring devices to obtain a calibration image, and fuse each calibration image to obtain a fused colorimetric image.

[0038] Since color processing is relatively complex, simply using a linear pixel weighted transformation algorithm for fusion can lead to color noise. Therefore, this embodiment mainly uses the U-component fusion algorithm to calibrate the speed overlap images of different monitoring devices, and then fuses the calibrated images to obtain a fused chromaticity image.

[0039] S250. The fused grayscale image and the fused chroma image are fused to obtain a target fused image, and a target monitoring image is generated based on the target fused image and the target video frame.

[0040] The target fused image is obtained by fusing grayscale and chroma images from different monitoring devices. The overlapping areas of the monitoring videos in the target video frames of different monitoring devices are then replaced with the target fused image to obtain the target monitoring video image.

[0041] This embodiment acquires target video frames from the monitoring videos of multiple monitoring devices. Then, based on the target video frames, it determines the overlapping areas of the monitoring videos from each device. Finally, it fuses the grayscale and chroma images corresponding to the overlapping areas of each monitoring device's video feed, thereby combining the monitoring images from multiple devices into a single image. This solves the problem of insufficient field of view in existing market solutions for monitoring cameras, while avoiding the issues of severe image distortion caused by wide-angle lenses that cannot be corrected or result in a loss of field of view after correction.

[0042] In one embodiment, the monitoring equipment includes a first monitoring device and a second monitoring device, and the step of determining the overlapping area of ​​the monitoring videos of the multiple monitoring devices based on the target video frame includes: The monitoring video range of the first monitoring device is determined based on the first target video frame of the first monitoring device. The monitoring video range of the second monitoring device is determined based on the second target video frame of the second monitoring device; the first target video frame and the second target video frame are video frames at the same time. The overlapping area of ​​the monitoring videos is determined based on the monitoring video range of the first monitoring device and the monitoring video range of the second monitoring device.

[0043] like Figure 3 As shown, the monitoring video range of both the first and second monitoring devices is w. Then, based on the monitoring video range and the labeled data or reference objects in the video, the size of the overlapping area of ​​the video is determined.

[0044] In one implementation, the step of generating a target surveillance image based on the target fused image and the target video frame includes: An initial monitoring image is generated based on the first target video frame and the second target video frame; The target monitoring image is obtained by replacing the overlapping area of ​​the monitoring video in the initial monitoring image with the target fused image.

[0045] Since the target fusion image is formed by fusing images corresponding to overlapping areas of the video, after obtaining the target fusion image, the overlapping areas of the first target video frame and the second target video frame can be replaced with the target fusion image, thereby realizing the fusion of the first target video frame and the second target video frame. This achieves unified output of the monitoring images of the first monitoring device and the second monitoring device, improving the monitoring range and the clarity of the monitoring images.

[0046] In one embodiment, the step of performing pixel transformation and fusion on the grayscale overlapping images of each of the monitoring devices to obtain a fused grayscale image includes: Calculate the weight of each pixel in the grayscale overlay image of each of the monitoring devices; The transformed pixel value of each pixel is calculated based on the weight of each pixel and the original pixel value; A fused grayscale value is calculated based on the transformed pixel value and the original pixel value, and the fused grayscale image is generated based on the fused grayscale value.

[0047] First, calculate the weight of each pixel: The weight of the j-th pixel in the i-th row is: (1) Where W is the calculated weight, n is the total number of pixels, and all pixel coordinates are counted starting from 1.

[0048] Then, the new pixel values ​​for each image are calculated.

[0049] The value of the j-th pixel in the i-th row is: (2) in, Let j be the pixel value of the i-th row and j-th pixel in the original image. Let be the newly obtained pixel value of the j-th pixel in the i-th row.

[0050] Finally, calculate the final merged grayscale value: The value of the j-th pixel in the i-th row is (3) in Here, x represents the merged pixel value, and x is the image number, which can take values ​​from [1, 2]. If N frames are used for fusion, then the value will be [1, N]. The value of the j-th pixel in the i-th row of the first image. This is the value of the j-th pixel in the i-th row of the second image.

[0051] In formula (1), j takes the value [1, a+1] and n takes the value a+1.

[0052] This embodiment achieves a smooth transition of the boundaries of the monitoring images from different monitoring devices by fusing grayscale overlapping images corresponding to different monitoring devices, thereby improving the clarity and realism of the fused monitoring image.

[0053] In one embodiment, the step of performing chromaticity calibration and color noise reduction on the chromaticity coincidence image corresponding to each of the monitoring devices to obtain a calibrated image includes: A reference image is selected from multiple chromaticity-matched images corresponding to multiple monitoring devices; Calculate the mean value of the reference U component corresponding to the reference image and the mean value of the U component of other chromaticity-matching images; the other chromaticity-matching images are any chromaticity-matching images other than the reference image; A correction value is calculated based on the mean of the reference U component and the mean of the U component of other chromaticity-overlapping images. A chromaticity calibration map is obtained by performing chromaticity calibration on the chromaticity-overlapping images based on the correction value. The calibration image is obtained by performing color noise removal on the chromaticity calibration map.

[0054] First, the chromaticity-overlapping images of both the first and second monitoring devices can be divided into multiple small blocks, such as 8×8 blocks, with the number dynamically adjusted according to the resolution. Then, using the chromaticity-overlapping image corresponding to the first monitoring device as the reference image, U-component calibration is performed on the chromaticity-overlapping image corresponding to the second monitoring device. The chromaticity-overlapping image corresponding to the first monitoring device is defined as the first image, and the chromaticity-overlapping image corresponding to the second monitoring device is defined as the second image.

[0055] First, calculate the mean value of the U component of each small patch in the first image. The mean of the U component of each small patch in the second image .

[0056] The correction value is calculated based on the mean value of the U component. The formula is as follows: (4) (5) Where u is the actual value of u in the second image, The latest u-value after correction of the second image. The latest u value after correction. W is the weight value obtained in equation (1).

[0057] In one embodiment, the step of performing color noise removal on the chromaticity calibration map to obtain the calibration image includes: Obtain the U-component gradient field of each chromaticity calibration image, and generate a target gradient field based on the U-component gradient field and the activation function; The target gradient field is used to replace the U component gradient field in the chromaticity calibration map to generate the calibration U component corresponding to the chromaticity calibration map; The calibration image is generated based on the calibration U component.

[0058] Calculate the U-component gradient fields of the first and second images respectively. , Because the changes between image pixels are not continuous, the gradient field cannot be calculated using mathematical formulas that involve differentiation. Instead, the rate of change of chroma values ​​in the horizontal and vertical directions can be calculated sequentially from the values ​​of adjacent pixels in the overlapping parts of the image. A smaller gradient value indicates smoother chroma and less color noise.

[0059] The activation function calculates a final output value for two images: (6) use replace , The inverse operation yields the two latest images. value.

[0060] The formula for calculating the fused image using the weight value W obtained in equation (1) is as follows: (7) In one embodiment, the step of generating the calibration image based on the calibration U component includes: A fused image is generated based on the calibrated U component and the weight of each pixel; The calibrated image is obtained by performing local mean calibration on the fused image.

[0061] The obtained fused U component image is subjected to local mean calibration, that is, the image is divided into 4×4 blocks and the mean is calculated for each block. If the mean of a block differs from the mean of the surrounding blocks by more than 5, it is considered noise and replaced with the median of the mean of the surrounding blocks to eliminate local color noise caused by weighting.

[0062] Finally, the fused Y and UV images are written into memory for subsequent processing. The size of the fused image is 2×w–a in width and h in height.

[0063] To perform the corresponding steps in the above embodiments and various possible methods, an implementation of an image fusion apparatus is given below. Please refer to [link / reference needed]. Figure 4 , Figure 4 This is a functional block diagram of an image fusion device provided in an embodiment of the present invention. It should be noted that the basic principle and technical effects of the image fusion device provided in this embodiment are the same as those in the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments. The image fusion device 400 includes: The video acquisition module 410 is used to acquire a target video frame from the monitoring videos of multiple monitoring devices, and determine the overlapping area of ​​the monitoring videos of the multiple monitoring devices based on the target video frame; The image extraction module 420 is used to extract the grayscale image and chroma image corresponding to each monitoring device from the target video frame, extract the corresponding grayscale overlapping image from the grayscale image of each monitoring device according to the overlapping area of ​​the monitoring video, and extract the corresponding chroma overlapping image from the chroma image of each monitoring device. The pixel transformation module 430 is used to perform pixel transformation and fusion on the grayscale overlapping images of each of the monitoring devices to obtain a fused grayscale image. The color noise reduction module 440 is used to perform color calibration and color noise reduction on the color overlap image corresponding to each of the monitoring devices to obtain a calibration image, and to fuse each calibration image to obtain a fused color image; The image fusion module 450 is used to fuse the fused grayscale image and the fused chroma image to obtain a target fused image, and generate a target monitoring image based on the target fused image and the target video frame.

[0064] Optionally, the above modules can be stored in the form of software or firmware. Figure 1 The memory shown is either stored in or embedded in the operating system (OS) of the electronic device, and can be used by... Figure 1 The processor executes the commands. Meanwhile, the data and program code required to execute these modules can be stored in memory.

[0065] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0066] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0067] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0068] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image fusion method, characterized in that, include: Obtain a target video frame from the monitoring videos of multiple monitoring devices, and determine the overlapping area of ​​the monitoring videos of the multiple monitoring devices based on the target video frame; Extract the grayscale image and chroma image corresponding to each monitoring device from the target video frame; extract the corresponding grayscale overlapping image from the grayscale image of each monitoring device according to the overlapping area of ​​the monitoring video; and extract the corresponding chroma overlapping image from the chroma image of each monitoring device. The grayscale images of each of the monitoring devices that overlap are transformed and fused to obtain a fused grayscale image. A calibrated image is obtained by performing chromaticity calibration and color noise reduction on the chromaticity coincidence image corresponding to each monitoring device, and then the calibrated images are fused to obtain a fused chromaticity image. The fused grayscale image and the fused chroma image are fused to obtain a target fused image, and a target monitoring image is generated based on the target fused image and the target video frame.

2. The image fusion method according to claim 1, characterized in that, The monitoring equipment includes a first monitoring device and a second monitoring device. The step of determining the overlapping area of ​​the monitoring videos of the multiple monitoring devices based on the target video frame includes: The monitoring video range of the first monitoring device is determined based on the first target video frame of the first monitoring device. The monitoring video range of the second monitoring device is determined based on the second target video frame of the second monitoring device; the first target video frame and the second target video frame are video frames at the same time. The overlapping area of ​​the monitoring videos is determined based on the monitoring video range of the first monitoring device and the monitoring video range of the second monitoring device.

3. The image fusion method according to claim 2, characterized in that, The step of generating a target surveillance image based on the target fused image and the target video frame includes: An initial monitoring image is generated based on the first target video frame and the second target video frame; The target monitoring image is obtained by replacing the overlapping area of ​​the monitoring video in the initial monitoring image with the target fused image.

4. The image fusion method according to claim 1, characterized in that, The step of performing pixel transformation and fusion on the grayscale overlapping images of each of the monitoring devices to obtain a fused grayscale image includes: Calculate the weight of each pixel in the grayscale overlay image of each of the monitoring devices; The transformed pixel value of each pixel is calculated based on the weight of each pixel and the original pixel value; A fused grayscale value is calculated based on the transformed pixel value and the original pixel value, and the fused grayscale image is generated based on the fused grayscale value.

5. The image fusion method according to claim 4, characterized in that, The step of performing chromaticity calibration and noise reduction on the chromaticity coincidence image corresponding to each of the monitoring devices to obtain a calibrated image includes: A reference image is selected from multiple chromaticity-matched images corresponding to multiple monitoring devices; Calculate the mean value of the reference U component corresponding to the reference image and the mean value of the U component of other chromaticity-matching images; the other chromaticity-matching images are any chromaticity-matching images other than the reference image; A correction value is calculated based on the mean of the reference U component and the mean of the U component of other chromaticity-overlapping images. A chromaticity calibration map is obtained by performing chromaticity calibration on the chromaticity-overlapping images based on the correction value. The calibration image is obtained by performing color noise removal on the chromaticity calibration map.

6. The image fusion method according to claim 5, characterized in that, The step of performing color noise removal on the color calibration map to obtain the calibration image includes: Obtain the U-component gradient field of each chromaticity calibration image, and generate a target gradient field based on the U-component gradient field and the activation function; The target gradient field is used to replace the U component gradient field in the chromaticity calibration map to generate the calibration U component corresponding to the chromaticity calibration map; The calibration image is generated based on the calibration U component.

7. The image fusion method according to claim 6, characterized in that, The step of generating the calibration image based on the calibration U component includes: A fused image is generated based on the calibrated U component and the weight of each pixel; The calibrated image is obtained by performing local mean calibration on the fused image.

8. An image fusion apparatus, characterized in that, include: The video acquisition module is used to acquire target video frames from the monitoring videos of multiple monitoring devices, and determine the overlapping area of ​​the monitoring videos of the multiple monitoring devices based on the target video frames; The image extraction module is used to extract the grayscale image and chroma image corresponding to each monitoring device from the target video frame, extract the corresponding grayscale overlapping image from the grayscale image of each monitoring device according to the overlapping area of ​​the monitoring video, and extract the corresponding chroma overlapping image from the chroma image of each monitoring device. The pixel transformation module is used to perform pixel transformation and fusion on the grayscale overlapping images of each of the monitoring devices to obtain a fused grayscale image. The color noise reduction module is used to perform color calibration and color noise reduction on the color overlap image corresponding to each of the monitoring devices to obtain a calibration image, and to fuse each calibration image to obtain a fused color image; The image fusion module is used to fuse the fused grayscale image and the fused chroma image to obtain a target fused image, and to generate a target monitoring image based on the target fused image and the target video frame.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor to implement the image fusion method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the image fusion method according to any one of claims 1-7.

Images