Image processing method and device, electronic equipment and storage medium

Abstract

The application discloses the technical field of image processing, and provides an image processing method, device, electronic equipment and storage medium, the method comprises the following steps: acquiring a weak light field image and an infrared field image in a target environment; the weak light field image is an image collected by a camera device in a weak light environment; the infrared field image is an image collected by an infrared camera device; the weak light field image is enhanced to obtain an enhanced image; the infrared field image and the enhanced image are fused to determine a clear image; the image processing method, device, electronic equipment and storage medium provided by the application fuse the enhanced image obtained by enhancing the weak light field image and the infrared field image, obtain an image with clear details and clear outlines, are suitable for the image enhancement scene of the weak light field, and facilitate the identification of target objects in the weak light field.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] The petroleum refining industry is characterized by high temperature, high pressure, flammability, and explosiveness in its production processes. Ensuring standardized operating procedures and worker safety is paramount, as it's impossible to have dedicated personnel to monitor worker behavior. This is where video surveillance becomes a crucial tool. Video surveillance allows for monitoring of most workers' activities. With the development of intelligent factories, video surveillance plays an increasingly important role in daily factory management. Cameras provide real-time environmental information and can perform monitoring tasks around the clock.

[0003] However, poor lighting conditions at night make it difficult to capture clear images of objects, especially pedestrians, making effective monitoring challenging. In intelligent recognition tasks, the accuracy of image recognition is low and recognition is difficult in poor lighting conditions. Currently, the monitoring equipment deployed in the refining and chemical environment cannot solve the problem of low accuracy. Summary of the Invention

[0004] To address the problems existing in the prior art, the present invention provides an image processing method, apparatus, electronic device, and storage medium.

[0005] In a first aspect, the present invention provides an image processing method, comprising:

[0006] Acquire low-light field images and infrared field images in the target environment; the low-light field images are images captured by the camera device in a low-light environment; the infrared field images are images captured by the infrared camera device.

[0007] The weak light field image is then enhanced to obtain an enhanced image;

[0008] The infrared field image and the enhanced image are fused to determine a clear image.

[0009] In one embodiment, the method further includes: when it is determined that no infrared field image has been acquired, performing contour recognition on the weak light field image to obtain a contour image;

[0010] The contour image and the enhanced image are fused to determine a clear image.

[0011] In one embodiment, enhancing the weak light field image to obtain an enhanced image includes:

[0012] Extract the low-light image features from the low-light field image, input the low-light image features into the enhancement model, and based on the mapping relationship between the low-light image features and the strong-light image features contained in the enhancement model, map the low-light image features to obtain the strong-light image features.

[0013] The weak light field image is enhanced based on the characteristics of the strong light image to obtain an enhanced image.

[0014] In one embodiment, the step of fusing the infrared field image and the enhanced image to determine a clear image includes:

[0015] Extract the infrared image features from the infrared field image;

[0016] The infrared image features and the strong light image features are used to determine the fused features using a feature fusion formula;

[0017] The enhanced image is fused according to the fusion features to determine the clear image;

[0018] The feature fusion formula includes:

[0019] x boost =α·x infra +β·x nature

[0020] Where, x boost For fusion features, x infra For infrared image features, α is x infra Matrices of the same size, x nature For strong light image features, β is x nature For matrices of the same scale, · represents the product of corresponding positions in the matrix, and α[i,j]+β[i,j]≤1; i,j are the rows and columns of the matrix.

[0021] In one embodiment, performing contour recognition on the weak light field image to obtain a contour image includes:

[0022] Extract the low-light image features from the low-light field image, input the low-light image features into the contour model, and based on the mapping relationship between the low-light image features and infrared image features contained in the contour model, map the low-light image features to obtain infrared image features.

[0023] Based on the infrared image features, a contour image is obtained by constructing the contour of the weak light field image.

[0024] In a second aspect, the present invention provides an image processing apparatus, comprising:

[0025] The acquisition module is used to acquire low-light field images and infrared field images in the target environment; the low-light field images are images captured by the camera device in a low-light environment; the infrared field images are images captured by the infrared camera device.

[0026] An enhancement module is used to enhance the weak light field image to obtain an enhanced image;

[0027] The fusion module is used to fuse the infrared field image and the enhanced image to determine a clear image.

[0028] In one embodiment, the device further includes an identification module, configured to perform contour recognition on the weak light field image to obtain a contour image when it is determined that no infrared field image has been acquired;

[0029] Accordingly, the fusion module is further configured to: perform image fusion on the contour image and the enhanced image to determine a clear image.

[0030] In one embodiment, the fusion module, in the process of fusing the infrared field image and the enhanced image to determine a clear image, is specifically used for:

[0031] Extract the infrared image features from the infrared field image;

[0032] The infrared image features and the strong light image features are used to determine the fused features using a feature fusion formula;

[0033] The enhanced image is fused according to the fusion features to determine the clear image;

[0034] The feature fusion formula includes:

[0035] x boost =α·x infra +β·x nature

[0036] Where, x boost For fusion features, x infra For infrared image features, α is x infra Matrices of the same size, x nature For strong light image features, β is x nature For matrices of the same scale, · represents the product of corresponding positions in the matrix, and α[i,j]+β[i,j]≤1; i,j are the rows and columns of the matrix.

[0037] Thirdly, the present invention provides an electronic device, including a memory and a memory storing a computer program, wherein the processor executes the program to implement the steps of the image processing method described in the first aspect.

[0038] Fourthly, the present invention provides a processor-readable storage medium storing a computer program for causing the processor to perform the steps of the image processing method described in the first aspect.

[0039] The image processing method, apparatus, electronic device, and storage medium provided by this invention fuse the enhanced image obtained after enhancing a weak light field image with an infrared field image to obtain an image with clear details and outlines. This method is suitable for image enhancement scenarios in weak light fields and facilitates the identification of target objects in weak light fields. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0041] Figure 1 This is a flowchart illustrating the image processing method provided by the present invention;

[0042] Figure 2 These are actual illustrations of the image processing method provided by this invention;

[0043] Figure 3 This is a schematic diagram of the image processing device provided by the present invention;

[0044] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0046] The following is combined Figures 1-4 The present invention describes the image processing method, apparatus, electronic device, and storage medium.

[0047] Figure 1 A flowchart illustrating an image processing method according to the present invention is shown below. Figure 1 The method includes:

[0048] 11. Acquire low-light field images and infrared field images in the target environment; the low-light field image is the image captured by the camera device in a low-light environment; the infrared field image is the image captured by the infrared camera device;

[0049] 12. Perform image enhancement on the weak light field image to obtain the enhanced image;

[0050] 13. Perform image fusion on the infrared field image and the enhanced image to determine the clear image.

[0051] Regarding steps 11 to 13, it should be noted that in this invention, the camera device, due to its installation angle range, can acquire images of a specific environment. Based on the division of light intensity in the environment, a weak light field (i.e., a low-light environment) and a strong light field (also known as a natural light field) can be distinguished. The image acquired by the camera device in a low-light environment is a weak light field image. If the camera device has infrared functionality, the acquired image is an infrared field image.

[0052] In low-light environments (such as the dim light from streetlights at night), the acquired low-light field images are relatively blurry, but they still provide overall image detail. Infrared field images, on the other hand, provide infrared contour features. Therefore, fusing the overall detail features of the low-light field image and the infrared contour features of the infrared field image yields the desired image.

[0053] However, low-light field images are inherently blurry, and therefore, the overall details they provide are not readily apparent. Therefore, before feature fusion, image enhancement is necessary to obtain an enhanced image. This enhanced image provides more distinct overall details, which is beneficial for obtaining a clearer image.

[0054] After the enhanced image is acquired, the infrared field image and the enhanced image are fused to determine the clear image. That is, the contour features provided by the infrared field image and the detail features provided by the enhanced image are fused to obtain an image with relatively clear details and contours. See details below. Figure 2 .

[0055] The image processing method provided by this invention fuses the enhanced image obtained after enhancing a weak light field image with an infrared field image to obtain an image with clear details and outlines. It is suitable for image enhancement scenarios in weak light fields and facilitates the identification of target objects in weak light fields.

[0056] In a further step of the above method, when the camera device does not have infrared functionality, it cannot acquire infrared field images, and therefore only weak light field images exist. Thus, it is necessary to perform contour recognition on the weak light field images to obtain contour images. These contour images are equivalent to the infrared field images.

[0057] Once the contour image is obtained, the contour image and the enhanced image are then fused to determine the clear image.

[0058] A further method of the present invention converts a weak light field image into contour recognition and image enhancement respectively, and then performs image fusion to obtain an image with clear details and clear contours. This method is suitable for image enhancement scenarios in weak light fields, facilitates the identification of target objects in weak light fields, and can also be applied to different weak light field scenarios.

[0059] The further method described above mainly involves explaining the process of enhancing a weak light field image to obtain an enhanced image, as detailed below:

[0060] Extract the low-light image features from the low-light field image, input the low-light image features into the enhancement model, and based on the mapping relationship between the low-light image features and the strong-light image features contained in the enhancement model, the low-light image features are mapped to the strong-light image features.

[0061] The weak light field image is enhanced based on the characteristics of the strong light image to obtain the enhanced image.

[0062] It should be noted that in this invention, a large number of acquired low-light field images and high-light field images (images acquired in high-light environments) are first used as training samples to train a network model, resulting in an enhancement model where the input image is a low-light field image and the output image is the desired enhanced image. During the training process of this model, a mapping relationship between low-light image features and high-light image features can be determined. Based on this mapping relationship, the input low-light field image can be converted into an enhanced image.

[0063] A more detailed explanation involves extracting the low-light image features from the low-light field image, inputting these features into the enhancement model, and then mapping the low-light image features to the strong-light image features based on the mapping relationship between the low-light image features and the strong-light image features contained in the enhancement model. Finally, the strong-light image features are used to adjust the features on the low-light field image to obtain the enhanced image.

[0064] A further method described above primarily involves explaining the process of fusing infrared field images and enhanced images to determine a clear image, as detailed below:

[0065] Extract infrared image features from infrared field images;

[0066] The fusion features are determined by using a feature fusion formula to combine infrared image features and strong light image features.

[0067] The enhanced images are fused based on fusion features to determine the clear image;

[0068] The feature fusion formula includes:

[0069] x boost =α·x infra +β·x nature

[0070] Where, x boost For fusion features, x infra For infrared image features, α is x infra Matrices of the same size, x nature For strong light image features, β is x nature For matrices of the same scale, · represents the product of corresponding positions in the matrix, and α[i,j]+β[i,j]≤1; i,j are the rows and columns of the matrix.

[0071] It should be noted that, in this invention, by using the above-mentioned feature fusion formula, infrared image features and strong light image features can be fused to obtain fused features, and then the fused features are used to adjust the features on the enhanced image to obtain a clear image.

[0072] A further method described above mainly involves contour recognition of the weak light field image to obtain a contour image. The processing steps are explained below:

[0073] Extract the low-light image features from the low-light field image, input the low-light image features into the contour model, and based on the mapping relationship between the low-light image features and infrared image features contained in the contour model, the low-light image features are mapped to obtain infrared image features.

[0074] Contour images are obtained by constructing contours from weak light field images based on infrared image features.

[0075] It should be noted that in this invention, a large number of acquired low-light field images and infrared field images are first used as training samples to train a network model, resulting in a contour model where the input image is a low-light field image and the output image is the desired contour image. During the training process of this model, a mapping relationship between low-light image features and infrared image features can be determined. Based on this mapping relationship, the input low-light field image can be converted into a contour image. It should be noted that the infrared field image mainly provides contour features, and the image obtained based on the low-light field image is not the acquired image; therefore, it is called a contour image and is used to characterize the infrared field image.

[0076] A more detailed explanation involves extracting the low-light image features from the low-light field image, inputting these features into the contour model, and then mapping the low-light image features to infrared image features based on the mapping relationship between the low-light image features and infrared image features contained in the enhancement model. Finally, the infrared image features are used to adjust the features on the low-light field image to obtain the contour image.

[0077] A further method involves fusing the contour image and the enhanced image to determine the clear image, and the process is explained as follows:

[0078] Extract infrared image features from the contour image;

[0079] The fusion features are determined by using a feature fusion formula to combine infrared image features and strong light image features.

[0080] The enhanced images are fused based on fusion features to determine the clear image;

[0081] The feature fusion formula includes:

[0082] x boost =α·x infra +β·x nature

[0083] Where, x boost For fusion features, x infra For infrared image features, α is x infra Matrices of the same size, x nature For strong light image features, β is x nature For matrices of the same scale, · represents the product of corresponding positions in the matrix, and α[i,j]+β[i,j]≤1; i,j are the rows and columns of the matrix.

[0084] It should be noted that, in this invention, by using the above-mentioned feature fusion formula, infrared image features and strong light image features can be fused to obtain fused features, and then the fused features are used to adjust the features on the enhanced image to obtain a clear image.

[0085] The image processing apparatus provided by the present invention will be described below. The image processing apparatus described below can be referred to in correspondence with the image processing method described above.

[0086] Figure 3 A schematic flowchart of an image processing apparatus provided by the present invention is shown below. Figure 3 The device includes an acquisition module 31, an enhancement module 32, and a fusion module 33, wherein:

[0087] The acquisition module 31 is used to acquire low-light field images and infrared field images in the target environment; the low-light field image is an image captured by the camera device in a low-light environment; the infrared field image is an image captured by the infrared camera device.

[0088] Enhancement module 32 is used to enhance the weak light field image to obtain an enhanced image;

[0089] The fusion module 33 is used to fuse the infrared field image and the enhanced image to determine a clear image.

[0090] In a further embodiment of the above-mentioned device, the device further includes an identification module for determining that when no infrared field image is acquired, the weak light field image is subjected to contour recognition to obtain a contour image.

[0091] Accordingly, the fusion module is also used to: fuse the contour image and the enhanced image to determine the clear image.

[0092] In a further embodiment of the aforementioned apparatus, the fusion module, in the process of fusing the infrared field image and the enhanced image to determine the clear image, is specifically used for:

[0093] Extract infrared image features from infrared field images;

[0094] The fusion features are determined by using a feature fusion formula to combine infrared image features and strong light image features.

[0095] The enhanced images are fused based on fusion features to determine the clear image;

[0096] The feature fusion formula includes:

[0097] x boost =α·x infra +β·x nature

[0098] Where, x boost For fusion features, x infra For infrared image features, α is x infra Matrices of the same size, x nature For strong light image features, β is x nature For matrices of the same scale, · represents the product of corresponding positions in the matrix, and α[i,j]+β[i,j]≤1; i,j are the rows and columns of the matrix.

[0099] In a further embodiment of the above-described apparatus, the enhancement module is specifically used for:

[0100] Extract the low-light image features from the low-light field image, input the low-light image features into the enhancement model, and based on the mapping relationship between the low-light image features and the strong-light image features contained in the enhancement model, the low-light image features are mapped to the strong-light image features.

[0101] The weak light field image is enhanced based on the characteristics of the strong light image to obtain the enhanced image.

[0102] In a further embodiment of the above-described device, the identification module is specifically used for:

[0103] Extract the low-light image features from the low-light field image, input the low-light image features into the contour model, and based on the mapping relationship between the low-light image features and infrared image features contained in the contour model, the low-light image features are mapped to obtain infrared image features.

[0104] Contour images are obtained by constructing contours from weak light field images based on infrared image features.

[0105] Since the device described in this embodiment of the invention is based on the same principle as the method described in the above embodiments, more detailed explanations will not be repeated here.

[0106] It should be noted that, in the embodiments of the present invention, the relevant functional modules can be implemented by a hardware processor.

[0107] The image processing device provided by the present invention fuses the enhanced image obtained after enhancing a weak light field image with an infrared field image to obtain an image with clear details and outlines. It is suitable for image enhancement scenarios in weak light fields and facilitates the identification of target objects in weak light fields.

[0108] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include a processor 41, a communication interface 42, a memory 43, and a communication bus 44. The processor 41, communication interface 42, and memory 43 communicate with each other via the communication bus 44. The processor 41 can call a computer program stored in the memory 43 to execute the steps of an image processing method, such as: acquiring a low-light field image and an infrared field image in the target environment; the low-light field image is an image captured by a camera in a low-light environment; the infrared field image is an image captured by an infrared camera; enhancing the low-light field image to obtain an enhanced image; and fusing the infrared field image and the enhanced image to determine a clear image.

[0109] Furthermore, the logical instructions in the aforementioned memory 43 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0110] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to perform the steps of an image processing method, such as: acquiring a low-light field image and an infrared field image in a target environment; the low-light field image being an image captured by a camera device in a low-light environment; the infrared field image being an image captured by an infrared camera device; enhancing the low-light field image to obtain an enhanced image; and fusing the infrared field image and the enhanced image to determine a clear image.

[0111] On the other hand, embodiments of this application also provide a processor-readable storage medium storing a computer program for causing the processor to execute steps of an image processing method, such as: acquiring a low-light field image and an infrared field image in a target environment; the low-light field image being an image captured by a camera device in a low-light environment; the infrared field image being an image captured by an infrared camera device; enhancing the low-light field image to obtain an enhanced image; and fusing the infrared field image and the enhanced image to determine a clear image.

[0112] The processor-readable storage medium can be any available medium or data storage device that the processor can access, including but not limited to magnetic memory (e.g., floppy disk, hard disk, magnetic tape, magneto-optical disk (MO)), optical memory (e.g., CD, DVD, BD, HVD), and semiconductor memory (e.g., ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)).

[0113] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0114] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image processing method, characterized by, The method comprises the following steps: obtaining a weak light field image and an infrared field image in a target environment; the weak light field image is an image collected by a camera in a weak light environment; the infrared field image is an image collected by an infrared camera; performing image enhancement on the weak light field image to obtain an enhanced image, comprising: extracting weak light image features of the weak light field image, inputting the weak light image features into an enhancement model, and mapping the weak light image features to strong light image features based on a mapping relationship between the weak light image features and the strong light image features contained in the enhancement model; wherein the enhancement model is obtained by training a network model using collected weak light field images and strong light field images as training samples; enhancing the weak light field image based on the strong light image features to obtain an enhanced image; performing image fusion on the infrared field image and the enhanced image to determine a clear image.

2. The image processing method of claim 1, wherein, The method further comprises: when the infrared field image is not obtained, performing contour recognition on the weak light field image to obtain a contour image; performing image fusion on the contour image and the enhanced image to determine a clear image.

3. The image processing method of claim 1, wherein, The image fusion of the infrared field image and the enhanced image to determine a clear image comprises: extracting infrared image features of the infrared field image; determining fusion features by using a feature fusion formula on the infrared image features and the strong light image features; performing fusion on the enhanced image based on the fusion features to determine a clear image; wherein the feature fusion formula comprises: x boost = a x infra + b x nature where x boost is the fusion feature, x infra is the infrared image feature, a is a matrix of the same dimension as x infra , x nature is the strong light image feature, b is a matrix of the same dimension as x nature , · is the multiplication of the corresponding positions of the matrices, and a[i,j]+b[i,j]≤1; i,j are the rows and columns of the matrix.

4. The image processing method of claim 2, wherein, The contour recognition on the weak light field image to obtain a contour image comprises: extracting weak light image features of the weak light field image, inputting the weak light image features into a contour model, and mapping the weak light image features to infrared image features based on a mapping relationship between the weak light image features and the infrared image features contained in the contour model; and performing contour construction on the weak light field image based on the infrared image features to obtain a contour image.

5. An image processing apparatus characterized by comprising: The method comprises the following steps: an acquisition module for acquiring a weak light field image and an infrared field image in a target environment; the weak light field image is an image collected by a camera in a weak light environment; the infrared field image is an image collected by an infrared camera; an enhancement module for performing image enhancement on the weak light field image to obtain an enhanced image, comprising: extracting weak light image features of the weak light field image, inputting the weak light image features into an enhancement model, and mapping the weak light image features to strong light image features based on a mapping relationship between the weak light image features and the strong light image features contained in the enhancement model; wherein the enhancement model is obtained by training a network model using collected weak light field images and strong light field images as training samples; enhancing the weak light field image based on the strong light image features to obtain an enhanced image; a fusion module for performing image fusion on the infrared field image and the enhanced image to determine a clear image.

6. The image processing apparatus according to claim 5, characterized by The device further comprises an identification module for, when the infrared field image is not obtained, performing contour recognition on the weak light field image to obtain a contour image; Correspondingly, the fusion module is further configured to: perform image fusion on the contour image and the enhanced image to determine a clear image.

7. The image processing apparatus according to claim 5, characterized by, In the process of performing image fusion on the infrared field image and the enhanced image to determine a clear image, the fusion module is specifically configured to: extract infrared image features of the infrared field image; determine fusion features by using a feature fusion formula on the infrared image features and the strong light image features; perform fusion on the enhanced image according to the fusion features to determine a clear image; wherein the feature fusion formula comprises: x boost = a x infra + b x nature Where, x boost For fusion features, x infra For infrared image features, α is x infra Matrices of the same size, x nature For strong light image features, β is x nature For matrices of the same scale, · represents the product of corresponding positions in the matrix, and α[i,j]+β[i,j]≤1; i,j are the rows and columns of the matrix.

8. An electronic device comprising a processor and a memory having a computer program stored therein, characterized in that The processor executes the computer program to implement the steps of the image processing method in any one of claims 1 to 4.

9. A processor-readable storage medium, comprising: The processor readable storage medium stores a computer program, and the computer program is configured to enable the processor to execute the steps of the image processing method in any one of claims 1 to 4.

Images