Image acquisition method, system and equipment in dark environment

Abstract

The invention discloses an image acquisition method, system and device in a dark environment, and belongs to the field of video monitoring in the dark environment. An imaging device is arranged, and a granular image shot in a dark environment is obtained; matching granular images shot in a dark environment with corresponding clear images to generate a sample set, and dividing the sample set into a training sample set and a test sample set; constructing an image embedded graph neural network model, and training the image embedded graph neural network model based on the training sample set to obtain a trained image embedded graph neural network model; testing and evaluating the trained image embedded graph neural network model based on the test sample set to obtain a final image embedded graph neural network model; through the image embedded graph neural network model provided by the invention, a full-color image with rich colors and clear details can be quickly and efficiently generated.

Description

Technical Field

[0001] The present invention relates to the field of video monitoring in dark environments, and more particularly to a method, system and device for acquiring images in dark environments. Background Art

[0002] In dim light environments, traditional imaging devices often find it difficult to capture clear, high-quality images. The dark environment image acquisition method based on neural networks can show significant advantages in processing images in low light, at night or in completely dark environments.

[0003] However, the existing neural network-based dark environment image acquisition methods mainly rely on deep learning technologies such as convolutional neural networks (CNNs). These methods, such as CN 112861987A, disclose a target detection method in a dark light environment. Although the learning ability of the model is increased by changing the sampling strategy in the downsampling process of the target detection algorithm, this multi-layer model structure will increase the number of nodes and layers in the middle layer when dealing with complex learning problems, which will lead to problems such as increased computational complexity and output delay, resulting in low efficiency and affecting the output rate and actual effect of image acquisition in a dark environment.

[0004] Therefore, how to provide a method, system and device for acquiring images in a dark environment that can solve the above-mentioned problems is an issue that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In view of this, the present invention provides a method, system and device for acquiring images in a dark environment, so as to solve the technical problems existing in the above-mentioned prior art.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A method for acquiring images in a dark environment comprises the following steps:

[0008] Setting up an imaging device and acquiring a grainy image taken in a dark environment;

[0009] Pair the grainy images taken in a dark environment with the corresponding clear images to generate a sample set, which is divided into a training sample set and a test sample set;

[0010] Construct an image embedding graph neural network model, train the image embedding graph neural network model based on the training sample set, and obtain the trained image embedding graph neural network model;

[0011] The trained image embedding graph neural network model is tested and evaluated based on the test sample set to obtain the final image embedding graph neural network model;

[0012] The image embedding graph neural network model outputs full-color images in dark environments.

[0013] Preferably, the imaging device is provided comprising:

[0014] Set the exposure time, ISO sensitivity and white balance parameters of the imaging device to increase the light throughput of the lens;

[0015] Improve lens coating by stacking lens coating layers of different refractive index and thickness.

[0016] Preferably, the image embedding graph neural network model is a hierarchical neural network architecture, comprising: an image feature extraction layer, a graph embedding representation learning layer, and an image reconstruction layer, each layer comprising a plurality of nodes;

[0017] Among them, the image feature extraction layer extracts the feature information of the granular image in the input training sample set; the graph embedding representation learning layer transforms the feature information and embeds it into the node vector in the low-dimensional space; the image reconstruction layer reconstructs the enhanced image corresponding to the granular image from the embedded node vector.

[0018] Preferably, the image embedding graph neural network model further includes: a weight training unit and a decision processing unit;

[0019] Among them, the weight training unit performs indirect connection and adjusts the weight parameters between the indirectly connected nodes; the decision processing unit uses the weight parameters optimized by the weight training unit to update the weight configuration between the indirectly connected nodes, and obtains a full-color image in a dark environment through the updated image embedding graph neural network model output.

[0020] Preferably, the weight training unit specifically includes:

[0021] Select some nodes for indirect connection based on preset strategies;

[0022] Using the back-propagation algorithm, the weight parameters between indirectly connected nodes are continuously adjusted according to the training sample set.

[0023] Preferably, the preset strategy includes: one of an importance evaluation strategy or a feature similarity strategy.

[0024] Preferably, the decision processing unit specifically includes:

[0025] Receiving weight parameters adjusted by the weight training unit;

[0026] The weight parameters are used to update the weight configuration between indirectly connected nodes in the image embedding graph neural network model.

[0027] Preferably, the decision processing unit further includes a post-processing operation, and the post-processing operation includes an image enhancement operation and a denoising operation.

[0028] An image acquisition system in a dark environment, comprising:

[0029] An imaging device for obtaining a grainy image taken in a dark environment;

[0030] A preprocessing module is used to pair the grainy images taken in a dark environment with the corresponding clear images to generate a sample set, and divide it into a training sample set and a test sample set;

[0031] A construction module is used to construct an image embedding graph neural network model, and train the image embedding graph neural network model based on a training sample set to obtain a trained image embedding graph neural network model;

[0032] The testing module is used to test and evaluate the trained image embedding graph neural network model based on the test sample set to obtain the final image embedding graph neural network model;

[0033] The output module outputs full-color images in dark environments through the image embedding graph neural network model.

[0034] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for acquiring images in a dark environment as described above is implemented.

[0035] It can be seen from the above technical solutions that compared with the prior art, the present invention discloses a method, system and device for acquiring images in a dark environment. Through pairing training of a large number of samples, the constructed image embedding graph neural network model can accurately capture the mapping relationship between images. At the same time, the weight training unit and the decision processing unit in the image embedding graph neural network model do not rely on learning data to form sparse interconnection, which can realize high-speed discriminant learning and discriminant processing, and is conducive to the rapid and efficient generation of full-color images with rich colors and clear details. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0037] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0038] Figure 2It is a schematic diagram of the system structure of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] See also Figure 1 The embodiment of the present invention discloses a method for acquiring an image in a dark environment, comprising the following steps:

[0041] Setting up an imaging device and acquiring a grainy image taken in a dark environment;

[0042] Pair the grainy images taken in a dark environment with the corresponding clear images to generate a sample set, which is divided into a training sample set and a test sample set;

[0043] Construct an image embedding graph neural network model, train the image embedding graph neural network model based on the training sample set, and obtain the trained image embedding graph neural network model;

[0044] The trained image embedding graph neural network model is tested and evaluated based on the test sample set to obtain the final image embedding graph neural network model;

[0045] The image embedding graph neural network model outputs full-color images in dark environments.

[0046] In a specific embodiment, providing an imaging device includes:

[0047] Set the exposure time, ISO sensitivity and white balance parameters of the imaging device to increase the light throughput of the lens;

[0048] Improve lens coating by stacking lens coating layers of different refractive index and thickness.

[0049] Specifically, by increasing the light transmittance of the lens, more light energy can pass through the lens and hit the target surface of the image sensor in a dark environment.

[0050] Improve the lens coating so that light in the ultraviolet, visible and infrared bands in dark environments can pass through the lens and hit the sensor target surface.

[0051] In a specific embodiment, the image embedding graph neural network model is a hierarchical neural network architecture, including: an image feature extraction layer, a graph embedding representation learning layer, and an image reconstruction layer, each layer including a plurality of nodes;

[0052] Among them, the image feature extraction layer extracts the feature information of the granular image in the input training sample set; the graph embedding representation learning layer transforms the feature information and embeds it into the node vector in the low-dimensional space; the image reconstruction layer reconstructs the enhanced image corresponding to the granular image from the embedded node vector.

[0053] Specifically, the main function of the image feature extraction unit is to automatically extract useful feature information from the input original image. These feature information can be color, texture, shape, etc., which are crucial for subsequent image analysis and processing. In this embodiment, this unit uses a convolutional neural network (CNN) to achieve automatic feature extraction.

[0054] The role of the graph embedding representation learning unit is to convert image features and possible graph structure information into low-dimensional, continuous vector representations. These vector representations can capture the complex relationships between image features and make subsequent processing in the graph neural network more efficient. In this embodiment, the unit uses a graph neural network (GNN) or other graph embedding technologies, such as DeepWalk, Node2Vec, etc.

[0055] The purpose of the image reconstruction unit is to restore or reconstruct the original image or image features from the embedded node vector. In this embodiment, this unit is implemented using a decoder network or other generative model, such as the generator part of a generative adversarial network (GAN). By reconstructing the image, the quality of the embedded representation can be evaluated and whether the model effectively captures the key information of the image can be verified.

[0056] In a specific embodiment, the image embedding graph neural network model further includes: a weight training unit and a decision processing unit;

[0057] Among them, the weight training unit selects some nodes in the hierarchical neural network architecture for indirect connection based on the vector matrix of the clear image, and adjusts the weight parameters between the indirectly connected nodes; the decision processing unit uses the weight parameters optimized by the weight training unit to update the weight configuration between the indirectly connected nodes, and obtains full-color images (videos and photos) in a dark environment through the output of the updated image embedding graph neural network model.

[0058] The weight training unit specifically includes:

[0059] Receiving vector matrices of a clear image as input, these vector matrices are usually extracted from the original clear image by an image feature extraction unit, and they contain key feature information of the image;

[0060] In the hierarchical neural network architecture, the weight training unit will select some nodes for indirect connection based on certain strategies (such as importance evaluation strategy, feature similarity strategy, etc.). This indirect connection helps to enhance information interaction and feature fusion between nodes while maintaining network complexity;

[0061] Using the back-propagation algorithm, the weight training unit continuously adjusts the weight parameters between the indirect connection nodes according to the training sample set (including clear images and their labels). The purpose of this process is to minimize the loss function and improve the performance of the model on specific tasks (such as full-color image generation in dark environments).

[0062] Specifically, the importance assessment strategy means that when using a machine learning model (image embedding graph neural network model), the model itself can output the importance score of the feature, which is usually based on the frequency of use or contribution of the feature in the model.

[0063] Specifically, the above process will be iterated multiple times until the performance of the model on the validation set reaches the preset standard or no longer improves significantly. In each iteration, the weight parameters are updated based on the error between the output of the current model and the true label.

[0064] Specifically, the feature similarity strategy is calculated by comparing features such as pixel value, texture, shape, etc. of the image, such as feature similarity FSIM (Feature Similarity Index Measure).

[0065] The decision processing unit specifically includes:

[0066] Receive weight parameters optimized by the weight training unit. These weight parameters have been adjusted and optimized for specific tasks and can better reflect the complex relationship between image features;

[0067] Using the weight parameters, the weight configuration between the indirectly connected nodes in the image embedding graph neural network model is updated. This step is the key to ensuring that the model can accurately reflect the relationship between image features.

[0068] Specifically, the decision processing unit also includes post-processing operations, which include image enhancement operations and denoising operations, which help to reduce noise and artifacts in the generated image and improve the clarity and realism of the image.

[0069] Specifically, the updated image embedding graph neural network model is used to process image data in dark environments. By utilizing the embedding representation learning and image reconstruction units in the model, the model is able to generate full-color images similar to clear images. This process involves multiple steps, including image feature extraction, embedding representation learning, image feature fusion, and final image reconstruction.

[0070] See also Figure 2 The embodiment of the present invention discloses an image acquisition system in a dark environment, comprising:

[0071] An imaging device for obtaining a grainy image taken in a dark environment;

[0072] A preprocessing module is used to pair the grainy images taken in a dark environment with the corresponding clear images to generate a sample set, and divide it into a training sample set and a test sample set;

[0073] A construction module is used to construct an image embedding graph neural network model, and train the image embedding graph neural network model based on a training sample set to obtain a trained image embedding graph neural network model;

[0074] The testing module is used to test and evaluate the trained image embedding graph neural network model based on the test sample set to obtain the final image embedding graph neural network model;

[0075] The output module outputs full-color images in dark environments through the image embedding graph neural network model.

[0076] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned method for acquiring images in a dark environment is implemented.

[0077] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0078] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for acquiring images in a dark environment, characterized in that: The following steps are involved: Setting up an imaging device and acquiring a grainy image taken in a dark environment; Pair the grainy images taken in a dark environment with the corresponding clear images to generate a sample set, which is divided into a training sample set and a test sample set; Construct an image embedding graph neural network model, train the image embedding graph neural network model based on the training sample set, and obtain the trained image embedding graph neural network model; The trained image embedding graph neural network model is tested and evaluated based on the test sample set to obtain the final image embedding graph neural network model; The image embedding graph neural network model outputs full-color images in dark environments.

2. The method for acquiring images in a dark environment according to claim 1, characterized in that: The imaging device is provided comprising: Set the exposure time, ISO sensitivity and white balance parameters of the imaging device to increase the light throughput of the lens; Improve lens coating by stacking lens coating layers of different refractive index and thickness.

3. The method for acquiring images in a dark environment according to claim 1, characterized in that: The image embedding graph neural network model is a hierarchical neural network architecture, including: an image feature extraction layer, a graph embedding representation learning layer, and an image reconstruction layer, each layer containing multiple nodes; Among them, the image feature extraction layer extracts the feature information of the granular image in the input training sample set; the graph embedding representation learning layer transforms the feature information and embeds it into the node vector in the low-dimensional space; the image reconstruction layer reconstructs the enhanced image corresponding to the granular image from the embedded node vector.

4. The method for acquiring images in a dark environment according to claim 3, characterized in that: The image embedding graph neural network model also includes: a weight training unit and a decision processing unit; Among them, the weight training unit performs indirect connection and adjusts the weight parameters between the indirectly connected nodes; the decision processing unit updates the weight configuration between the indirectly connected nodes, and obtains a full-color image in a dark environment through the output of the updated image embedding graph neural network model.

5. The method for acquiring images in a dark environment according to claim 4, characterized in that: The weight training unit specifically includes: Select some nodes for indirect connection based on preset strategies; Using the back-propagation algorithm, the weight parameters between indirectly connected nodes are continuously adjusted according to the training sample set.

6. The method for acquiring images in a dark environment according to claim 5, characterized in that: The preset strategy includes: one of an importance evaluation strategy or a feature similarity strategy.

7. The method for acquiring images in a dark environment according to claim 4, characterized in that: The decision processing unit specifically includes: Receiving weight parameters adjusted by the weight training unit; The weight parameters are used to update the weight configuration between indirectly connected nodes in the image embedding graph neural network model.

8. The method for acquiring images in a dark environment according to claim 7, characterized in that: The decision processing unit also includes post-processing operations, and the post-processing operations include image enhancement operations and denoising operations.

9. A dark environment image acquisition system using the dark environment image acquisition method according to any one of claims 1 to 8, characterized in that: include: An imaging device for obtaining a grainy image taken in a dark environment; A preprocessing module is used to pair the grainy images taken in a dark environment with the corresponding clear images to generate a sample set, and divide it into a training sample set and a test sample set; A construction module is used to construct an image embedding graph neural network model, and train the image embedding graph neural network model based on a training sample set to obtain a trained image embedding graph neural network model; The testing module is used to test and evaluate the trained image embedding graph neural network model based on the test sample set to obtain the final image embedding graph neural network model; The output module outputs full-color images in dark environments through the image embedding graph neural network model.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for acquiring images in a dark environment as described in any one of claims 1 to 8 is implemented.

Images