Method for generating data sets with different illumination intensity ranges by adding synthetic noise
By adding synthetic noise to clean images, image data sets with different light intensity ranges are generated, which solves the problem of difficulty in image data acquisition in extremely low-light environments, reduces costs and improves the accuracy and robustness of the model.
Patent Information
- Application Number
- CN202411735227.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-29
- Publication Date
- 2025-04-25
AI Technical Summary
In extremely low-light environments, it is difficult to acquire high-quality image data, and the existing technology is difficult to meet the demand for large-scale data by deep learning models, and it is expensive.
By adding synthetic noise to a clean image, various types of noise are calculated based on different light intensities and converted into brightness values to the image, an image dataset with a specific light intensities is generated.
Rapidly generating a large number of image data with different lighting intensity ranges reduces the data acquisition cost by about 70%, and improves the representativeness of image data and the accuracy and robustness of the model.
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of extremely weak light full-color imaging, and is particularly suitable for scenes such as security monitoring and nighttime natural observation in low-light environments. The present invention specifically relates to a method for adding synthetic noise to generate data sets with different light intensity ranges. Background Art
[0002] In extremely weak light full-color imaging, in order to achieve high-quality image recognition and analysis, a large amount of representative image data is required to train the model. However, there are many difficulties in collecting image data under extremely low light conditions in actual environments. On the one hand, the extremely low light environment has extremely high performance requirements for the acquisition equipment, requiring professional equipment with high sensitivity and low noise, which is expensive. On the other hand, the acquisition process is greatly affected by environmental factors such as weather and time, and it is difficult to ensure the stability and continuity of the data. In addition, the amount of data actually collected is often limited, which makes it difficult to meet the needs of large-scale data for model training such as deep learning. Summary of the invention
[0003] The object of the present invention is to provide a method for generating data sets with different illumination intensity ranges by adding synthetic noise, which can quickly generate a large amount of image data with different illumination intensity ranges.
[0004] The technical solution provided by this application is:
[0005] In a first aspect, the present application provides a method for adding synthetic noise to generate data sets with different illumination intensity ranges, comprising:
[0006] Collect a large number of clean images;
[0007] For the clean image, various types of noise are calculated according to different light intensities; the calculated various types of noise are converted into brightness values, and added to corresponding pixels of the clean image to generate an image with a specific light intensity;
[0008] The images generated with specific light intensity constitute a data set.
[0009] The above generated image is a synthesized image with a specific light intensity.
[0010] In a possible implementation, the various types of noise include: photon noise;
[0011] For the clean image, photon noise is calculated according to different light intensities, and the calculated photon noise is converted into a brightness value, including:
[0012] The expected photon arrival rate is calculated according to the formula R = I × QE × t; where I is the light intensity, QE is the quantum efficiency of the detector, and t is the exposure time;
[0013] According to the formula Np=I×QE×t×η, Np represents the number of photon noise, and η is a random factor with a value between 0 and 1.
[0014] For a given light intensity I, combined with the known detector quantum efficiency QE and the preset exposure time t, the expected photon arrival rate R = I × QE × t can be calculated;
[0015] The photon arrival rate reflects the number of photons expected to arrive at the detector per unit time, while the photon noise is the actual photon noise calculated after considering the light intensity, quantum efficiency, exposure time and random factors. The photon arrival rate is an important intermediate quantity for calculating the photon noise. The photon noise is ultimately determined by combining the photon arrival rate with random factors and other factors, which in turn affects the generation of photon events and brightness changes at each pixel position of the image.
[0016] A random number generator is used to generate random numbers; a certain number of photon events are generated for each pixel of the input clean image according to the generated random numbers and the calculated photon arrival rate; each photon event increases the brightness of the corresponding pixel by a first random value to simulate the random effect of the photon event on the pixel brightness, and the first random value obeys a first normal distribution.
[0017] The photon arrival rate determines the average frequency of photon events expected at each pixel location. For example, if the photon arrival rate is high, then more photon events are more likely to occur at that pixel location; when a random number generator is used, the generated random number is used to actually determine the number of photon events that occur at each pixel location based on the photon arrival rate. Specifically, the random number can be mapped or compared with the photon arrival rate to determine whether a photon event occurs at the pixel location and how many photon events are generated (for example, when the random number is less than a certain value related to the photon arrival rate, a photon event is generated, and the final number of photon events is determined by multiple such operations).
[0018] In a possible implementation, the various types of noise include: dark current noise;
[0019] For the clean image, dark current noise is calculated according to different light intensities, and the calculated dark current noise is converted into a brightness value, including:
[0020] Determine the dark current noise parameter under a specific light intensity, assuming that it obeys the second normal distribution;
[0021] For each pixel of the clean image, a second random value obeying a second normal distribution is generated for each pixel of the input clean image to simulate the dark current noise Nd;
[0022] The dark current noise Nd is converted into a brightness value and added to the corresponding pixel.
[0023] Among them, to convert this dark current noise into a brightness value, generally speaking, it is necessary to determine the conversion relationship according to the characteristics of the imaging system (including the detector) and related parameters, or to convert it through a pre-calibrated correspondence table of dark current noise and brightness value, or to calculate and convert it according to parameters such as the conversion coefficient of the detector to convert the electronic signal into the brightness signal. For example, if the conversion coefficient of the detector's electronic signal to the brightness signal is k (obtained through experimental measurement or equipment manual), then the brightness value L = k × Nd.
[0024] In a possible implementation, the various types of noise include: readout noise;
[0025] For the clean image, the readout noise is calculated, and the calculated readout noise is converted into a brightness value, including:
[0026] Determine the readout noise parameter and assume it obeys the third normal distribution;
[0027] For each pixel of the clean image, respectively, a third random value obeying a third normal distribution is generated to simulate the readout noise;
[0028] The readout noise is converted to brightness values and added to the corresponding pixels.
[0029] Readout noise is the noise introduced in the readout process of the imaging system. Simulating readout noise can make the generated image data closer to the noise situation in the actual imaging process. In extremely weak light full-color imaging, readout noise will affect the image quality. By simulating it, this effect can be reflected in the generated synthetic noise image, making the generated data set more realistic and representative, which helps to train a model that is more adaptable to the actual imaging environment and improve the model's ability to handle noise in actual images and its accuracy.
[0030] The conversion of the readout noise into the brightness value is similar to the conversion of the dark current noise into the brightness value, and can be determined according to the characteristics of the imaging system. Assuming that the conversion coefficient is m, the brightness value L'=m×the readout noise value.
[0031] In a possible implementation, the various types of noise include: thermal noise;
[0032] For the clean image, thermal noise is calculated, and the calculated thermal noise is converted into a brightness value, including:
[0033] The thermal noise intensity is calculated according to the thermodynamic formula Nt=K×T×Δf; where Nt represents the thermal noise intensity, K is the Boltzmann constant, T is the detector temperature, and Δf is the bandwidth;
[0034] Convert thermal noise intensity to brightness value and add it to the corresponding image.
[0035] The conversion of thermal noise intensity into brightness value is similar to the conversion of dark current noise into brightness value, and can be determined according to the characteristics of the imaging system. For example, it is necessary to consider the mechanism by which the detector converts the electronic signal generated by thermal noise into a brightness signal. There is a conversion factor related to the detector, and the corresponding brightness value is obtained by multiplying the calculated thermal noise intensity by this conversion factor. Assuming that the conversion factor of the detector to thermal noise is n, the brightness value L" = m × Nt.
[0036] In a possible implementation, the clean images include images of different scenes, different lighting conditions and different objects, such as city streets, country roads, forests, various natural landscapes and the like.
[0037] In a possible implementation, the range of the light intensity is from 0.0001 lux to 1 lux.
[0038] For each clean image, various types of noise are calculated and added according to the set light intensity to generate an image with a specific light intensity. Repeat the above steps to generate thousands to tens of thousands of images of different scenes to ensure that the dataset is diverse and representative enough.
[0039] In a second aspect, the present application provides an electronic device, including: a memory and a processor;
[0040] The memory is used to store computer programs;
[0041] The processor is used to call the computer program to execute the method as described above.
[0042] In a third aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed on an electronic device, the electronic device implements the method as described above.
[0043] In a fourth aspect, the present application provides a computer program product, including a computer program, and when the computer program is executed on an electronic device, the electronic device implements the method as described above.
[0044] The specific implementation methods of the second to fourth aspects of the present application can refer to the implementation methods of the first aspect, which will not be repeated here.
[0045] Beneficial effects:
[0046] The method of the present invention generates image data sets of different light intensity ranges by adding synthetic noise to clean images, which can quickly generate a large amount of image data of different light intensity ranges, effectively solving the problem of data acquisition difficulties in extremely weak light full-color imaging. Compared with traditional data acquisition methods, the cost is reduced by about 70%.
[0047] Image datasets covering different scenes, different lighting conditions and different objects can be generated, which are highly representative. Rich and representative image data can be provided for model training, thereby significantly improving the accuracy and robustness of the extremely weak light full-color imaging model. Experimental verification shows that the accuracy of the model trained using the dataset generated by the present invention is 30% higher than that of the model trained using the existing dataset, and the robustness is also significantly improved.
[0048] Synthetic data can be customized and expanded as needed to meet the needs of different application scenarios. It can play a greater role in practical applications such as security monitoring in low-light environments and nighttime natural observation. DETAILED DESCRIPTION
[0049] In order to enable those skilled in the art to better understand the present application, the technical solution of the present application will be described in detail below in conjunction with specific embodiments.
[0050] It should be noted that the terms such as "first", "second" and the like in the specification and claims of the present application are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is a specific relationship or order between these entities or operations. It should be understood that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first" and "second" and the like do not limit necessarily different. It should be understood that the data used in this way can be interchanged under appropriate circumstances. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, article or device including a series of elements includes not only those elements clearly listed, but also other elements that are not clearly listed, or may also include elements inherent to such process, method, product or device. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0051] Embodiment 1:
[0052] This embodiment discloses a method for adding synthetic noise to generate data sets with different illumination intensity ranges, including:
[0053] Collect 1,000 clean images of different scenes, including city night scenes, rural starry skies, forest morning glow, etc. These images come from public image databases and high-quality images taken by ourselves.
[0054] Set the illumination intensity range to 0.0001 lux to 0.01 lux. According to this illumination intensity range, determine the parameter ranges of photon noise, dark current noise, and other noises. For example, for dark current noise, determine the mean and standard deviation of dark current noise by measuring the dark current characteristics of the actual detector.
[0055] The clean image is input into the synthetic noise model.
[0056] A synthetic noise model is used to calculate the expected photon arrival rate according to the formula R=I×QE×t, where I is the light intensity, QE is the quantum efficiency of the detector, and t is the exposure time. A random number generator is used to generate random numbers. According to the generated random numbers and the calculated photon arrival rate, a certain number of photon events are generated for each pixel of the input clean image. Each photon event increases the brightness of the corresponding pixel by a first random value to simulate the random effect of the photon event on the pixel brightness, and the first random value obeys a first normal distribution.
[0057] In some embodiments, the quantum efficiency QE of the detector can be set to 0.5 and the exposure time t can be set to 0.1 second.
[0058] In some embodiments, the first normal distribution is a normal distribution with a mean of 0.5 and a standard deviation of 0.2.
[0059] The synthetic noise model generates a second random value that obeys a second normal distribution for each pixel of the input clean image to simulate the dark current noise Nd. The dark current noise Nd is converted into a brightness value and added to the corresponding pixel.
[0060] In some embodiments, the second normal distribution is a normal distribution with a mean of 0.05 electrons / second and a standard deviation of 0.02 electrons / second.
[0061] For each clean image, a synthetic noise model is applied to generate an image with a specific light intensity.
[0062] The quality of the generated image is evaluated, including image clarity, color authenticity, noise distribution, etc. If problems are found, the noise parameters are adjusted in time. The noise parameters are adjusted through multiple trials to ensure that the generated image visually meets the characteristics of low light intensity.
[0063] Among them, whether the noise parameters need to be adjusted is determined based on the visual effect of the generated image. For example, in terms of image clarity, if the generated image is too blurry or has abnormal blurry areas, it may be that the noise has excessively affected the image details, and the noise parameters need to be adjusted; in terms of color authenticity, if the image color has abnormal deviations or distortions, it may be that the noise has an unreasonable impact on the color signal, and the noise parameters need to be adjusted; in terms of noise distribution, if it is observed that the noise distribution in the image does not meet the noise characteristics that should be expected under low light intensity (such as the noise is too concentrated or unevenly dispersed, etc.), the noise parameters also need to be adjusted to ensure that the generated image can visually reasonably simulate the characteristics of real images under low light intensity.
[0064] Repeat the above steps to generate 5000 images of different scenes.
[0065] Embodiment 2:
[0066] This embodiment discloses a method for adding synthetic noise to generate data sets with different illumination intensity ranges, including:
[0067] Collect 2,000 clean images of different scenes, covering different lighting conditions of urban streets during the day and night, lighting changes of rural roads in different seasons, etc.
[0068] Set the light intensity range to 0.01 lux to 0.1 lux, and analyze the characteristics and impact of various noises within this light intensity range.
[0069] Design a synthetic noise model, combine the experience of Example 1, and further optimize the selection of noise parameters. For readout noise and thermal noise, determine their distribution parameters based on actual measured data, and perform reasonable simulations. Specifically:
[0070] The synthetic noise model simulates the readout noise by generating a third random value obeying a third normal distribution for each pixel of the clean image; and converts the readout noise into a brightness value and adds it to the corresponding pixel.
[0071] In some embodiments, the third normal distribution is a normal distribution with a mean of 0 and a standard deviation of 0.1.
[0072] The synthetic noise model also calculates the thermal noise intensity according to the thermodynamic formula Nt=K×T×Δf; wherein Nt represents the thermal noise intensity, and K is the Boltzmann constant (1.38×10 -23 J / K), T is the detector temperature (set to 300K), Δf is the bandwidth (set to 100 Hz); the thermal noise intensity is converted to a brightness value and added to the corresponding image.
[0073] Clean images are processed one by one, and synthetic noise models are applied to generate images with specific light intensity. During the generation process, attention is paid to maintaining the details and overall structure of the image to avoid excessive noise affecting the image quality.
[0074] The steps are repeated to generate 8,000 images of different scenes, and strict quality control and screening are performed to ensure the quality of the dataset.
[0075] Embodiment three:
[0076] This embodiment discloses a method for adding synthetic noise to generate data sets with different illumination intensity ranges, including:
[0077] Collect 3,000 clean images of different scenes, including natural landscapes with different lighting changes at different time periods, objects in indoor low-light environments, etc.
[0078] Set the illumination intensity range to 0.1 lux to 1 lux. Considering the relatively good illumination conditions within this illumination intensity range, adjust the synthetic noise model appropriately and consider the interaction of various noise factors. For photon noise, adjust the distribution of random values of brightness increase according to the change of illumination intensity. For dark current noise and other noise, make corresponding adjustments. Make the generated image more consistent with the real image under different illumination intensities.
[0079] Among them, adjusting the noise parameters includes adjusting the distribution of the random value of the photon noise brightness increase related to the change in light intensity, as well as the corresponding adjustment of the dark current noise, readout noise, and thermal noise. For example, under relatively good lighting conditions with a light intensity range of 0.1 lux to 1 lux, the standard deviation of the random value of the photon noise brightness increase may be appropriately reduced to make the pixel brightness change more consistent with the actual situation under the light intensity; for dark current noise, etc., adjust its parameters such as the mean or standard deviation according to the change in light intensity to optimize the quality and authenticity of the generated image.
[0080] The synthetic noise model is applied to generate images, and the generated images are post-processed, such as adjusting contrast, color balance, etc., to improve the quality and visual effect of the images.
[0081] Among them, adjusting contrast can be achieved by changing the brightness distribution of image pixels, for example, using a contrast stretching algorithm to expand or compress the pixel values of the darker and brighter parts of the image to enhance the layering of the image. Adjusting color balance can be achieved by performing different degrees of gain or offset operations on the red, green, and blue color channels of the image to make the color of the image more consistent with the visual perception of the human eye or the color characteristics of the actual scene. The specific algorithm and parameter adjustment need to be determined based on the actual situation of the image and the expected effect.
[0082] Repeat the above steps to generate 10,000 images of different scenes. Organize and classify the entire data set to facilitate subsequent model training and application.
[0083] Embodiment 4
[0084] This embodiment provides an electronic device, including: a memory and a processor;
[0085] The memory is used to store computer programs;
[0086] The processor is used to call the computer program to execute the methods described in embodiments 1 to 3.
[0087] Embodiment 5
[0088] This embodiment provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed on an electronic device, the electronic device implements the methods described in Embodiments 1 to 3.
[0089] Embodiment 6
[0090] This embodiment provides a computer program product, including a computer program. When the computer program is executed on an electronic device, the electronic device implements the methods described in Embodiments 1 to 3.
[0091] The specific implementation methods of a system, electronic device, computer-readable storage medium, and computer program product provided in the embodiments of the present application can refer to the specific embodiments of the above-mentioned method and will not be repeated here.
[0092] Obviously, those skilled in the art should understand that the above-mentioned units or steps of the present application can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the present application is not limited to any specific combination of hardware and software.
[0093] The above description of the embodiments of the present application is only a partial embodiment of the present application, which is used to enable professionals in the field to implement or use the content of the present application, and is not used to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.
Claims
1. A method for generating data sets with different light intensity ranges by adding synthetic noise, characterized in that: include: Collect a large number of clean images; For the clean image, various types of noise are calculated respectively according to different light intensities; The various types of noise calculated are converted into brightness values and added to the corresponding pixels of the clean image to generate an image with a specific light intensity; The images generated with specific light intensity constitute a data set.
2. The method according to claim 1, characterized in that The various types of noise include: photon noise; For the clean image, photon noise is calculated according to different light intensities, and the calculated photon noise is converted into a brightness value, including: The expected photon arrival rate is calculated according to the formula R = I × QE × t; where I is the light intensity, QE is the quantum efficiency of the detector, and t is the exposure time; A random number generator is used to generate a random number; a certain number of photon events are generated for each pixel of an input clean image according to the generated random number and the calculated photon arrival rate; each photon event increases the brightness of the corresponding pixel by a first random value; the first random value obeys a first normal distribution.
3. The method according to claim 1, characterized in that The various types of noise include: dark current noise; For the clean image, dark current noise is calculated according to different light intensities, and the calculated dark current noise is converted into a brightness value, including: Determine the dark current noise parameter under a specific light intensity, assuming that it obeys the second normal distribution; For each pixel of the clean image, a second random value obeying a second normal distribution is generated for each pixel of the input clean image to simulate the dark current noise Nd; The dark current noise Nd is converted into a brightness value and added to the corresponding pixel.
4. The method according to claim 1, characterized in that The various types of noise include: read noise; For the clean image, the readout noise is calculated, and the calculated readout noise is converted into a brightness value, including: Determine the readout noise parameter and assume it obeys the third normal distribution; For each pixel of the clean image, respectively, a third random value obeying a third normal distribution is generated to simulate the readout noise; The readout noise is converted to brightness values and added to the corresponding pixels.
5. The method according to claim 1, characterized in that The various types of noise include: thermal noise; For the clean image, thermal noise is calculated, and the calculated thermal noise is converted into a brightness value, including: The thermal noise intensity is calculated according to the thermodynamic formula Nt=K×T×Δf; where Nt represents the thermal noise intensity, K is the Boltzmann constant, T is the detector temperature, and Δf is the bandwidth; Convert thermal noise intensity to brightness value and add it to the corresponding image.
6. The method according to claim 1, characterized in that include: The clean images include images of different scenes, different lighting conditions and different objects.
7. The method according to claim 1, characterized in that The light intensity ranges from 0.0001 lux to 1 lux.
8. An electronic device, characterized in that: include: Memory and processor; The memory is used to store computer programs; The processor is configured to call the computer program to execute the method according to any one of claims 1 to 7.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on an electronic device, the electronic device implements the method according to any one of claims 1 to 7.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed on an electronic device, the electronic device implements the method according to any one of claims 1 to 7.