Image generation method, apparatus, device, and storage medium

Abstract

The present application belongs to the technical field of image processing, and discloses an image generation method, device, equipment and storage medium. According to the statistical brightness information corresponding to the statistical image contained in the current frame image, the control parameter adopted by the current frame image is adjusted to the target control parameter, and when the adjusted statistical brightness information does not satisfy the preset condition, the state of the light filtering assembly is adjusted, the light signal is generated according to the target control parameter and the state of the light filtering assembly, the image signal of the first preset number of frames after the current frame is acquired, the digital gain and compression processing of the image signal of the first preset number of frames after the current frame are carried out based on the target control parameter, the target image of the first preset number of frames after the current frame is obtained, the light filtering device with the function of reducing the amount of light is used to avoid the flicker phenomenon caused by the reduction of the exposure time, the control parameter of the image acquisition and processing is adjusted by using the statistical brightness information, and the result image with appropriate brightness and appropriate dynamic range can be obtained under different scenes.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image generation method, apparatus, device, and storage medium. Background Technology

[0002] The current image processing method involves acquiring a first image; determining whether the scene captured by the imaging device is a high dynamic range (HMR) scene based on the first image; if the scene is determined to be an HMR scene, reducing the exposure of the imaging device; and performing local brightness correction on the reference image acquired after reducing the exposure. This method can directly utilize the pixel statistics data of the imaging device and does not require long-short frame exposure fusion, resulting in less computation and saving CPU power. Furthermore, it can prevent or mitigate overexposure without relying on high dynamic range sensors, saving costs. However, this method has problems: reducing the brightness of the captured image by reducing the exposure of the imaging device cannot avoid flickering; performing local brightness correction on the reference image acquired after reducing the exposure makes it difficult to guarantee the relative brightness relationship between different parts of the same scene; and it does not establish the relationship between brightness adjustment and dynamic range adjustment, making it difficult to guarantee the brightness stability and dynamic range adaptability of the resulting image.

[0003] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main objective of this invention is to provide an image generation method, apparatus, device, and storage medium, which aims to solve the technical problems of existing technologies that cannot avoid flickering when obtaining underexposed images by reducing exposure time, and that it is difficult to guarantee the brightness stability and dynamic range adaptability of the generated images.

[0005] To achieve the above objectives, the present invention provides an image generation method, the method comprising the following steps:

[0006] The control parameters used in the current frame image are adjusted according to the statistical brightness information corresponding to the statistical image contained in the current frame image to obtain the target control parameters, and the adjusted statistical brightness information is obtained according to the target control parameters. The control parameters are calculated from the preset number of frames before the current frame. The target control parameters are used to control the acquisition and processing of the preset number of frames after the current frame.

[0007] If the adjusted statistical brightness information does not meet the preset conditions, the state of the filter component is adjusted until the adjusted statistical brightness information meets the preset conditions. The filter component is used to adjust the amount of light entering the image during image acquisition.

[0008] Based on the target control parameters and the state of the filter component, light signals are acquired, and a preset number of frame image signals are generated after the current frame based on the light signals.

[0009] Based on the target control parameters, digital gain processing and compression processing are performed on the image signal of the preset number of frames after the current frame to obtain the target image of the preset number of frames after the current frame. The pixel value range of the target image is the target pixel value range. The pixel value range of the image after digital gain processing is not less than the pixel value range of the image before digital gain processing. The target pixel value range is preset based on the application requirements of the current frame image.

[0010] Optionally, the step of performing digital gain processing and compression processing on the image signal of the preset number of frames after the current frame based on the target control parameters to obtain the target image of the preset number of frames after the current frame includes:

[0011] The image signal of the preset number of frames following the current frame is preprocessed to obtain a preprocessed image;

[0012] Based on the target processing control parameters, the preprocessed image is subjected to a gain processing step to obtain a reference image;

[0013] The reference image is compressed to obtain the target image of the preset number of frames after the current frame.

[0014] Optionally, the step of performing digital gain processing and compression processing on the image signal of the preset number of frames after the current frame based on the target control parameters to obtain the target image of the preset number of frames after the current frame includes:

[0015] Based on the target processing control parameters, the image signal of the preset number of frames after the current frame is subjected to gain processing once to obtain the first reference image;

[0016] The first reference image is compressed to obtain the second reference image;

[0017] The second reference image is post-processed to obtain the target image of the preset number of frames after the current frame.

[0018] Optionally, the step of performing digital gain processing and compression processing on the image signal of the preset number of frames after the current frame based on the target control parameters to obtain the target image of the preset number of frames after the current frame includes:

[0019] The image signal of the preset number of frames following the current frame is preprocessed to obtain a preprocessed image;

[0020] The preprocessed image is subjected to a gain processing step according to the target processing control parameters to obtain a first reference image;

[0021] The first reference image is compressed to obtain the second reference image;

[0022] The second reference image is post-processed to obtain the target image of the preset number of frames after the current frame.

[0023] Optionally, the step of performing digital gain processing and compression processing on the image signal of the preset number of frames after the current frame based on the target control parameters to obtain the target image of the preset number of frames after the current frame includes:

[0024] A first gain processing control parameter and a second gain processing control parameter are determined based on the target processing control parameter, and the product of the first gain processing control parameter and the second gain processing control parameter is equal to the target processing control parameter.

[0025] Based on the first gain processing control parameter, the image signal of the preset number of frames after the current frame is subjected to the first gain processing to obtain the first reference image;

[0026] The first reference image is compressed to obtain the second reference image;

[0027] The second reference image is processed to obtain the third reference image;

[0028] The third reference image is subjected to a second gain processing based on the second gain processing control parameters to obtain a fourth reference image;

[0029] The fourth reference image is compressed to obtain the target image of the preset number of frames after the current frame, wherein the compressed images have the same range of brightness values.

[0030] Optionally, the gain processing may not truncate the bit width or may truncate the bit width to a preset brightness value range, wherein the preset brightness value range is not less than the brightness value range of the image signal of the preset number of frames after the current frame, and the compression parameters of the data compression are related to the gain of the gain processing.

[0031] Optionally, the statistical brightness information includes statistical brightness values ​​and dynamic range expansion values, and the step of adjusting the control parameters used in the current frame image based on the statistical brightness information corresponding to the statistical image contained in the current frame image to obtain the target control parameters includes:

[0032] The target control parameters are obtained by adjusting the control parameters used in the current frame image based on the statistical brightness value and the dynamic range extension value.

[0033] Optionally, the step of adjusting the control parameters of the current frame image based on the statistical brightness value and the dynamic range extension value to obtain the target control parameters includes:

[0034] The control parameters used in the current frame image are adjusted based on the statistical brightness value to obtain the initial control parameters;

[0035] The initial control parameters are adjusted based on the dynamic range extension value to obtain the target control parameters.

[0036] Optionally, the step of adjusting the control parameters of the current frame image based on the statistical brightness value to obtain initial control parameters includes:

[0037] Compare the statistical brightness value with the preset brightness value;

[0038] Determine the direction of parameter adjustment based on the comparison results;

[0039] The control parameters used to adjust the direction of the current frame image according to the parameters are used to obtain the initial control parameters.

[0040] Optionally, the step of adjusting the control parameters used to adjust the direction of the current frame image according to the parameters to obtain the initial control parameters includes:

[0041] When the parameter adjustment direction is upward, if the adjusted control parameter is greater than the corresponding first parameter threshold, then the first parameter threshold is used as the adjusted control parameter, and the next control parameter is adjusted according to the parameter adjustment direction.

[0042] Return to the step of if the adjusted control parameter is greater than the corresponding first parameter threshold, then take the first parameter threshold as the adjusted control parameter, and continue to adjust the next control parameter according to the parameter adjustment direction to obtain the initial control parameter;

[0043] When the parameter adjustment direction is downward, if the adjusted control parameter is less than the corresponding second parameter threshold, then the second parameter threshold is used as the adjusted control parameter, and the next control parameter is adjusted according to the parameter adjustment direction.

[0044] If the adjusted control parameter is less than the corresponding second parameter threshold, then the second parameter threshold is used as the adjusted control parameter, and the next control parameter is adjusted according to the parameter adjustment direction to obtain the initial control parameter, wherein the first parameter threshold is greater than the second parameter threshold.

[0045] Optionally, when the parameter adjustment direction is upward, the next control parameter is determined by the first parameter adjustment order; when the parameter adjustment direction is downward, the next control parameter is determined by the second parameter adjustment order, wherein the first parameter adjustment order is the opposite of the second parameter adjustment order.

[0046] Optionally, adjusting the initial control parameters according to the dynamic range extension value to obtain the target control parameters includes:

[0047] The initial digital gain parameters are obtained based on the initial control parameters.

[0048] When the initial digital gain parameter is less than the dynamic range extension value, the initial digital gain parameter is adjusted based on the dynamic range extension value to obtain the target digital gain parameter, and other current initial control parameters other than the initial digital gain parameter are reduced.

[0049] If the reduced other initial control parameters are less than the corresponding parameter threshold, then the parameter threshold is used as the adjusted control parameter, and the next other initial control parameter corresponding to the current other initial control parameter is further reduced;

[0050] Return to the step of reducing the next initial control parameter corresponding to the current initial control parameter if the reduced initial control parameter is less than the corresponding parameter threshold, so as to obtain the target control parameter corresponding to the other initial control parameter. The target control parameter includes the target digital gain parameter and the target control parameter corresponding to the other initial control parameter.

[0051] Optionally, if the adjusted statistical brightness information does not meet the preset conditions, the state of the filter component is adjusted until the adjusted statistical brightness information meets the preset conditions, including:

[0052] If the adjusted statistical brightness value is still greater than the preset brightness value, the filter component is adjusted to the target state until the adjusted statistical brightness value reaches the preset brightness value. The target state is a first state, a combination of the first and second states, or a combination of the first and third states. The first state is a low light transmittance state, the second state is a state that allows visible light to pass through, and the third state is a state that allows both visible and infrared light to pass through.

[0053] Furthermore, to achieve the above objectives, the present invention also proposes an image generation apparatus, the image generation apparatus comprising:

[0054] The adjustment module is used to adjust the control parameters used in the current frame image according to the statistical brightness information corresponding to the statistical image contained in the current frame image, to obtain the target control parameters, and to obtain the adjusted statistical brightness information according to the target control parameters. The control parameters are calculated from the preset number of frames before the current frame. The target control parameters are used to control the acquisition and processing of the preset number of frames after the current frame.

[0055] The adjustment module is also used to adjust the state of the filter component if the adjusted statistical brightness information does not meet the preset conditions, until the adjusted statistical brightness information meets the preset conditions. The filter component is used to adjust the amount of light entering the image during image acquisition.

[0056] The processing module is used to acquire optical signals based on the target control parameters and the state of the filter component, and generate a preset number of frame image signals after the current frame based on the optical signals;

[0057] The processing module is further configured to perform digital gain processing and compression processing on the image signal of the preset number of frames after the current frame based on the target control parameters, to obtain the target image of the preset number of frames after the current frame. The pixel value range of the target image is the target pixel value range. The pixel value range of the image after digital gain processing is not less than the pixel value range of the image before digital gain processing. The target pixel value range is preset based on the application requirements of the current frame image.

[0058] Furthermore, to achieve the above objectives, the present invention also proposes an image generation device, the image generation device comprising: a memory, a processor, and an image generation program stored in the memory and executable on the processor, the image generation program being configured to implement the steps of the image generation method as described above.

[0059] Furthermore, to achieve the above objectives, the present invention also proposes a storage medium storing an image generation program, which, when executed by a processor, implements the steps of the image generation method described above.

[0060] This invention adjusts the control parameters of the current frame image to target control parameters based on the statistical brightness information corresponding to the statistical image contained in the current frame image. When the adjusted statistical brightness information does not meet the preset conditions, the state of the filter component is adjusted. The light signal collected based on the target control parameters and the state of the filter component generates a preset number of frame image signals after the current frame. Based on the target control parameters, the preset number of frame image signals after the current frame are subjected to digital gain processing and compression processing to obtain the target image of the preset number of frames after the current frame. By using a filter device that can reduce the amount of light entering the image, the flicker phenomenon caused by the reduction of exposure time is avoided. By adjusting the control parameters of image acquisition and processing using the statistical brightness information of the image, it is possible to obtain result images with good quality, appropriate brightness, and appropriate dynamic range in different scenarios. Attached Figure Description

[0061] Figure 1 This is a schematic diagram of the structure of the image generation device in the hardware operating environment involved in the embodiments of the present invention;

[0062] Figure 2 This is the overall workflow framework for image acquisition and processing in the image generation method of this invention;

[0063] Figure 3 This is a schematic diagram of the image acquisition unit in the image generation method of the present invention;

[0064] Figure 4 This is a schematic diagram of the image processing unit in the image generation method of the present invention;

[0065] Figure 5 This is a schematic diagram of the statistical unit structure in the image generation method of the present invention;

[0066] Figure 6 This is a schematic diagram of the exposure control unit in the image generation method of the present invention;

[0067] Figure 7 This is a schematic flowchart of the first embodiment of the image generation method of the present invention;

[0068] Figure 8 This is a flowchart illustrating the second embodiment of the image generation method of the present invention;

[0069] Figure 9 This is a schematic diagram of the preprocessing module in one embodiment of the image generation method of the present invention;

[0070] Figure 10 This is a schematic diagram of the post-processing module in one embodiment of the image generation method of the present invention;

[0071] Figure 11 This is a schematic diagram showing the addition of a preprocessing module and a postprocessing module in one embodiment of the image generation method of the present invention;

[0072] Figure 12 This is a schematic diagram showing the addition of an intermediate module, a second gain processing module, and a second data compression module in one embodiment of the image generation method of the present invention.

[0073] Figure 13 This is a flowchart illustrating the third embodiment of the image generation method of the present invention;

[0074] Figure 14 This is a structural block diagram of the first embodiment of the image generation device of the present invention.

[0075] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0076] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0077] Reference Figure 1 , Figure 1 This is a schematic diagram of the image generation device structure in the hardware operating environment involved in the embodiments of the present invention.

[0078] like Figure 1 As shown, the image generation device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0079] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the image generation device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0080] like Figure 1As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and an image generation program.

[0081] exist Figure 1 In the image generation device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the image generation device of the present invention can be set in the image generation device, and the image generation device calls the image generation program stored in the memory 1005 through the processor 1001 and executes the image generation method provided in the embodiment of the present invention.

[0082] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0083] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0084] It should be understood that although the terms first, second, third, etc., may be used herein to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this document, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if," as used herein, may be interpreted as "when," "when," or "in response to determination." Furthermore, as used herein, the singular forms "a," "an," and "the" are intended to also include the plural forms unless the context indicates otherwise. It should be further understood that the terms "comprising," "including," indicate the presence of the stated feature, step, operation, element, component, item, kind, and / or group, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms "or," "and / or," "including at least one of the following," etc., as used in this application, may be interpreted as inclusive, or mean any one or any combination thereof. For example, "including at least one of the following: A, B, C" means "any one of the following: A; B; C; A and B; A and C; B and C; A and B and C." Similarly, "A, B, or C" or "A, B, and / or C" means "any one of the following: A; B; C; A and B; A and C; B and C; A and B and C." Exceptions to this definition only occur when the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.

[0085] It should be understood that although the steps in the flowcharts of this application's embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0086] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0087] It should be noted that step designations such as S10 and S20 are used in this document for the purpose of more clearly and concisely describing the corresponding content, and do not constitute a substantial limitation on the order. In specific implementation, those skilled in the art may execute S20 first and then S10, etc., but these should all be within the protection scope of this application.

[0088] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0089] This application proposes an image generation method, which may involve the following concepts:

[0090] Image dynamic range: The range of brightness differences between the darkest and brightest parts of an image that it can display.

[0091] Exposure control: The brightness of the captured image is controlled by adjusting the aperture size of the lens, the exposure time (shutter speed) of the image sensor, gain, etc.

[0092] Digital gain: Gain methods used in digital signal processing are usually achieved by performing arithmetic operations on digital signals, such as multiplication.

[0093] Analog gain: Gain methods used in analog signal processing are usually achieved by changing the current or voltage of components in a circuit, such as transformers or amplifiers.

[0094] Stroboscopic phenomenon: The stripe phenomenon produced by the light source in the captured image due to the interaction between the frequency of the light source and the exposure time of the image sensor.

[0095] Overall workflow framework for image acquisition and processing: Refer to Figure 2 As shown. The image acquisition unit acquires light signals, converts them into image signals, and outputs a first image signal to the image processing unit. The image processing unit receives the first image signal, processes it according to application requirements, and outputs a first output image; it also outputs a first statistical image and a second statistical image to the statistics unit according to statistical requirements. The statistics unit receives the first and second statistical images, performs dynamic range statistics and brightness statistics respectively, and outputs statistical information to the exposure control unit. The first and second statistical images can be the same image or different images. The exposure control unit receives the statistical information and outputs the exposure control parameters and processing control parameters for the Kth frame. The buffer unit outputs the exposure control parameters for the KNth (N≥1)th frame to the image acquisition unit for the exposure of the Kth frame image; it also outputs the processing control parameters for the KNth frame to the image processing unit for the processing of the Kth frame image. Simultaneously, it buffers the input exposure control parameters and processing control parameters for the Kth frame.

[0096] The specific composition of the image acquisition unit is as follows: Figure 3 As shown, the image acquisition unit includes a lens that receives and focuses light to form an image, and the amount of light entering can be controlled by an aperture. The image acquisition unit includes a filter module that controls the spectral range received by the image sensor and has a switching device capable of switching to a first state that reduces the transmittance of incident light. The filter module can also switch to a second state that allows only visible light to pass through. Furthermore, the filter module can switch to a third state that allows both visible and infrared light to pass through. It can also switch to a combination of the first and second states, or a combination of the first and third states. The light signal is converted into a first image signal and transmitted to the image processing unit. Each pixel in the image sensor can sense at least one of the three visible light sources: red, green, and blue, or simultaneously sense multiple visible light sources.

[0097] The specific structure of the image processing unit is as follows: Figure 4As shown, the image processing unit includes a gain processing module and a data compression module. The maximum data value B2 of the second image (the output image of the gain processing module) is not less than the maximum data value IB1 of the first image signal. The maximum data value OB1 of the first output image (the output image of the data compression module) is not greater than the maximum data value B2 of the second image. When the gain value of the gain processing module is too large, in order to avoid the data bit width of the intermediate processing module being too high, the gain can be allocated. At this time, the image processing unit needs to add a second gain processing module and a second data compression module. The maximum data value B2 of the second image is not less than the maximum data value IB1 of the first image signal. The gain processing module can be combined with the data compression module, such as simultaneously achieving image data gain and data compression through nonlinear compression. Optionally, a conventional image processing module can be included as an intermediate processing module, such as white balance, interpolation, contrast enhancement, etc., which can be selected according to the resources of the hardware platform or application requirements. Optionally, a preprocessing module can be added before the gain processing module, which may include, but is not limited to, one or more of noise reduction, white balance, and deblurring. Optionally, a postprocessing module can be added after the data compression module, which may include, but is not limited to, one or more of noise reduction, interpolation, enhancement, and deblurring. The image unprocessed by the gain processing module is output as the first statistical image to the statistics unit. This first statistical image can be the first image signal output by the image acquisition unit or a pre-processed image processed by the pre-processing module. The image processed by the gain processing module can be output as the second statistical image to the statistics unit. The image processed by the data compression module can also be output as the second statistical image to the statistics unit. The post-processed image processed by the post-processing module can also be output as the second statistical image to the statistics unit. Further, the gain processing module performs gain processing on the input first image signal and outputs the processed second image. Gain processing refers to multiplying the input first image signal by a certain digital gain dg. The digital gain dg is provided by the exposure control unit. No data truncation is performed after gain processing. The maximum value B2 of the output second image after gain processing is not less than the maximum value IB1 of the input first image signal. When the digital gain dg is greater than the preset gain threshold dg_max, a portion of the digital gain needs to be allocated to the second gain processing module. In this case, the digital gain of the gain processing module is dg1 = dg_max, and the digital gain of the second gain processing module is dg2 = dg / dg_max. The data compression module is used to compress the data of the second image and output the processed first output image. The maximum data value OB1 of the first output image is no greater than the maximum data value B2 of the second image. Data compression can be performed by mapping the second image using a preset nonlinear mapping curve to obtain the mapped first output image. The nonlinear mapping curve is related to the digital gain dg of the gain processing module.Correlation refers to the fact that different digital gains dg result in different maximum values ​​B2 of the second image. Nonlinear mapping curves need to compress high dynamic range images with different value ranges into the same value range.

[0098] The specific composition of a statistical unit is as follows: Figure 5 As shown, the statistical unit includes a dynamic range statistics module and a brightness statistics module, outputting dynamic range expansion values ​​and statistical brightness values ​​to the exposure control unit. The dynamic range statistics module performs dynamic range statistics on the first statistical image to obtain an initial dynamic range expansion value. The initial dynamic range expansion value is related to the brightness distribution of the bright areas in the first statistical image. When the initial dynamic range expansion value is greater than a preset dynamic range expansion threshold, the dynamic range expansion value dr_rat is the smaller of the initial dynamic range expansion value and the dynamic range expansion threshold. The preset dynamic range expansion threshold controls the degree of noise amplification caused by digital gain within a certain range. The preset dynamic range expansion threshold can be set according to the computing power of the logic platform, the level of noise reduction processing, etc. The dynamic range expansion value is not less than 1. The dynamic range expansion value determines the maximum value range that can be obtained from the acquired high dynamic range image. The brightness statistics module performs brightness statistics on the second statistical image to obtain a statistical brightness value. The statistical brightness value is related to the brightness distribution of each region in the second statistical image.

[0099] The specific composition of the exposure control unit is as follows: Figure 6 As shown, the exposure control unit includes a first exposure control module and a second exposure control module. The first exposure control module compares the input statistical brightness value with the preset target brightness and performs frame exposure control based on the frame exposure control parameter EPO used during the acquisition and processing of the current frame image. K-N and frame processing control parameter PRO K-N (K represents the frame number of the current frame image, N≥1, KN represents the frame numbers of the historical frames image) Perform the first adjustment process to obtain the initial value of the exposure control parameter EPO for the Kth frame. Kini The initial value of the processing control parameter PRO for the Kth frame. Kini The preset target brightness refers to the brightness value that makes the image more suitable for subsequent applications. The second exposure control module, based on the input dynamic range extension value and the initial value of the Kth frame exposure control parameter EPO... Kini and the initial value of the processing control parameter PRO for the Kth frame Kini Perform a second adjustment process to obtain the exposure control parameter EPO for the Kth frame. K and the processing control parameter PRO for the Kth frame K The adjustment process includes two aspects: adjustment direction and adjustment step size. The first exposure control module, during its initial adjustment process, adjusts the direction based on EPO. K-N Increase or decrease the value to obtain the initial value of the exposure control parameter EPO in the Kth frame.Kini When the first exposure control module performs its initial adjustment, its adjustment direction includes adjustments based on PRO. K-N Increase or decrease the value to obtain the initial value PRO of the processing control parameter in the Kth frame. Kini When the first exposure control module performs its initial adjustment, its adjustment step size refers to the amount of change used in the adjustment direction. This can be set as a fixed step size or a dynamically changing step size. A fixed step size can be set according to application requirements. A dynamically changing step size can be set based on the difference between the statistical brightness value and the preset target brightness; the greater the difference, the larger the dynamically changing step size. The initial value of the exposure control parameter EPO for the Kth frame is... Kini Includes, but is not limited to, the initial value of the sensor gain parameter sg Kini Initial aperture parameter value lr Kini Initial value of exposure time parameter t Kini And the filter switching signal. During the first exposure control module's initial adjustment process, the filter switching signal is in either the second or third state, and its switching is adjusted according to application requirements. The initial value of the control parameter PRO for the Kth frame processing. Kini Includes, but is not limited to, the initial value of the digital gain parameter dg of the gain processing module. Kini When the statistical brightness value is less than the preset target brightness, based on EPO... K-N and PRO K-N Increase the size to obtain EPO Kini and PRO Kini This ensures that the statistical brightness value after the first adjustment is close to the preset target brightness. When the statistical brightness value is less than the preset target brightness, the preferred parameter adjustment order is exposure time, aperture, sensor gain, and digital gain. When the statistical brightness value is greater than the preset target brightness, based on EPO... K-N and PRO K-N Reduce to obtain EPO Kini and PRO Kini This ensures that the statistical brightness value after the first adjustment is close to the preset target brightness. When the statistical brightness value is greater than the preset target brightness, the preferred parameter adjustment order is digital gain, sensor gain, aperture, and exposure time. The statistical brightness value after the first adjustment refers to the EPO value. Kini With EPO K-N Changes between and PRO Kini With PRO K-N The changes between these parameters collectively affect the statistical luminance value obtained. When the statistical luminance value equals the preset target luminance, no parameter adjustment is required, and EPO... Kini =EPO K-N PRO Kini =PRO K-N .

[0100] Second exposure control module, exposure control parameter EPO for frame K. K Includes, but is not limited to, the sensor gain parameter sg K Aperture parameters lr K Exposure time parameter t K Filter switching signals, etc. Kth frame processing control parameters PRO K Digital gain parameters dg, including but not limited to the gain processing module K When the second exposure control module performs the second adjustment process, EPO K =EPO Kini Prioritize PRO Kini The initial value of the digital gain parameter dg in Kini Make adjustments to obtain PRO. K The digital gain parameter dg in K The digital gain parameter dg K The input dynamic range extension value dr_rat and the initial value of the digital gain parameter dg Kini Take the larger value between them, i.e., dg K =max(dr_rat,dg Kini ). dg K Greater than dg Kini If the statistical brightness value after the second adjustment is greater than the statistical brightness value after the first adjustment, then the statistical brightness value after the second adjustment is greater than the statistical brightness value after the first adjustment. The statistical brightness value after the second adjustment refers to the value after adjusting the EPO... K With EPO Kini Changes between and PRO K With PRO Kini The changes between these factors collectively affect the luminance value obtained after the first adjustment. To ensure that the statistical luminance value after the second adjustment equals the statistical luminance value after the first adjustment, the EPO needs to be reduced. K The preferred parameter adjustment order is as follows: sensor gain, aperture, exposure time, and filter switching. The adjustment method is as follows: When Greater than or equal to the preset sensor gain lower limit threshold sg min At that time, the sensor gain should be adjusted first. Aperture parameters lr K =lr Kini Exposure time parameter t K =t Kini This ensures that the statistical brightness value after the second adjustment is equal to the statistical brightness value after the first adjustment. Less than the preset sensor gain lower limit threshold sg min At that time, prioritize adjusting the sensor gain to sg. K =sg min Then continue to reduce the aperture parameter lr K <lr Kini,like

[0101] Aperture parameters lr K If the statistical brightness value after the second adjustment is greater than or equal to the preset aperture threshold, the condition that the statistical brightness value after the first adjustment is equal to the statistical brightness value after the second adjustment is met. Therefore, there is no need to continue adjusting the exposure time parameter, i.e., t. K =t Kini Otherwise, the exposure time t needs to be further reduced. K <t Kini This ensures that the statistical brightness value after the second adjustment is equal to the statistical brightness value after the first adjustment. If the exposure time parameter t... K When the light transmittance is greater than or equal to the preset lower limit threshold, the condition that the statistical brightness value after the second adjustment is equal to the statistical brightness value after the first adjustment is met. Therefore, there is no need to adjust the filter switching device. Otherwise, the filter switching device needs to switch to a first state with lower light transmittance to meet the above condition. The filter switching can also be a combination of the first and second states, or a combination of the first and third states. The preset lower limit threshold can prevent flickering in the acquired image frames. K equals dg Kini If the statistical luminance value after the second adjustment is equal to the statistical luminance value after the first adjustment, then no adjustment of EPO is required. K The exposure control parameter EPO of the Kth frame. K and the processing control parameter PRO for the Kth frame K Output to the buffer unit.

[0102] The buffer unit is used to buffer the exposure control parameters and processing control parameters output by the exposure control unit for each frame. Before the exposure of the Kth frame begins, it buffers the exposure control parameters EPO of the KNth (N≥1)th frame. K-N The sensor receives the data to control the exposure of the Kth frame image. When the image processing unit starts processing the Kth frame image data, the KN processing control parameter PRO is input. K-N The parameters are passed to the image processing unit to apply to the Kth frame image data. The buffer unit stores the exposure control parameters EPO for the Kth frame. K and the processing control parameter PRO of the Kth frame K When caching, the exposure control parameter EPO of the KNth frame that has been passed to the image acquisition unit and image processing unit can be replaced. K-N and the processing control parameter PRO of frame KN K-N This reduces the consumption of storage space resources.

[0103] This invention provides an image generation method, referring to... Figure 7 , Figure 7 This is a schematic flowchart of a first embodiment of an image generation method according to the present invention.

[0104] In this embodiment, the image generation method includes the following steps:

[0105] Step S10: Adjust the control parameters used in the current frame image according to the statistical brightness information corresponding to the statistical image contained in the current frame image to obtain the target control parameters, and obtain the adjusted statistical brightness information according to the target control parameters.

[0106] It should be noted that the execution subject of this embodiment is an image generation device, which has functions such as data processing, data communication and program execution. The image generation device can be a terminal device such as a computer, or other devices with similar functions. This embodiment does not limit this.

[0107] It should be noted that the current image processing method involves acquiring a first image; determining whether the current scene captured by the imaging device is a high dynamic range (HMR) scene based on the first image; if the current scene is determined to be an HMR scene, reducing the exposure of the imaging device; and performing local brightness correction on the reference image acquired after reducing the exposure. This method can directly utilize the pixel statistics data of the imaging device and does not require long-short frame exposure fusion, resulting in less computation and saving CPU power. Furthermore, it can prevent or mitigate overexposure without relying on high dynamic range photosensitive elements, thus saving costs. However, this method has the problem that reducing the exposure of the imaging device to decrease the brightness of the captured image cannot avoid flickering. Performing local brightness correction on the reference image acquired after reducing the exposure makes it difficult to ensure the relative brightness relationship of different parts of the same scene, and it does not establish the relationship between brightness adjustment and dynamic range adjustment, making it difficult to guarantee the brightness stability and dynamic range adaptability of the resulting image.

[0108] To address the aforementioned technical issues, this embodiment adjusts the control parameters used in the current frame image to target control parameters based on the statistical brightness information corresponding to the statistical image contained in the current frame image. When the adjusted statistical brightness information does not meet preset conditions, the state of the filter component is adjusted. Based on the target control parameters and the state of the filter component, the light signal acquired generates a preset number of frame image signals after the current frame. Based on the target control parameters, the preset number of frame image signals after the current frame undergo digital gain processing and compression processing to obtain the target image. By utilizing a filter device that can reduce the amount of light entering the image, the flickering phenomenon caused by the reduction in exposure time is avoided. By adjusting the control parameters for image acquisition and processing using the statistical brightness information of the image, a result image with good quality, appropriate brightness, and suitable dynamic range can be obtained in different scenarios.

[0109] In this specific implementation, the light signal needs to be acquired first to obtain the current frame image signal, and then the statistical brightness information corresponding to the statistical image contained in the current frame image is obtained. The statistical image can be obtained from any process of image processing. The control parameters are used to control the acquisition and processing of the image. In this embodiment, the control parameters used for the current frame image are calculated from the preset number of frames before the current frame. The adjusted control parameters are the target control parameters. The target control parameters can be used as the control parameters corresponding to the preset number of frames after the current frame for the acquisition and processing of the preset number of frames after the current frame.

[0110] Step S20: If the adjusted statistical brightness information does not meet the preset conditions, adjust the state of the filter component until the adjusted statistical brightness information meets the preset conditions.

[0111] In specific implementation, after adjusting the target control parameters corresponding to the preset number of frames before the current frame used in the current frame image, the corresponding statistical brightness information will change accordingly. In this embodiment, it is necessary to further determine whether the adjusted statistical brightness information meets the preset conditions. If the statistical brightness information cannot meet the preset conditions, the state of the filter component will be further adjusted in this embodiment. The state of the filter component will affect the amount of light entering the image during image acquisition, and will ultimately affect the statistical brightness information of the image.

[0112] Step S30: Acquire optical signals based on the target control parameters and the state of the filter component, and generate a preset number of frame image signals after the current frame based on the optical signals.

[0113] In practice, the target control parameters include exposure control parameters and image processing control parameters. The exposure control parameters and the state of the filter components are used to control image acquisition. Light signals are acquired using the exposure control parameters and the state of the filter components, and then converted into image signals.

[0114] Step S40: Based on the target control parameters, perform digital gain processing and compression processing on the image signal of the preset number of frames after the current frame to obtain the target image of the preset number of frames after the current frame.

[0115] In specific implementation, the target control parameters are used to further process the image signal of the preset number of frames after the current frame, such as digital gain processing and compression processing, so as to obtain the target image. The target image is the result image that meets the requirements of brightness and dynamic range under different scenes. In this embodiment, the gain processing does not truncate the bit width or truncates to a preset brightness value range. The preset brightness value range is not less than the brightness value range of the image signal of the preset number of frames after the current frame. The compression parameters of the data compression are related to the gain of the gain processing.

[0116] It should be noted that gain processing refers to increasing the brightness range of the image signal to compensate for the brightness of the image. Specifically, it can be to adjust the brightness range of the image signal according to the gain value. The gain value can be preset according to actual needs or calculated based on the image signal. This embodiment does not limit this.

[0117] It should be understood that, compared to existing brightness compensation methods that truncate data exceeding the bit width during image gain processing, resulting in information loss in the image, especially for high dynamic range images where the loss is more pronounced, this embodiment does not truncate the bit width during gain processing. This allows the image to maintain a larger dynamic range, retain more brightness information, better maintain the brightness details and contrast in the image, and make the processed image closer to the real scene, thereby improving the visual effect of the image.

[0118] Alternatively, compared to the existing technology where bit width truncation involves cutting the gain-processed image to the brightness value range of the image signal, this embodiment truncates the gain processing to a preset brightness value range, which is larger than the brightness value range of the image signal. This reduces the loss of brightness information, retains more brightness information, and thus improves the visual effect of the image.

[0119] In a specific implementation, the brightness value range of the image signal is [IB', IB], and the preset brightness value range is [PB', PB]. A preset brightness value range greater than the brightness value range of the image signal can mean PB-PB'>IB-IB'. For example, the brightness value range of the image signal is [0, 1023], and the preset brightness value range is [0, 4095]. Of course, a preset brightness value greater than the brightness value range of the image signal can also mean PB>IB, and PB>IB'. However, in practical applications, since the lower limit of the brightness value range is fixed at 0, a preset brightness value range greater than the brightness value range of the image signal can also mean PB>IB. This embodiment does not impose any restrictions on this.

[0120] This embodiment adjusts the control parameters used in the current frame image to target control parameters based on the statistical brightness information corresponding to the statistical image contained in the current frame image. When the adjusted statistical brightness information does not meet the preset conditions, the state of the filter component is adjusted. The light signal collected based on the target control parameters and the state of the filter component generates the preset number of frame image signals after the current frame. Based on the target control parameters, the preset number of frame image signals after the current frame are subjected to digital gain processing and compression processing to obtain the target image of the preset number of frames after the current frame. By using a filter device that can reduce the amount of light entering the image, the flicker phenomenon caused by the reduction of exposure time is avoided. By adjusting the control parameters of image acquisition and processing using the statistical brightness information of the image, a result image with good quality, appropriate brightness and appropriate dynamic range can be obtained in different scenarios.

[0121] refer to Figure 8 , Figure 8 This is a schematic flowchart of a second embodiment of an image generation method according to the present invention.

[0122] Based on the first embodiment described above, in the image generation method of this embodiment, step S40 specifically includes:

[0123] Step S401: Perform preprocessing on the image signal of the preset number of frames after the current frame to obtain a preprocessed image.

[0124] Step S402: Perform a gain processing on the preprocessed image based on the target processing control parameters to obtain a reference image.

[0125] In practical implementation, a preprocessing module is added before the gain processing module, for example... Figure 9 As shown, it is used to process image signals. In this embodiment, the preprocessing includes, but is not limited to, one or more of noise reduction, white balance, and deblurring. Then, the preprocessed image is amplified to obtain a second image. In the gain processing, the bit width is not truncated or is truncated to a preset brightness value range. The preset brightness value range is greater than the brightness value range of the image signal. The statistical brightness of the second image reaches the target brightness.

[0126] Step S403: Compress the reference image to obtain the target image of the preset number of frames after the current frame.

[0127] In specific implementation, after the gain is completed, the reference image is finally compressed to obtain the final image of the preset number of frames after the current frame. It is important to emphasize that the compression parameters in this embodiment are related to the gain of the gain processing, and the reference image in this embodiment is... Figure 9 The second image shown is the target image of the preset number of frames after the current frame, i.e. Figure 9The first output image shown.

[0128] In an optional embodiment, a post-processing module can be added after the data compression module. The post-processing in this embodiment includes, but is not limited to, one or more of the following: noise reduction, enhancement, dehazing, deblurring, and sharpening. First, the image signal gain is processed according to the above method to obtain a first reference image. Then, the first reference image is compressed to obtain a second reference image. Finally, the second reference image is post-processed to obtain a preset number of frames after the current frame. For example... Figure 10 As shown, the first reference image in this embodiment is... Figure 10 The second image in the second reference image is... Figure 10 Post-processed images in the image.

[0129] In an optional embodiment, a preprocessing module and a postprocessing module may be added simultaneously, for example... Figure 11 As shown, preprocessing occurs before gain calculation, and postprocessing occurs after compression. Similar to the process described above, it will not be repeated here.

[0130] In an alternative embodiment, an intermediate processing module may also be added, for example... Figure 12 As shown, an intermediate processing module is added to the image processing unit. Intermediate processing includes, but is not limited to, one or more of noise reduction, white balance, and deblurring. To avoid resource increases in the intermediate processing module due to excessively high digital gain and large data bit width caused by the gain processing module, the digital gain can be split into two parts, adding a second gain processing module to the image processing unit. Additionally, a second data compression module is also needed. The compression parameters for compressing the second image are related to the first gain value. Specifically, different first gain values ​​can result in different brightness ranges for the second image. The compression parameters are adjusted according to these different brightness ranges to ensure the compressed third image has the same brightness range, thus reducing the data bit width. Gain processing is applied to the fourth image based on the second gain value to obtain the fifth image. This can be achieved by multiplying the fourth image by the second gain value. The compression parameters for compressing the fifth image are also related to the second gain value. Specifically, different second gain values ​​can result in different brightness ranges for the fifth image. The compression parameters are adjusted according to these different brightness ranges to ensure the compressed first output image has the same brightness range. Furthermore, the brightness range of the third image is no greater than the brightness range of the second image, the brightness range of the fourth image is equal to the brightness range of the third image, the brightness range of the fifth image is no less than the brightness range of the fourth image, and the brightness range of the first output image is no greater than the brightness range of the fifth image. In this embodiment, the first reference image is... Figure 12The second image in the image, the second reference image is... Figure 12 The third image in the image, the third reference image is... Figure 12 The fourth image in the image, the fourth reference image is... Figure 12 The fifth image in the sequence, the target image of the preset number of frames after the current frame is... Figure 12 The first output image in the process.

[0131] It should be understood that, in this embodiment, by limiting the brightness value range of the third image to no greater than the brightness value range of the second image, the bit width of the image before intermediate processing can be reduced, thereby reducing resource consumption. In this embodiment, by limiting the brightness value range of the fourth image to be equal to the brightness value range of the third image, the brightness value range of the images before and after intermediate processing can be ensured to remain unchanged, thus maintaining the stability of the image brightness value range. In this embodiment, by limiting the brightness value range of the fifth image to no less than the brightness value range of the fourth image, the image after secondary gain processing can be ensured to have a higher brightness range than the image signal after the first gain processing, ensuring that the gain effect of the gain processing after splitting is the same as that of the gain processing before splitting. In this embodiment, by limiting the brightness value range of the first output image to no greater than the brightness value range of the fifth image, the image after compression processing can be ensured to meet the bit width requirements and the brightness value range requirements of subsequent applications.

[0132] In the specific implementation, the brightness range of the second image is [B2', B2], the brightness range of the image signal is [IB', IB], and B2-B2'≥IB-IB'; the brightness range of the third image is [B3', B3], and B3-B3'≤B2-B2'; the brightness range of the fourth image is [B4', B4], and B4-B4'=B3-B3'; the brightness range of the fifth image is [B5', B5], and B5-B5'≥B4-B4'; and the brightness range of the first output image is [OB1', OB1], and OB1-OB1'≤B5-B5'. The above method can compress high dynamic range images of different data ranges into a unified data range. Compared with a single gain, the product of the digital gain values ​​corresponding to each gain processing in multiple gain processing is equal to the digital gain value corresponding to the first gain processing. For example, assuming the digital gain value of the first gain processing is DG, the digital gain value of the first gain processing is DG1, and the digital gain value of the second gain processing is DG2, then DG = DG1 * DG2.

[0133] This embodiment obtains high dynamic range (HDR) images by performing untruncated digital gain processing on a single frame image. This method can easily handle wide dynamic range scenes and achieve HDR image effects without requiring special hardware, thus reducing costs. It also enables images with different brightness levels to achieve a uniform target brightness, resulting in a more stable HDR image output. It can unify the processing of HDR images with different pixel value ranges, ensuring that the output image has a uniform pixel value range, simplifying subsequent processing. Furthermore, data compression ensures that the output image has a uniform data range, allowing the image processing unit to easily access different high bit-width data generation schemes, such as multi-frame wide dynamic range data. The scheme is simple and the effect is stable. When the gain is too large, the gain can be decomposed, with part of the gain processed in a second gain processing module, thus keeping the intermediate processing within a limited bit width and achieving a better balance between resources and effect.

[0134] refer to Figure 13 , Figure 13 This is a flowchart illustrating a third embodiment of an image generation method according to the present invention.

[0135] Based on the first embodiment described above, in the image generation method of this embodiment, step S10 specifically includes:

[0136] Step S101: Adjust the control parameters used in the current frame image according to the statistical brightness value and the dynamic range expansion value to obtain the target control parameters.

[0137] In this embodiment, the statistical brightness information includes statistical brightness values ​​and dynamic range expansion values. Specifically, the target control parameters corresponding to the preset number of frames of images preceding the current frame can be adjusted based on the statistical brightness values ​​and dynamic range expansion values ​​to obtain the target control parameters corresponding to the current frame.

[0138] In an optional embodiment, the target control parameters corresponding to the preset number of frames before the current frame can be adjusted according to the statistical brightness value to obtain the initial control parameters. Then, the initial control parameters can be adjusted according to the dynamic range expansion value to obtain the target control parameters corresponding to the current frame.

[0139] In specific implementation, the process of adjusting the target control parameters corresponding to the preset number of frames preceding the current frame image based on the statistical brightness value to obtain the initial control parameters involves comparing the statistical brightness value with the preset brightness value, determining the parameter adjustment direction based on the comparison result, and then adjusting the control parameters according to the parameter adjustment direction. There are three possible relationships between the statistical brightness value and the preset brightness value: when the statistical brightness value is less than the preset brightness value, the parameter adjustment direction is to increase the parameter value; when the statistical brightness value is greater than the preset brightness value, the parameter adjustment direction is to decrease the parameter value; and when the statistical brightness value is equal to the preset brightness value, no adjustment of the control parameters is required. The preset brightness value can be set according to actual needs, and this embodiment does not impose any restrictions on this.

[0140] Furthermore, after determining the parameter adjustment direction, in this embodiment, the target control parameters corresponding to the preset number of frames preceding the current frame are adjusted according to the parameter adjustment direction. If the adjusted control parameter is greater than the corresponding parameter threshold, then in this embodiment, the adjusted control parameter is first modified to the corresponding parameter threshold, and then the next control parameter is adjusted according to the parameter adjustment direction. The above process is repeated until all control parameters are adjusted. In this embodiment, the parameter adjustment order is related to the parameter adjustment direction, that is, the relationship between the statistical brightness value and the preset brightness value. For example, when the statistical brightness value is less than the preset brightness value, the parameter adjustment order is exposure time, aperture, sensor gain, and digital gain. When the statistical brightness value is greater than the preset brightness value, the parameter adjustment order is digital gain, sensor gain, aperture, and exposure time. Of course, adjustments can be made according to the actual situation, and this embodiment does not impose any restrictions on this.

[0141] It's important to emphasize that when the statistical brightness value is less than the preset brightness value, the parameter adjustment direction is upward. In this case, the corresponding first parameter threshold is the upper limit of the parameter. Conversely, when the statistical brightness value is greater than the preset brightness value, the parameter adjustment direction is downward. In this case, the corresponding second parameter threshold is the lower limit of the parameter. Therefore, the first parameter threshold is greater than the second parameter threshold. Furthermore, the parameter adjustment order is also related to the parameter direction. When the parameter adjustment direction is upward, the corresponding parameter adjustment order is the first parameter adjustment order; when the parameter adjustment direction is downward, the corresponding parameter adjustment order is the second parameter adjustment order. The first parameter adjustment order and the second parameter adjustment order are reversed. For example, the first parameter adjustment order is exposure time, aperture, sensor gain, digital gain; the second parameter adjustment order is digital gain, sensor gain, aperture, exposure time. For instance, when the statistical brightness value is less than the preset target brightness, the exposure time t is increased first. k =t k-1*y_dst / y_sta, where k represents the frame number, t k-1 t represents the target exposure time control parameter corresponding to the (K-1)th frame. k This indicates the adjusted exposure time parameter. Increasing the exposure time results in an adjusted statistical brightness value. The adjusted exposure time is equal to the preset target brightness, and the smaller of the adjusted exposure time and the preset exposure time upper limit threshold t_max is taken. This preset exposure time upper limit threshold prevents excessively long exposure times from causing motion blur in the acquired images, and also prevents the image sensor from capturing too few image frames per unit time, which would not meet application requirements. If t k If ≤ t_max, then no further adjustment is needed; the aperture, sensor gain, and digital gain remain unchanged, i.e., lr. k =lr k-1 sg k =sg k-1 dg k =dg k-1 If t k >t_max, then t k =t_max, and other parameters need further adjustment. When further adjustment is needed, first increase the aperture lr. k >lr k-1 Increase the amount of light entering the camera so that the adjusted statistical brightness value equals the preset target brightness, and then take the smaller of the adjusted aperture parameter and the preset aperture upper limit threshold lr_max. If lr k If ≤lr_max, then no further adjustment is needed; the sensor gain and digital gain remain unchanged, i.e., sg k =sg k-1 dg k =dg k-1 If lr k >lr_max, then lr k =lr_max, and other parameters need to be adjusted further. When further adjustments are needed, first increase the sensor gain sg. k >sg k-1 Further increase the adjusted statistical brightness value to make it equal to the preset target brightness, and then take the smaller of the adjusted sensor gain parameter and the preset sensor gain upper limit threshold sg_max. k If dg ≤ sg_max, then no further adjustment is needed; the digital gain remains unchanged. k =dg k-1 If sg k >sg_max, then sg k =sg_max, and other parameters need further adjustment. Finally, if further adjustments are needed, increase the digital gain dg. k >dg k-1, further increase the adjusted statistical brightness value to make it equal to the preset target brightness. Another example is when the statistical brightness value is greater than the preset target brightness, first reduce the digital gain to Reduce the digital magnification factor so that the adjusted statistical brightness value is equal to the preset target brightness, and take the maximum of the adjusted digital gain parameter and the preset digital gain lower threshold dg_min. If dg k ≥dg_min, there is no need to continue adjusting, and the sensor gain, aperture, and exposure time remain unchanged, that is, sg k =sg k-1 、lr k =lr k-1 、t k =t k-1 。If dg k <dg_min, then dg k =dg_min, and other parameters need to be adjusted continuously. When further adjustment is needed, first reduce the sensor gain to Reduce the sensor magnification factor so that the adjusted statistical brightness is equal to the preset target brightness, and take the maximum of the adjusted sensor gain parameter and the preset sensor gain lower threshold sg_min. If sg k ≥sg_min, there is no need to continue adjusting, and the aperture and exposure time remain unchanged, that is, lr k =lr k-1 、t k =t k-1 。If sg k <sg_min, then sg k =sg_min, and other parameters need to be adjusted continuously. When further adjustment is needed, first reduce the aperture lr k <lr k-1 , if the adjusted statistical brightness value can be made equal to the preset target brightness, there is no need to continue adjusting, and the exposure time remains unchanged, that is, t k =t k-1 。If the adjusted statistical brightness value is still greater than the preset target brightness when the aperture is reduced to the lower limit of the preset aperture parameter, other parameters need to be adjusted continuously. Finally, when further adjustment is needed, reduce the exposure time t k <t k-1 , so that the further adjusted statistical brightness value is equal to the preset target brightness.

[0142] In one optional embodiment, the process of adjusting the initial control parameters based on the dynamic range extension value to obtain the target control parameters involves obtaining an initial digital gain parameter based on the initial control parameters, then comparing the initial digital gain parameter with the dynamic range extension value. If the initial digital gain parameter is less than the dynamic range extension value, the initial control parameters are adjusted to obtain the target control parameters. If the initial digital gain parameter is greater than or equal to the dynamic range extension value, no adjustment of the initial control parameters is required.

[0143] In practical implementation, when the initial digital gain parameter is less than the dynamic range extension value, the process of adjusting the initial digital gain parameter involves using the dynamic range extension value as the target digital gain parameter to obtain the target digital gain parameter. After adjustment in this way, the adjusted statistical brightness value will be greater than the preset target brightness. To make the adjusted statistical brightness value equal to the preset target brightness, the exposure control parameters need to be lowered. The order of parameter adjustment is as follows: sensor gain, aperture, exposure time, and filter state switching. If other initial control parameters after reduction are less than the corresponding parameter threshold, the adjusted control parameters are modified to the corresponding parameter threshold, and the next other initial control parameter corresponding to the current other initial control parameter is further reduced. Here, the parameter threshold is the lower limit of the parameter. For example, when When the sensor gain is greater than or equal to the preset lower limit threshold, the sensor gain is adjusted first. The aperture and exposure time parameters use the initial values ​​provided by the first exposure control module, ensuring that the adjusted statistical brightness equals the preset target brightness. If the sensor gain is below the preset lower limit threshold, further adjustments are needed. Prioritize adjusting the sensor gain to sg. k =sg_min, where sg_min represents the preset lower limit threshold of sensor gain, and then the aperture parameter lr is reduced. k ' <lr k To reduce the amount of light entering the camera, if the adjusted statistical brightness value equals the preset target brightness, no further adjustment is needed, and the exposure time parameter uses the initial value given by the first exposure control module. If, when the aperture is reduced to the lower limit of the preset aperture parameter, the adjusted statistical brightness value is still greater than the preset target brightness, then the exposure time parameter needs further adjustment, reducing the exposure time t. k ' <t k This ensures that the adjusted statistical brightness value equals the preset target brightness.

[0144] Furthermore, if the adjusted statistical brightness value is still greater than the preset brightness value, the filter component is adjusted to the target state until the adjusted statistical brightness value reaches the preset brightness value. In this embodiment, the target state is the first state, a combination of the first and second states, or a combination of the first and third states. The first state is a low light transmittance state, the second state is a state that allows visible light to pass through, and the third state is a state that allows both visible and infrared light to pass through. For example, if the adjusted statistical brightness value is still greater than the preset target brightness when the exposure time is reduced to the preset lower limit threshold, the filter component needs to be switched. Based on the existing state (such as the second or third state), the first state is added to reduce the amount of light entering the filter, so that the further adjusted statistical brightness value is equal to the preset target brightness.

[0145] This embodiment obtains initial control parameters by adjusting the target control parameters corresponding to a preset number of frames preceding the current frame based on the statistical brightness value; it then adjusts the initial control parameters based on the dynamic range extension value to obtain target control parameters. By adjusting the control parameters according to different parameter adjustment directions and order, the final target control parameters corresponding to the current frame are obtained. The target control parameters obtained above can reduce noise amplification, avoid flickering, and simultaneously ensure that the subsequent output image has a stable effect and adaptive high dynamic range.

[0146] Furthermore, embodiments of the present invention also propose a storage medium storing an image generation program, which, when executed by a processor, implements the steps of the image generation method described above.

[0147] Reference Figure 14 , Figure 14 This is a structural block diagram of the first embodiment of the image generation device of the present invention.

[0148] like Figure 14 As shown, the image generation apparatus proposed in this embodiment of the invention includes:

[0149] The adjustment module 10 is used to adjust the control parameters used in the current frame image according to the statistical brightness information corresponding to the statistical image contained in the current frame image, to obtain target control parameters, and to obtain the adjusted statistical brightness information according to the target control parameters. The control parameters are calculated from the preset number of frame images before the current frame. The target control parameters are used to control the acquisition and processing of the preset number of frame images after the current frame.

[0150] The adjustment module 10 is further configured to adjust the state of the filter component if the adjusted statistical brightness information does not meet the preset conditions, until the adjusted statistical brightness information meets the preset conditions. The filter component is used to adjust the amount of light entering the image during image acquisition.

[0151] The processing module 20 is used to acquire optical signals based on the target control parameters and the state of the filter component, and generate a preset number of frame image signals after the current frame based on the optical signals.

[0152] The processing module 20 is further configured to perform digital gain processing and compression processing on the image signal of the preset number of frames after the current frame based on the target control parameters, to obtain the target image of the preset number of frames after the current frame. The pixel value range of the target image is the target pixel value range. The pixel value range of the image after digital gain processing is not less than the pixel value range of the image before digital gain processing. The target pixel value range is preset based on the application requirements of the current frame image.

[0153] This embodiment adjusts the control parameters used in the current frame image to target control parameters based on the statistical brightness information corresponding to the statistical image contained in the current frame image. When the adjusted statistical brightness information does not meet the preset conditions, the state of the filter component is adjusted. The light signal collected based on the target control parameters and the state of the filter component generates the preset number of frame image signals after the current frame. Based on the target control parameters, the preset number of frame image signals after the current frame are subjected to digital gain processing and compression processing to obtain the target image of the preset number of frames after the current frame. By using a filter device that can reduce the amount of light entering the image, the flicker phenomenon caused by the reduction of exposure time is avoided. By adjusting the control parameters of image acquisition and processing using the statistical brightness information of the image, a result image with good quality, appropriate brightness and appropriate dynamic range can be obtained in different scenarios.

[0154] In one embodiment, the processing module 20 is further configured to preprocess the image signal of the preset number of frames after the current frame to obtain a preprocessed image; perform a gain processing on the preprocessed image based on the target processing control parameters to obtain a reference image; and perform data compression on the reference image to obtain the target image of the preset number of frames after the current frame.

[0155] In one embodiment, the processing module 20 is further configured to perform gain processing on the image signal of the preset number of frames after the current frame based on the target processing control parameters to obtain a first reference image; perform data compression on the first reference image to obtain a second reference image; and perform post-processing on the second reference image to obtain a target image of the preset number of frames after the current frame.

[0156] In one embodiment, the processing module 20 is further configured to preprocess the image signal of a preset number of frames after the current frame to obtain a preprocessed image; perform a gain processing on the preprocessed image according to the target processing control parameters to obtain a first reference image; perform data compression on the first reference image to obtain a second reference image; and perform postprocessing on the second reference image to obtain a target image of a preset number of frames after the current frame.

[0157] In one embodiment, the processing module 20 is further configured to: determine a first gain processing control parameter and a second gain processing control parameter according to the target processing control parameter, wherein the product of the first gain processing control parameter and the second gain processing control parameter is equal to the target processing control parameter; perform a first gain processing on the image signal of a preset number of frames after the current frame according to the first gain processing control parameter to obtain a first reference image; perform data compression on the first reference image to obtain a second reference image; perform intermediate processing on the second reference image to obtain a third reference image; perform a second gain processing on the third reference image according to the second gain processing control parameter to obtain a fourth reference image; and perform data compression on the fourth reference image to obtain a target image of a preset number of frames after the current frame, wherein the compressed images have the same brightness value range.

[0158] In one embodiment, the gain processing does not truncate the bit width or truncates the bit width to a preset brightness value range, wherein the preset brightness value range is not less than the brightness value range of the image signal of the preset number of frames after the current frame, and the compression parameters of the data compression are related to the gain of the gain processing.

[0159] In one embodiment, the statistical brightness information includes a statistical brightness value and a dynamic range extension value. The adjustment module 10 is further configured to adjust the control parameters used in the current frame image according to the statistical brightness value and the dynamic range extension value to obtain target control parameters.

[0160] In one embodiment, the adjustment module 10 is further configured to adjust the control parameters used in the current frame image according to the statistical brightness value to obtain initial control parameters; and adjust the initial control parameters according to the dynamic range extension value to obtain target control parameters.

[0161] In one embodiment, the adjustment module 10 is further configured to compare the statistical brightness value with a preset brightness value; determine the parameter adjustment direction based on the comparison result; and adjust the control parameters used in the current frame image according to the parameter adjustment direction to obtain initial control parameters.

[0162] In one embodiment, the adjustment module 10 is further configured to: when the parameter adjustment direction is upward, if the adjusted control parameter is greater than the corresponding first parameter threshold, then use the first parameter threshold as the adjusted control parameter, and continue to adjust the next control parameter according to the parameter adjustment direction; return to execute the step of using the first parameter threshold as the adjusted control parameter and continue to adjust the next control parameter according to the parameter adjustment direction if the adjusted control parameter is greater than the corresponding first parameter threshold, to obtain an initial control parameter; when the parameter adjustment direction is downward, if the adjusted control parameter is less than the corresponding second parameter threshold, then use the second parameter threshold as the adjusted control parameter, and continue to adjust the next control parameter according to the parameter adjustment direction; return to execute the step of using the second parameter threshold as the adjusted control parameter and continue to adjust the next control parameter according to the parameter adjustment direction if the adjusted control parameter is less than the corresponding second parameter threshold, to obtain an initial control parameter, wherein the first parameter threshold is greater than the second parameter threshold.

[0163] In one embodiment, when the parameter adjustment direction is upward, the next control parameter is determined by the first parameter adjustment order; when the parameter adjustment direction is downward, the next control parameter is determined by the second parameter adjustment order, wherein the first parameter adjustment order is the opposite of the second parameter adjustment order.

[0164] In one embodiment, the adjustment module 10 is further configured to obtain an initial digital gain parameter based on the initial control parameters; when the initial digital gain parameter is less than the dynamic range extension value, adjust the initial digital gain parameter based on the dynamic range extension value to obtain a target digital gain parameter, and reduce other current initial control parameters besides the initial digital gain parameter; if the reduced other initial control parameters are less than the corresponding parameter threshold, then the parameter threshold is used as the adjusted control parameter, and the next other initial control parameter corresponding to the current other initial control parameter is further reduced; return to the step of reducing the next other initial control parameter corresponding to the current other initial control parameter if the reduced other initial control parameters are less than the corresponding parameter threshold, to obtain a target control parameter corresponding to the other initial control parameters, wherein the target control parameter includes the target digital gain parameter and the target control parameter corresponding to the other initial control parameters.

[0165] In one embodiment, the adjustment module 10 is further configured to adjust the filter component to a target state if the adjusted statistical brightness value is still greater than the preset brightness value, until the adjusted statistical brightness value reaches the preset brightness value. The target state is a first state, a combination of the first state and the second state, or a combination of the first state and the third state. The first state is a low light transmittance state, the second state is a state that allows visible light to pass through, and the third state is a state that allows both visible light and infrared light to pass through.

[0166] It should be understood that the above are merely illustrative examples and do not constitute any limitation on the technical solutions of the present invention. In specific applications, those skilled in the art can make settings as needed, and the present invention does not impose any restrictions on this.

[0167] It should be understood that although the steps in the flowcharts of this application's embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0168] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this invention. In practical applications, those skilled in the art can select some or all of the workflow to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0169] Furthermore, it should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0170] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0171] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0172] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. An image generation method, characterized in that, The image generation method includes: The control parameters used in the current frame image are adjusted according to the statistical brightness information corresponding to the statistical image contained in the current frame image to obtain the target control parameters, and the adjusted statistical brightness information is obtained according to the target control parameters. The control parameters are calculated from the preset number of frames before the current frame. The target control parameters are used to control the acquisition and processing of the preset number of frames after the current frame. If the adjusted statistical brightness information does not meet the preset conditions, the state of the filter component is adjusted until the adjusted statistical brightness information meets the preset conditions. The filter component is used to adjust the amount of light entering the image during image acquisition. Based on the target control parameters and the state of the filter component, light signals are acquired, and a preset number of frame image signals are generated after the current frame based on the light signals. Based on the target control parameters, digital gain processing and compression processing are performed on the image signal of the preset number of frames after the current frame to obtain the target image of the preset number of frames after the current frame. In the gain processing, bit width truncation is not performed or the gain processing is truncation to a preset brightness value range. The pixel value range of the target image is the target pixel value range. The pixel value range of the image after digital gain processing is not less than the pixel value range of the image before digital gain processing. The target pixel value range is preset based on the application requirements of the current frame image.

2. The image generation method as described in claim 1, characterized in that, The step of performing digital gain processing and compression processing on the image signal of the preset number of frames after the current frame based on the target control parameters to obtain the target image of the preset number of frames after the current frame includes: The image signal of the preset number of frames following the current frame is preprocessed to obtain a preprocessed image; Based on the target control parameters, the preprocessed image is subjected to a gain processing step to obtain a reference image; The reference image is compressed to obtain the target image of the preset number of frames after the current frame.

3. The image generation method as described in claim 1, characterized in that, The step of performing digital gain processing and compression processing on the image signal of the preset number of frames after the current frame based on the target control parameters to obtain the target image of the preset number of frames after the current frame includes: Based on the target control parameters, the image signal of the preset number of frames after the current frame is subjected to gain processing once to obtain the first reference image; The first reference image is compressed to obtain the second reference image; The second reference image is post-processed to obtain the target image of the preset number of frames after the current frame.

4. The image generation method as described in claim 1, characterized in that, The step of performing digital gain processing and compression processing on the image signal of the preset number of frames after the current frame based on the target control parameters to obtain the target image of the preset number of frames after the current frame includes: The image signal of the preset number of frames following the current frame is preprocessed to obtain a preprocessed image; The preprocessed image is subjected to a gain processing step based on the target control parameters to obtain a first reference image; The first reference image is compressed to obtain the second reference image; The second reference image is post-processed to obtain the target image of the preset number of frames after the current frame.

5. The image generation method as described in claim 1, characterized in that, The step of performing digital gain processing and compression processing on the image signal of the preset number of frames after the current frame based on the target control parameters to obtain the target image of the preset number of frames after the current frame includes: A first gain processing control parameter and a second gain processing control parameter are determined based on the target control parameter, and the product of the first gain processing control parameter and the second gain processing control parameter is equal to the target control parameter. Based on the first gain processing control parameter, the image signal of the preset number of frames after the current frame is subjected to the first gain processing to obtain the first reference image; The first reference image is compressed to obtain the second reference image; The second reference image is processed to obtain the third reference image; The third reference image is subjected to a second gain processing based on the second gain processing control parameters to obtain a fourth reference image; The fourth reference image is compressed to obtain the target image of the preset number of frames after the current frame, wherein the compressed images have the same range of brightness values.

6. The image generation method according to any one of claims 1 to 5, characterized in that, The preset brightness value range is not less than the brightness value range of the image signal of the preset number of frames after the current frame, and the compression parameters of the data compression are related to the gain of the gain processing.

7. The image generation method as described in claim 1, characterized in that, The statistical brightness information includes statistical brightness values ​​and dynamic range extension values. The step of adjusting the control parameters used for the current frame image based on the statistical brightness information corresponding to the statistical image contained in the current frame image to obtain the target control parameters includes: The target control parameters are obtained by adjusting the control parameters used in the current frame image based on the statistical brightness value and the dynamic range extension value.

8. The image generation method as described in claim 7, characterized in that, The step of adjusting the control parameters of the current frame image based on the statistical brightness value and the dynamic range extension value to obtain the target control parameters includes: The control parameters used in the current frame image are adjusted based on the statistical brightness value to obtain the initial control parameters; The initial control parameters are adjusted based on the dynamic range extension value to obtain the target control parameters.

9. The image generation method as described in claim 8, characterized in that, The step of adjusting the control parameters of the current frame image based on the statistical brightness value to obtain the initial control parameters includes: Compare the statistical brightness value with the preset brightness value; Determine the direction of parameter adjustment based on the comparison results; The control parameters used to adjust the direction of the current frame image according to the parameters are used to obtain the initial control parameters.

10. The image generation method as described in claim 9, characterized in that, The step of adjusting the control parameters used to adjust the direction of the current frame image according to the parameters to obtain the initial control parameters includes: When the parameter adjustment direction is upward, if the adjusted control parameter is greater than the corresponding first parameter threshold, then the first parameter threshold is used as the adjusted control parameter, and the next control parameter is adjusted according to the parameter adjustment direction. Return to the step of if the adjusted control parameter is greater than the corresponding first parameter threshold, then take the first parameter threshold as the adjusted control parameter, and continue to adjust the next control parameter according to the parameter adjustment direction to obtain the initial control parameter; When the parameter adjustment direction is downward, if the adjusted control parameter is less than the corresponding second parameter threshold, then the second parameter threshold is used as the adjusted control parameter, and the next control parameter is adjusted according to the parameter adjustment direction. If the adjusted control parameter is less than the corresponding second parameter threshold, then the second parameter threshold is used as the adjusted control parameter, and the next control parameter is adjusted according to the parameter adjustment direction to obtain the initial control parameter, wherein the first parameter threshold is greater than the second parameter threshold.

11. The image generation method as described in claim 10, characterized in that, When the parameter adjustment direction is upward, the next control parameter is determined by the first parameter adjustment order; when the parameter adjustment direction is downward, the next control parameter is determined by the second parameter adjustment order, and the first parameter adjustment order is the opposite of the second parameter adjustment order.

12. The image generation method as described in claim 8, characterized in that, The step of adjusting the initial control parameters according to the dynamic range extension value to obtain the target control parameters includes: The initial digital gain parameters are obtained based on the initial control parameters. When the initial digital gain parameter is less than the dynamic range extension value, the initial digital gain parameter is adjusted based on the dynamic range extension value to obtain the target digital gain parameter, and other current initial control parameters other than the initial digital gain parameter are reduced. If the reduced other initial control parameters are less than the corresponding parameter threshold, then the parameter threshold is used as the adjusted control parameter, and the next other initial control parameter corresponding to the current other initial control parameter is further reduced; Return to the step of reducing the next initial control parameter corresponding to the current initial control parameter if the reduced initial control parameter is less than the corresponding parameter threshold, so as to obtain the target control parameter corresponding to the other initial control parameter. The target control parameter includes the target digital gain parameter and the target control parameter corresponding to the other initial control parameter.

13. The image generation method as described in claim 12, characterized in that, If the adjusted statistical brightness information does not meet the preset conditions, the state of the filter component is adjusted until the adjusted statistical brightness information meets the preset conditions, including: If the adjusted statistical brightness value is still greater than the preset brightness value, the filter component is adjusted to the target state until the adjusted statistical brightness value reaches the preset brightness value. The target state is a first state, a combination of the first and second states, or a combination of the first and third states. The first state is a low light transmittance state, the second state is a state that allows visible light to pass through, and the third state is a state that allows both visible and infrared light to pass through.

14. An image generation apparatus, characterized in that, The image generation device includes: The adjustment module is used to adjust the control parameters used in the current frame image according to the statistical brightness information corresponding to the statistical image contained in the current frame image, to obtain the target control parameters, and to obtain the adjusted statistical brightness information according to the target control parameters. The control parameters are calculated from the preset number of frames before the current frame. The target control parameters are used to control the acquisition and processing of the preset number of frames after the current frame. The adjustment module is also used to adjust the state of the filter component if the adjusted statistical brightness information does not meet the preset conditions, until the adjusted statistical brightness information meets the preset conditions. The filter component is used to adjust the amount of light entering the image during image acquisition. The processing module is used to acquire optical signals based on the target control parameters and the state of the filter component, and generate a preset number of frame image signals after the current frame based on the optical signals; The processing module is further configured to perform digital gain processing and compression processing on the image signal of the preset number of frames after the current frame based on the target control parameters to obtain the target image of the preset number of frames after the current frame. In the gain processing, bit width truncation is not performed or the gain processing is truncation to a preset brightness value range. The pixel value range of the target image is the target pixel value range. The pixel value range of the image after digital gain processing is not less than the pixel value range of the image before digital gain processing. The target pixel value range is preset based on the application requirements of the current frame image.

15. An image generation device, characterized in that, The image generation device includes: a memory, a processor, and an image generation program stored in the memory and running on the processor, the image generation program being configured to implement the image generation method as described in any one of claims 1 to 13.

16. A storage medium, characterized in that, The storage medium stores an image generation program, which, when executed by a processor, implements the image generation method as described in any one of claims 1 to 13.

Images