Image generation methods and electronic devices
By acquiring and fusing images, using the time domain integral operation of the light source brightness function to generate striped images, solving the problem of difficulty in obtaining paired training data, realizing the generation of simulated strobe band images in dynamic scenes, and supporting the research of image processing algorithms .
Patent Information
- Application Number
- CN202110859072.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-07-28
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2041-07-28
AI Technical Summary
In the prior art, paired training data is difficult to obtain in large quantities, especially in dynamic scenarios, which leads to difficulties in researching image processing algorithms.
By acquiring the first image and determining the light source brightness function, performing M time domain integration operations to obtain the brightness value, generating a second image with stripe characteristics, and fusing it with the first image to generate a strobe band image, and simulate it into pairs Training data.
It realizes that a large number of non-repetitive strobe band images are generated without the need for complex scene reshoots, which meets the needs of algorithm research and solves the problem of difficulty in obtaining paired training data.
Smart Images

Figure CN113592754B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of image processing technology, and specifically relates to an image generation method and electronic device. Background Art
[0002] If you use a high-speed shutter to take a photo under AC-powered lighting, the captured image will have alternating light and dark stripes. In order to remove the alternating light and dark stripes on the image, people need to collect paired training data for related algorithm research. At present, in order to obtain the paired training data, a fixed camera device is often used to take a set of images with different shutter speeds in the same scene: images with low shutter speeds (no alternating light and dark stripes) and images with high shutter speeds (there are alternating light and dark stripes).
[0003] However, this method is cumbersome to operate and requires the camera device and the photographed area to be highly stable. Therefore, it is difficult to obtain a large amount of paired training data and it is difficult to implement in dynamic scenes. It can be seen that in the related art, there is a problem that paired training data is difficult to obtain in large quantities and difficult to obtain in dynamic scenes. Summary of the invention
[0004] The purpose of the embodiments of the present application is to provide an image generation method and electronic device that can solve the problem that paired training data is difficult to obtain in large quantities and difficult to obtain in dynamic scenes.
[0005] In a first aspect, an embodiment of the present application provides a method for image generation, comprising: acquiring a first image, the first image comprising M×N pixels, wherein M and N are positive integers; determining a light source brightness function; based on preset parameters, performing M time-domain integration operations on the light source brightness function to obtain M brightness values; based on the M brightness values, generating a second image, the second image being a stripe image having M×N pixels, wherein the brightness values of M rows of pixels in the second image respectively correspond to the M brightness values; based on the first image and the second image, fusing and generating a stroboscopic stripe image.
[0006] In a second aspect, an embodiment of the present application provides an image generation device, comprising: an acquisition module, used to acquire a first image, the first image comprising M×N pixels, wherein M and N are positive integers; a determination module, used to determine a light source brightness function; an operation module, used to perform M time-domain integration operations on the light source brightness function based on preset parameters to obtain M brightness values; an image generation module, used to generate a second image based on the M brightness values, the second image being a stripe image having M×N pixels, wherein the brightness values of M rows of pixels in the second image respectively correspond to the M brightness values; an image fusion module, used to fuse and generate a stroboscopic stripe image based on the first image and the second image.
[0007] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.
[0008] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.
[0009] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.
[0010] In an embodiment of the present application, a first image is acquired, wherein the first image includes M×N pixels, wherein M and N are positive integers; a light source brightness function is determined; based on preset parameters, the light source brightness function is subjected to M time-domain integration operations to obtain M brightness values; based on the M brightness values, a second image is generated, wherein the second image is a stripe image having M×N pixels, wherein the brightness values of M rows of pixels in the second image correspond to the M brightness values, respectively; based on the first image and the second image, a stroboscopic stripe image is generated by fusion, which can solve the problem that paired training data is difficult to acquire in large quantities and difficult to acquire in dynamic scenes. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Figure 1 is a schematic flow chart of a method for generating an image according to an embodiment of the present application;
[0012] Figure 2 is a schematic flow chart of an image generating method according to another embodiment of the present application;
[0013] Figure 3 is a schematic flow chart of an image generating method according to another embodiment of the present application;
[0014] Figure 4 is a schematic flow chart of an image generating method according to another embodiment of the present application;
[0015] Figure 5 is a schematic flow chart of an image generating method according to another embodiment of the present application;
[0016] Figure 6 is a schematic flow chart of an image generating method according to another embodiment of the present application;
[0017] Figure 7 is a schematic diagram of a second image according to an embodiment of the present application;
[0018] Figure 8a-8b is a schematic diagram of a synthetic stroboscopic stripe image according to an embodiment of the present application;
[0019] Fig. 9 is a structural schematic diagram of an image generating device according to an embodiment of the present application;
[0020] Fig.10 is a schematic structural diagram of an electronic device according to another embodiment of the present application;
[0021] Fig.11 It is a schematic diagram of the hardware structure of an electronic device according to another embodiment of the present application. DETAILED DESCRIPTION
[0022] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.
[0023] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0024] The image generation method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.
[0025] like Figure 1 As shown, an embodiment of the present application provides a method 100 for generating an image. The method can be executed by a terminal device. In other words, the method can be executed by software or hardware installed in the terminal device. The method includes the following steps:
[0026] S110: Acquire a first image, where the first image includes M×N pixels, where M and N are positive integers.
[0027] The first image is an image without alternating light and dark stripes. The size of the first image is determined. For example, it is determined that the first image includes 40×30 pixels, that is, the first image includes 40 rows of pixels, and each row of pixels includes 30 pixels.
[0028] S120: Determine the light source brightness function.
[0029] The light source brightness function is used to represent the functional relationship between the light source brightness and time. For example, when the power supply changes periodically over time, the light source brightness will also change periodically over time, that is, a periodic function can be used to represent the functional relationship between the light source brightness and time.
[0030] The light source brightness function may be at least one of pre-set light source brightness functions, or may be at least one of light source brightness functions established according to relevant parameters after acquiring the first image.
[0031] S130: Based on preset parameters, perform M time-domain integration operations on the light source brightness function to obtain M brightness values.
[0032] Considering that in reality, the light collection of the photosensitive element in the camera device is essentially the number of photons collected by the photosensitive element during the exposure time, the light collection of the photosensitive element can be simulated by performing a time domain integration operation on the light source brightness function.
[0033] The preset parameters are used to determine the time domain interval based on which the time domain integration operation of the light source brightness function is performed. For example, the light source brightness function can be represented by L=f(t), where L represents the light source brightness and t represents time. The integral of the light source brightness function in the time domain interval T0 to T1 can be represented by Indicates that the preset parameters are used to determine T0 and T1.
[0034] M groups of time domain intervals are determined based on the preset parameters, and based on the M groups of time domain intervals, M time domain integration operations are performed on the light source brightness function to obtain M brightness values.
[0035] For example, if the first image includes 40×30 pixels, 40 groups of time domain intervals are determined based on the preset parameters, and based on the 40 groups of time domain intervals, 40 time domain integration operations are performed on the light source brightness function to obtain 40 brightness values.
[0036] The preset parameters can be set by the user according to the needs of algorithm research. By setting different preset parameters, it is possible to simulate and generate light and dark alternating stripes that would be produced by shooting with a high-speed shutter under different shooting conditions.
[0037] S140: Generate a second image based on the M brightness values, where the second image is a stripe image having M×N pixels, wherein the brightness values of M rows of pixels in the second image correspond to the M brightness values respectively.
[0038] It can be understood that the M brightness values are used to determine the brightness values of M rows of pixels in the second image.
[0039] In one implementation, the brightness value of the pixel point in the i-th row in the second image is determined according to the i-th brightness value among M brightness values, where i=1,…M.
[0040] For example, after performing 40 time-domain integration operations on the light source brightness function, 40 brightness values are obtained, and it is determined that the brightness values of the first row of pixels in the second image are all the first brightness value among the 40 brightness values, and it is determined that the brightness values of the second row of pixels in the second image are all the second brightness values among the 40 brightness values, and it is determined that the brightness values of the third row of pixels in the second image are all the third brightness values among the 40 brightness values…Thus, based on the M brightness values, a second image is generated, and the second image is a stripe image with M×N pixels, and the brightness values of the M rows of pixels in the second image correspond to the M brightness values respectively.
[0041] S150: Generate a stroboscopic stripe image by fusing the first image and the second image.
[0042] The sizes of the first image and the second image are exactly the same, satisfying the fusion condition, and no size adjustment is required. The M×N pixels included in the first image correspond one-to-one to the M×N pixels included in the second image, and the first image and the second image can be fused by multiplication. Since the first image is a normal scene image without alternating light and dark strips, and the second image is a stripe image, the first image and the second image are fused, that is, the stripe image is superimposed on the scene image to obtain a stroboscopic stripe image, and the stroboscopic stripe image and the first image constitute paired training data.
[0043] Therefore, the image generation method provided by the embodiment of the present application obtains a first image, wherein the first image includes M×N pixels, wherein M and N are positive integers; determines a light source brightness function; based on preset parameters, performs M time-domain integration operations on the light source brightness function to obtain M brightness values; based on the M brightness values, generates a second image, wherein the second image is a stripe image having M×N pixels, wherein the brightness values of M rows of pixels in the second image correspond to the M brightness values respectively; based on the first image and the second image, a stroboscopic stripe image is generated by fusion, and a stroboscopic stripe image paired with the first image can be simulated and generated according to the generation principle of the stroboscopic stripe, and a large number of non-repetitive stroboscopic stripe images paired with the first image can be simulated and generated based on different light source brightness functions and multiple sets of preset parameters, thereby meeting the needs of algorithm research, without the need to use a camera device to capture an image with a high shutter speed in the same scene as the first image to generate a stroboscopic stripe image, thereby solving the problem that paired training data is difficult to obtain in large quantities and difficult to obtain in dynamic scenes.
[0044] like Figure 2 As shown, an embodiment of the present application provides a method 200 for generating an image. Figure 1 The step S120 of the embodiment is further defined and includes the following steps:
[0045] S220: Establishing the light source brightness function according to a preset frequency, wherein the light source brightness function is a sine function.
[0046] Considering that in reality, the current of AC power supply usually presents a sinusoidal fluctuation of a certain frequency, for example, in different countries or regions, the current of AC power supply presents a sinusoidal fluctuation of 50Hz or 60Hz. The brightness of the light source is positively correlated with the current strength of the AC power supply. Therefore, the functional relationship between the brightness of the light source and time can be expressed as:
[0047] L=|sin(at)|
[0048] By changing the coefficient a, the frequency of the light source brightness change can be set.
[0049] The preset frequency can be set by the user according to the needs of algorithm research. By setting different preset frequencies, the light source brightness function under different frequency AC power supply is established, and then the alternating light and dark stripes produced by shooting with a high-speed shutter under different light conditions are simulated.
[0050] In the embodiment of the present application, steps S210, S230-S250 can be implemented by Figure 1 The description of steps S110, S130-S150 of the embodiment will not be repeated here.
[0051] Therefore, by establishing the light source brightness function according to the preset frequency, the light source brightness function is a sine function, which can simulate the AC-powered light source through the sine function, and by setting different preset frequencies, the light source brightness function under AC power supply of different frequencies is established, thereby simulating the generation of stroboscopic stripe images that will be produced when shooting with a high-speed shutter under different light source conditions.
[0052] like Figure 3 As shown, an embodiment of the present application provides a method 300 for generating an image. Figure 1 The step S130 of the embodiment is further limited. In the embodiment of the present application, the preset parameters include shooting frequency and exposure time. The step S130 includes the following steps:
[0053] S331: Determine the exposure start time of the i-th row of pixels according to the shooting frequency, i=1, ...M.
[0054] Considering that the exposure of the camera device in reality is usually "rolling shutter" exposure, the amount of light entering each row of pixels is related to two factors: the exposure start time and the exposure duration. Among them, the exposure start time of each row is determined by the shooting frequency. Therefore, the shooting frequency can be preset to determine the exposure start time of each row of pixels.
[0055] S332: Determine the exposure end time of the pixel points in the i-th row according to the exposure start time and the exposure duration of the pixel points in the i-th row.
[0056] The exposure end time of each row of pixels is determined according to the exposure start time of each row of pixels and the preset exposure duration. Thus, the exposure start time and exposure end time of M rows of pixels constitute M time domain intervals.
[0057] S333: According to the exposure start time of the pixel points in the i-th row and the exposure end time of the pixel points in the i-th row, perform M time-domain integration operations on the light source brightness function to obtain the M brightness values.
[0058] According to the M time domain intervals, M time domain integration operations are performed on the light source brightness function, so that M brightness values corresponding to the M rows of pixels can be obtained.
[0059] For example, when the light source brightness function is determined to be L=|sin(at)|, the light source brightness function is subjected to M time domain integration operations to determine the brightness values corresponding to M rows of pixels in the second image. The brightness value corresponding to each row of pixels can be expressed as:
[0060]
[0061] Wherein, Pi represents the brightness value corresponding to the pixel point in the i-th row, i=1,…M, Ti represents the exposure start time of the i-th row determined by the shooting frequency, and tp represents the exposure time of each row.
[0062] In the embodiment of the present application, steps S310, S320, S340, and S350 may be implemented as follows: Figure 1 The description of the steps S110, S120, S140, and S150 of the embodiment, step S320 also adopts Figure 2 The description of step S220 of the embodiment will not be repeated here.
[0063] Therefore, by determining the exposure start time of the i-th row of pixels according to the shooting frequency, i=1,...M; determining the exposure end time of the i-th row of pixels according to the exposure start time and the exposure duration; performing M time-domain integration operations on the light source brightness function according to the exposure start time and the exposure end time of the i-th row of pixels to obtain the M brightness values, different shooting conditions can be simulated by setting different shooting frequencies and exposure durations, and then simulating the generation of stroboscopic stripe images that would be produced by shooting with a high-speed shutter under different shooting conditions.
[0064] like Figure 4 As shown, an embodiment of the present application provides a method 400 for generating an image. Figure 1 The step S140 of the embodiment is further defined and includes the following steps:
[0065] S441: Based on the M brightness values, determine the brightness value of the j-th pixel in the i-th row according to Poisson distribution, where i=1, ...M, j=1, ...N.
[0066] Taking into account that in reality, the amount of light entering each pixel will be affected by various factors, the brightness values of each row of pixels are not exactly the same, and will fluctuate randomly within a certain range and obey the law of Poisson distribution. Therefore, in an embodiment of the present application, in the process of generating the second image based on the M brightness values, the brightness value of the jth pixel in the i-th row is also determined according to the Poisson distribution.
[0067] For example, the brightness value corresponding to each row of pixels is determined as:
[0068]
[0069] On this basis, according to the Poisson distribution, the brightness value of the jth pixel in the i-th row is determined:
[0070]
[0071] in, represents the brightness value of the j-th pixel in the i-th row, and ρ(i,j) represents the influence coefficient of the j-th pixel in the i-th row under the Poisson distribution.
[0072] Thus, through step S441, the randomness of each row of pixels in the second image is increased, so that the brightness value of each row of pixels in the second image is closer to the actual situation, such as Figure 7 shown.
[0073] S442: Generate the second image according to the brightness value of the j-th pixel in the i-th row.
[0074] Since the brightness value of the jth pixel point in the ith row is closer to the actual situation, the second image generated based on the brightness value of the jth pixel point in the ith row is closer to the actual situation, thereby making the stroboscopic strip image generated by fusion based on the first image and the second image closer to the actual situation.
[0075] In the embodiment of the present application, steps S410-S430 and S450 may be performed by Figure 1 The description of the steps S110-S130 and S150 of the embodiment, step S420 can also be adopted Figure 2 In the description of step S220 of the embodiment, step S430 can also be adopted Figure 3 The description of steps S331-S333 of the embodiment will not be repeated here.
[0076] Therefore, by determining the brightness value of the jth pixel in the ith row according to the Poisson distribution based on the M brightness values, i=1,…M, j=1,…N; generating the second image according to the brightness value of the jth pixel in the ith row, the brightness value of each row of pixels in the second image can be randomly fluctuated within a certain range according to the actual light input characteristics of each pixel, which is closer to the actual situation, thereby making the stroboscopic stripe image obtained based on the fusion of the second image more in line with the actual situation.
[0077] like Figure 5 As shown, an embodiment of the present application provides a method 500 for generating an image. Figure 1 The step S150 of the embodiment is further defined and includes the following steps:
[0078] S551: Determine whether there is a point light source in the first image.
[0079] Whether there is a point light source in the first image, the embodiment of the present application does not limit the detection method. Considering that in reality, if there is a point light source in the shooting scene, the point light source will directly affect the brightness near the location of the point light source, that is, the existence of the point light source will destroy the distribution of light and dark strips, and the second image generated by step S540 is a stripe image, which does not take into account the influence of the point light source. Therefore, it is necessary to determine whether there is a point light source in the first image. If there is a point light source, the influence of the point light source on the light and dark strips can be superimposed in the second image based on the point light source in the first image.
[0080] S552: If there is a point light source in the first image, based on the position and intensity of the point light source in the first image and according to a two-dimensional Gaussian distribution, adjust the brightness value at the corresponding position in the second image to obtain a third image.
[0081] If there is a point light source in the first image, determine the position P of the point light source in the first image. loc (x,y) and intensity P s (x, y). Since the brightness of a point light source follows a two-dimensional Gaussian distribution, the brightness value at the corresponding position in the second image can be adjusted based on the position and intensity of the point light source in the first image according to the two-dimensional Gaussian distribution. Specifically, the brightness value of the point light source can be simulated first:
[0082]
[0083] Where P(i,j) represents the brightness value generated by the point light source at the jth pixel in the i-th row, A represents the empirical intensity coefficient, and loc x ,loc y represents the position of the point light source, and σ represents the empirical attenuation coefficient.
[0084] Then, the brightness value of the point light source is superimposed on the second image to generate a third image, such as Figure 8a shown.
[0085] S553: Generate the stroboscopic strip image by fusing the first image and the third image.
[0086] Since the third image is superimposed with the influence of the point light source on the basis of the stripe image, the distribution of light and dark stripes in the third image is closer to the actual situation. Therefore, the stroboscopic stripe image generated by fusion based on the first image and the third image is closer to the actual situation. Figure 8b shown.
[0087] In the embodiment of the present application, steps S510-S540 may be performed by Figure 1 In the description of the steps S110-S140 of the embodiment, step S520 can also be adopted Figure 2 In the description of step S220 of the embodiment, step S530 can also be adopted Figure 3 In the description of the steps S331-S333 of the embodiment, step S540 can also be adopted Figure 4 The description of steps S441-S442 of the embodiment will not be repeated here.
[0088] Therefore, by determining whether there is a point light source in the first image; if there is a point light source in the first image, based on the position and intensity of the point light source in the first image, according to the two-dimensional Gaussian distribution, the brightness value at the corresponding position in the second image is adjusted to obtain the third image; based on the first image and the third image, the stroboscopic strip image is fused to simulate the actual influence of the point light source on the distribution of light and dark strips, so that the fused stroboscopic strip image is more in line with the actual situation.
[0089] like Figure 6 As shown, an embodiment of the present application provides a method 600 for generating an image. Figure 1 The step S150 of the embodiment is further defined and includes the following steps:
[0090] S651: Determine a brightness coefficient based on the brightness value of the first image.
[0091] In the embodiment of the present application, considering that the brightness value of the second image should match the brightness value of the first image, for example, if the first image is shot in a darker scene, then the brightness of the generated second image should also be darker, so that the fused stroboscopic stripe image will be more in line with the actual situation. Therefore, before the first image is fused with the second image, a brightness coefficient can be determined based on the brightness value of the first image, and the brightness coefficient is used to adjust the brightness value of the second image so that the overall brightness of the generated stripe image is more in line with the brightness in the shooting scene of the first image.
[0092] In one implementation, the first image is converted into a grayscale image, the average grayscale value of the grayscale image is calculated, and the average grayscale value is normalized to a range of 0 to 1 to obtain a brightness coefficient.
[0093] S652: Adjust the brightness value of the second image according to the brightness coefficient to obtain a fourth image.
[0094] The brightness value of each pixel in the second image is multiplied by the brightness coefficient to obtain a fourth image.
[0095] S653: Generate the stroboscopic strip image by fusing the first image and the fourth image.
[0096] Since the overall brightness of the fourth image is adjusted based on the brightness coefficient, the overall brightness of the fourth image is more consistent with the brightness in the shooting scene of the first image. The stroboscopic strip image generated by fusion based on the first image and the fourth image is more in line with the actual situation.
[0097] In one implementation, step S653 may further include steps S551-S552, and steps S551-S552 may be before step S651 or after step S652.
[0098] In the embodiment of the present application, steps S610-S640 may be performed by Figure 1 In the description of the steps S110-S140 of the embodiment, step S620 can also be adopted Figure 2 In the description of step S220 of the embodiment, step S630 can also be adopted Figure 3 In the description of steps S331-S333 of the embodiment, step S640 can also be adopted Figure 4 The description of steps S441-S442 of the embodiment will not be repeated here.
[0099] Therefore, by determining the brightness coefficient based on the brightness value of the first image; adjusting the brightness value of the second image according to the brightness coefficient to obtain the fourth image; and generating the stroboscopic strip image by fusing the first image and the fourth image, the brightness of the light and dark strips can be more closely matched to the actual brightness of the shooting scene based on the overall brightness of the first image, so that the fused stroboscopic strip image is more closely matched to the actual situation.
[0100] It should be noted that the image generation method provided in the embodiment of the present application can be executed by an image generation device or a control module in the image generation device for executing the image generation method. In the embodiment of the present application, the image generation method executed by the image generation device is taken as an example to illustrate the image generation device provided in the embodiment of the present application.
[0101] Fig. 9 Schematic diagram of the structure of the image generation device according to the embodiment of the present application. Fig. 9 As shown, the image generation device 900 includes: an acquisition module 910 , a determination module 920 , a calculation module 930 , an image generation module 940 and an image fusion module 950 .
[0102] An acquisition module 910 is used to acquire a first image, wherein the first image includes M×N pixels, wherein M and N are positive integers; a determination module 920 is used to determine a light source brightness function; an operation module 930 is used to perform M time-domain integration operations on the light source brightness function based on preset parameters to obtain M brightness values; an image generation module 940 is used to generate a second image based on the M brightness values, wherein the second image is a stripe image having M×N pixels; an image fusion module 950 is used to fuse the first image and the second image to generate a stroboscopic stripe image.
[0103] In one implementation, the determination module is used to establish the light source brightness function according to a preset frequency, and the light source brightness function is a sine function.
[0104] In one implementation, the preset parameters include a shooting frequency and an exposure duration, and the calculation module is used to determine the exposure start time of the i-th row of pixels according to the shooting frequency, i=1,…M; determine the exposure end time of the i-th row of pixels according to the exposure start time and the exposure duration of the i-th row of pixels; and perform M time-domain integration operations on the light source brightness function according to the exposure start time and the exposure end time of the i-th row of pixels to obtain the M brightness values.
[0105] In one implementation, the image generation module is used to determine the brightness value of the jth pixel in the i-th row based on the M brightness values according to Poisson distribution, i=1,...M, j=1,...N; and generate the second image according to the brightness value of the jth pixel in the i-th row.
[0106] In one implementation, the image fusion module is used to determine whether there is a point light source in the first image; if there is a point light source in the first image, based on the position and intensity of the point light source in the first image, according to the two-dimensional Gaussian distribution, adjust the brightness value at the corresponding position in the second image to obtain a third image; based on the first image and the third image, the stroboscopic strip image is generated by fusion.
[0107] In one implementation, the image fusion module is used to determine a brightness coefficient based on the brightness value of the first image; adjust the brightness value of the second image according to the brightness coefficient to obtain a fourth image; and generate the stroboscopic strip image based on the first image and the fourth image by fusing them.
[0108] Therefore, an image generation device provided by an embodiment of the present application obtains a first image, wherein the first image includes M×N pixels, wherein M and N are positive integers; determines a light source brightness function; based on preset parameters, performs M time-domain integration operations on the light source brightness function to obtain M brightness values; based on the M brightness values, generates a second image, wherein the second image is a stripe image having M×N pixels, wherein the brightness values of M rows of pixels in the second image correspond to the M brightness values respectively; based on the first image and the second image, generates a stroboscopic stripe image by fusing, and can simulate and generate a stroboscopic stripe image paired with the first image according to the generation principle of the stroboscopic stripe, and based on different light source brightness functions and multiple sets of preset parameters, can simulate and generate a large number of non-repetitive stroboscopic stripe images paired with the first image, thereby meeting the needs of algorithm research, without the need to use a camera device to shoot an image with a high shutter speed in the same scene of the first image to generate a stroboscopic stripe image, thereby solving the problem that paired training data is difficult to obtain in large quantities and difficult to obtain in dynamic scenes.
[0109] The image generation device in the embodiments of the present application may be a device, or a component, integrated circuit, or chip in a terminal. The device may be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device may be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. The non-mobile electronic device may be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc., which is not specifically limited in the embodiments of the present application.
[0110] The image generating device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.
[0111] The image generation device provided in the embodiment of the present application can achieve Figures 1 to 6 To avoid repetition, the various processes implemented by the method embodiment are not described here.
[0112] Alternatively, if Fig.10As shown, an embodiment of the present application also provides an electronic device 1000, including a processor 1001, a memory 1002, and a program or instruction stored in the memory 1002 and executable on the processor 1001. When the program or instruction is executed by the processor 1001, each process of the above-mentioned image generation method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0113] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.
[0114] Fig.11 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.
[0115] The electronic device 1100 includes but is not limited to: a radio frequency unit 1101, a network module 1102, an audio output unit 1103, an input unit 1104, a sensor 1105, a display unit 1106, a user input unit 1107, an interface unit 1108, a memory 1109, and a processor 1110 and other components.
[0116] Those skilled in the art will appreciate that the electronic device 1100 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 1110 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.11 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.
[0117] Among them, the processor 1110 is used to obtain a first image, which includes M×N pixels, where M and N are positive integers; determine a light source brightness function; based on preset parameters, perform M time-domain integration operations on the light source brightness function to obtain M brightness values; based on the M brightness values, generate a second image, which is a stripe image with M×N pixels, where the brightness values of M rows of pixels in the second image correspond to the M brightness values respectively; based on the first image and the second image, fuse and generate a stroboscopic stripe image.
[0118] In one implementation, the processor 1110 is configured to establish the light source brightness function according to a preset frequency, and the light source brightness function is a sine function.
[0119] In one implementation, the preset parameters include a shooting frequency and an exposure duration, and the processor 1110 is used to determine the exposure start time of the i-th row of pixels according to the shooting frequency, i=1,…M; determine the exposure end time of the i-th row of pixels according to the exposure start time and the exposure duration of the i-th row of pixels; and perform M time-domain integration operations on the light source brightness function according to the exposure start time and the exposure end time of the i-th row of pixels to obtain the M brightness values.
[0120] In one implementation, the processor 1110 is used to determine the brightness value of the jth pixel in the i-th row based on the M brightness values according to Poisson distribution, where i=1,…M, j=1,…N; and generate the second image according to the brightness value of the jth pixel in the i-th row.
[0121] In one implementation, the processor 1110 is used to determine whether there is a point light source in the first image; if there is a point light source in the first image, based on the position and intensity of the point light source in the first image, adjust the brightness value at the corresponding position in the second image according to a two-dimensional Gaussian distribution to obtain a third image; based on the first image and the third image, the stroboscopic strip image is generated by fusion.
[0122] In one implementation, the processor 1110 is used to determine a brightness coefficient based on the brightness value of the first image; adjust the brightness value of the second image according to the brightness coefficient to obtain a fourth image; and generate the stroboscopic strip image by fusing the first image and the fourth image.
[0123] It should be understood that in the embodiment of the present application, the input unit 1104 may include a graphics processor (Graphics Processing Unit, GPU) 11041 and a microphone 11042, and the graphics processor 11041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 1106 may include a display panel 11061, and the display panel 11061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1107 includes a touch panel 11071 and other input devices 11072. The touch panel 11071 is also called a touch screen. The touch panel 11071 may include two parts: a touch detection device and a touch controller. Other input devices 11072 may include, but are not limited to, a physical keyboard, a function key (such as a volume control button, a switch button, etc.), a trackball, a mouse, and a joystick, which will not be repeated here. The memory 1109 can be used to store software programs and various data, including but not limited to applications and operating systems. The processor 1110 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 1110.
[0124] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned image generation method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0125] The processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0126] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image generation method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0127] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0128] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0129] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0130] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. A method for generating an image, characterized in that: The method comprises: Acquire a first image, the first image comprising M×N pixels, where M and N are positive integers; Determine a light source brightness function, where the light source brightness function is used to represent a functional relationship between light source brightness and time; Based on preset parameters, performing M time-domain integration operations on the light source brightness function to obtain M brightness values, wherein the preset parameters include shooting frequency and exposure duration; Based on the M brightness values, a second image is generated, where the second image is a stripe image having M×N pixels, wherein the brightness values of M rows of pixels in the second image respectively correspond to the M brightness values; A stroboscopic stripe image is generated by fusing the first image and the second image.
2. The method according to claim 1, characterized in that Based on the preset parameters, the light source brightness function is subjected to M time domain integration operations to obtain M brightness values, including: According to the shooting frequency, determining the exposure start time of the i-th row of pixels, i=1, ...M; Determining an exposure end time of the i-th row of pixels according to an exposure start time of the i-th row of pixels and the exposure duration; According to the exposure start time of the pixel points in the i-th row and the exposure end time of the pixel points in the i-th row, the light source brightness function is subjected to M time-domain integration operations to obtain the M brightness values.
3. The method according to claim 1, characterized in that The step of determining the light source brightness function comprises: According to the preset frequency, the light source brightness function is established, and the light source brightness function is a sine function.
4. The method according to claim 1, characterized in that: The generating a second image based on the M brightness values comprises: Based on the M brightness values, determine the brightness value of the j-th pixel in the i-th row according to Poisson distribution, where i=1, ...M, j=1, ...N; The second image is generated according to the brightness value of the j-th pixel in the i-th row.
5. The method according to claim 1, characterized in that Based on the first image and the second image, a stroboscopic strip image is generated by fusing, including: Determine whether there is a point light source in the first image; If there is a point light source in the first image, based on the position and intensity of the point light source in the first image, adjusting the brightness value at the corresponding position in the second image according to a two-dimensional Gaussian distribution to obtain a third image; The stroboscopic strip image is generated by fusing the first image and the third image.
6. The method according to claim 1, characterized in that Based on the first image and the second image, a stroboscopic strip image is generated by fusing, including: Determining a brightness coefficient based on a brightness value of the first image; According to the brightness coefficient, adjusting the brightness value of the second image to obtain a fourth image; The stroboscopic strip image is generated by fusing the first image and the fourth image.
7. An image generating device, characterized in that: The device comprises: An acquisition module, configured to acquire a first image, wherein the first image includes M×N pixels, where M and N are positive integers; A determination module, used to determine a light source brightness function, where the light source brightness function is used to represent a functional relationship between light source brightness and time; A calculation module, configured to perform M time-domain integration operations on the light source brightness function based on preset parameters to obtain M brightness values, wherein the preset parameters include a shooting frequency and an exposure time; An image generation module, configured to generate a second image based on the M brightness values, wherein the second image is a stripe image having M×N pixels, wherein the brightness values of M rows of pixels in the second image respectively correspond to the M brightness values; An image fusion module is used to generate a stroboscopic strip image based on the first image and the second image.
8. The device according to claim 7, characterized in that The computing module is used for: According to the shooting frequency, determining the exposure start time of the i-th row of pixels, i=1, ...M; Determining an exposure end time of the i-th row of pixels according to an exposure start time of the i-th row of pixels and the exposure duration; According to the exposure start time of the pixel points in the i-th row and the exposure end time of the pixel points in the i-th row, the light source brightness function is subjected to M time-domain integration operations to obtain the M brightness values.
9. An electronic device, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method according to any one of claims 1 to 6.
10. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
