Photographing method and apparatus, electronic device, and medium

Abstract

The present disclosure relates to a photographing method, device, electronic device and medium. The photographing method comprises: obtaining brightness change information of an image acquisition area; determining flicker information of the image acquisition area according to the brightness change information; and executing a target strategy according to the flicker information, the target strategy being used to eliminate the influence of the stroboscopic of a light source on an image. By executing the target strategy to eliminate the influence of the stroboscopic, the existence of stripes with light and dark alternation in the collected image is avoided, thereby improving the quality of image shooting. Meanwhile, by eliminating the influence of the stroboscopic through the flicker information of the image acquisition area, the stripes with light and dark alternation caused by the stroboscopic of a local light source in the collected image are avoided, thereby further improving the quality of image shooting.

Description

Technical Field

[0001] This disclosure relates to the field of photography technology, and more particularly to a photography method, apparatus, electronic device, and medium. Background Technology

[0002] Artificial light sources are present in people's daily environments to illuminate their surroundings or display desired images. However, due to the flickering of artificial light sources, the captured images show alternating bright and dark stripes, affecting the quality of the image. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides a shooting method, apparatus, electronic device, and medium.

[0004] According to a first aspect of the present disclosure, a shooting method is provided, the shooting method comprising:

[0005] Acquire brightness change information in the image acquisition area;

[0006] Based on the brightness change information, determine the flicker information of the image acquisition area;

[0007] Based on the flickering information, a target strategy is executed to eliminate the effect of light source flicker on the image.

[0008] In some embodiments of this disclosure, the brightness change information includes event information; acquiring the brightness change information of the image acquisition area includes:

[0009] Acquire raw information from the output of event-based visual sensors;

[0010] The original information is decoded to obtain the event information.

[0011] In some embodiments of this disclosure, the flicker information includes light source intensity and flicker frequency; determining the flicker information of the image acquisition area based on the brightness change information includes:

[0012] The time difference between adjacent event information and two adjacent original information is obtained to determine the light source intensity and the flicker frequency at each location in the image acquisition area.

[0013] In some embodiments of this disclosure, determining the flicker information of the image acquisition area based on the brightness change information includes:

[0014] The brightness change information is input into a first preset model to obtain the flickering information.

[0015] In some embodiments of this disclosure, executing the target strategy based on the flashing information includes:

[0016] If the flickering information reflects that the proportion of the flickering area in the image acquisition area is greater than or equal to a preset proportion, the target strategy of adjusting the exposure parameters is executed.

[0017] If the flickering information indicates that the proportion of the flickering area is less than the preset proportion, the target strategy of locally compensating the acquired image is executed.

[0018] In some embodiments of this disclosure, the execution of the target strategy for adjusting exposure parameters includes:

[0019] Increase the exposure time in the exposure parameters; and / or,

[0020] The target strategy for performing local compensation on the acquired image includes:

[0021] The acquired image is processed using a compensation algorithm to obtain a locally compensated image.

[0022] In some embodiments of this disclosure, the target strategy for performing local compensation on the acquired image includes:

[0023] The acquired image and the flickering information are input into the second preset model to obtain the locally compensated image.

[0024] In some embodiments of this disclosure, after determining the flicker information of the image acquisition area based on the brightness change information, the shooting method further includes:

[0025] Semantic analysis is performed on the acquired images to obtain semantic information;

[0026] Depth estimation is performed on the acquired images to obtain depth information;

[0027] The position of the light source is determined based on the semantic information, the depth information, and the flickering information.

[0028] In some embodiments of this disclosure, determining the light source position based on the semantic information, the depth information, and the flicker information includes:

[0029] The semantic information, the depth information, and the flickering information are input into a third preset model to obtain the position of the light source.

[0030] In some embodiments of this disclosure, after determining the light source position based on the semantic information, the depth information, and the flicker information, the shooting method further includes:

[0031] Based on the light source position and the flicker information, color and / or brightness compensation is performed on the acquired image; and / or,

[0032] The white balance parameters are adjusted based on the position of the light source and the flicker information.

[0033] According to a second aspect of the present disclosure, a shooting device is provided, the shooting device comprising:

[0034] The acquisition module is configured to acquire brightness change information of the image acquisition area;

[0035] A determining module is configured to determine flicker information of the image acquisition area based on the brightness change information;

[0036] An execution module is configured to execute a target strategy based on the flickering information, the target strategy being used to eliminate the effect of light source flicker on the image.

[0037] According to a third aspect of the present disclosure, an electronic device is provided, the electronic device comprising:

[0038] processor;

[0039] Memory used to store the processor's executable instructions;

[0040] The processor is configured to perform the shooting method described above.

[0041] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein when instructions in the storage medium are executed by a processor of a terminal, the terminal is enabled to perform the shooting method described above.

[0042] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0043] The brightness variation information of the image acquisition area is acquired to determine the brightness changes within that area. Since light source flicker is reflected in brightness variations, flicker information within the image acquisition area is determined based on this brightness variation information. Because flicker information reflects the flicker of the light source, a target strategy is executed to eliminate the impact of light source flicker on the image. By implementing this target strategy to eliminate flicker, alternating bright and dark stripes are avoided in the acquired image, thus improving image quality. Simultaneously, by using flicker information from the image acquisition area to eliminate flicker, the effects of localized light source flicker are prevented from producing alternating bright and dark stripes in the acquired image, further improving image quality.

[0044] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0045] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0046] Figure 1 This is a flowchart illustrating a shooting method according to an exemplary embodiment;

[0047] Figure 2 This is a flowchart illustrating a shooting method according to another exemplary embodiment;

[0048] Figure 3 This is a flowchart illustrating a shooting method according to another exemplary embodiment;

[0049] Figure 4 This is a flowchart illustrating a shooting method according to another exemplary embodiment;

[0050] Figure 5 This is a block diagram of a shooting device according to an exemplary embodiment;

[0051] Figure 6 This is a block diagram of an electronic device according to an exemplary embodiment.

[0052] In the picture:

[0053] 100 - Acquisition module; 200 - Determination module; 300 - Execution module; 400 - Electronic device; 402 - Processing component; 404 - Memory; 406 - Power supply component; 408 - Multimedia component; 410 - Audio component; 412 - Input / output interface; 414 - Sensor component; 416 - Communication component; 420 - Processor. Detailed Implementation

[0054] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims. It should also be understood that the term “and / or” as used in this disclosure refers to and includes any or all possible combinations of one or more of the associated listed items.

[0055] Artificial light sources such as lamps and displays are present in people's daily environments. Lamps illuminate the surroundings, and displays show the desired image. Due to the flickering of artificial light sources, images captured by electronic devices using rolling shutter exposure exhibit alternating bright and dark stripes, affecting the quality of the image.

[0056] In related technologies, an electronic device is provided, which incorporates a flicker sensor to detect flicker from a light source and process the acquired image to eliminate its effects. However, because the flicker sensor can only detect flicker at an unknown location within the image acquisition area and cannot detect flicker across the entire area, it struggles to effectively eliminate the flicker effect, resulting in poor image quality. For example, if the image acquisition area includes a display screen, and the flicker sensor detects flicker outside the screen, the alternating bright and dark stripes caused by the screen's flicker will still appear in the acquired image. Furthermore, in low-intensity light conditions, the flicker sensor cannot detect the flicker that causes alternating bright and dark stripes in the image, further contributing to poor image quality.

[0057] To address the aforementioned technical problems, this disclosure provides an image capture method that determines flicker information by analyzing brightness variation information in the image acquisition area. Based on this flicker information, a target strategy is executed to eliminate alternating bright and dark stripes in the captured image, thereby improving image capture quality. Furthermore, by determining flicker information using brightness variation information in the image acquisition area, flicker can be detected even under weak light conditions, further enhancing image capture quality.

[0058] This disclosure provides a shooting method, such as... Figure 1 As shown, the method includes:

[0059] S100: Obtain brightness change information of the image acquisition area.

[0060] S200. Determine the flicker information of the image acquisition area based on the brightness change information.

[0061] S300. Based on the flicker information, execute the target strategy, which is used to eliminate the effect of light source flicker on the image.

[0062] In this embodiment, brightness change information of the image acquisition area is acquired to determine the brightness variation of the image acquisition area. Since the flicker of the light source is reflected in the brightness change, flicker information of the image acquisition area is determined based on the brightness change information. Since the flicker information reflects the flicker of the light source, a target strategy is executed to eliminate the influence of light source flicker on the image. By executing the target strategy to eliminate the influence of flicker, alternating bright and dark stripes are avoided in the acquired image, thereby improving the image quality. Simultaneously, by using the flicker information of the image acquisition area to eliminate the influence of flicker, alternating bright and dark stripes in the acquired image caused by the flicker of local light sources are avoided, further improving the image quality.

[0063] For example, the image acquisition area can be the maximum area that the camera in the electronic device can cover, or it can be a portion of the area that the camera can cover. The brightness change information can reflect the direction of brightness change at each location in the image acquisition area, or it can reflect the magnitude of brightness change at each location.

[0064] In one embodiment, the brightness change information includes event information. For example... Figure 2 As shown, the brightness change information of the image acquisition area in step S100 is determined in the following way:

[0065] S110. Obtain the raw information output by the event-based visual sensor.

[0066] S120. Decode the original information to obtain event information.

[0067] In this embodiment, the event-based vision sensor (EVS) can detect brightness changes at each pixel, acquiring raw information output by the sensor to obtain raw information related to brightness changes. Since the raw information cannot directly reflect the brightness changes at each pixel, it is decoded to obtain event information reflecting brightness changes in the image acquisition area. Obtaining event information from the raw information detected by the event-based vision sensor allows for brightness change detection at every location in the image acquisition area, avoiding the omission of flickering light sources and thus improving image capture quality. Furthermore, because the event-based vision sensor is reusable, there is no need for an additional anti-flicker light sensor, reducing the complexity of the electronic device structure.

[0068] For example, the event information includes event frames. An event frame consists of event pixels, which are generated when the brightness of the corresponding pixel changes, and not generated when the brightness of the corresponding pixel does not change. That is, the number of event pixels can be the same as the number of pixels in the image acquisition area, or it can be 0, or it can be between 0 and the number of pixels in the image acquisition area. Event pixels can be represented in the following form:

[0069] event = (t, x, y, p);

[0070] Where event represents the event pixel, t represents the timestamp, (x,y) represents the coordinates of the pixel whose brightness changed, and p represents the polarity, which is 1 when the brightness increases and 0 when the brightness decreases.

[0071] In one embodiment, the flicker information includes the light source intensity and flicker frequency (i.e., flicker position distribution). The flicker information of the image acquisition area in step S200, determined based on the brightness change information, can be determined in the following way:

[0072] The time difference between multiple adjacent event information and two adjacent raw information is obtained to determine the light source intensity and the flicker frequency at each location in the image acquisition area.

[0073] In this embodiment, since the event-based visual sensor outputs raw information at regular time intervals, the time it takes for the brightness at each location to change from brightest to darkest (or darkest to brightest) can be determined. By obtaining the time difference between multiple adjacent event information and two adjacent raw information, the time it takes for the brightness at each location to change from brightest to darkest (or darkest to brightest) is analyzed to determine the light source intensity and the flicker frequency at each location within the image acquisition area. By using an event-based visual sensor to obtain the light source intensity and the flicker frequency at each location, flicker information can be determined from multiple dimensions to eliminate the effects of flicker, thereby improving the quality of image capture.

[0074] For example, the steps described above, which involve obtaining the time difference between multiple adjacent event information and two adjacent original information, and determining the light source intensity and the flicker frequency at each location in the image acquisition area, can be performed by combining the image acquired by the image sensor with the determination of the light source intensity, or without combining the image acquired by the image sensor with the determination of the light source intensity. When determining the light source intensity by combining the image acquired by the image sensor with the determination of the light source intensity, the brightness of each location in the acquired image at different times, as well as the brightness changes of each location in the event information at different times, can be used to determine the light source intensity more accurately.

[0075] In one embodiment, determining the flicker information of the image acquisition area based on the brightness change information in step S200 can also be achieved in the following way:

[0076] The brightness change information is input into the first preset model to obtain the flicker information.

[0077] In this embodiment, the flicker information is obtained by processing the brightness change information through a pre-trained first preset model. The flicker information is determined quickly and is consistent with the shooting environment, so as to effectively eliminate the influence caused by flicker, thereby improving the reliability of image shooting.

[0078] For example, an electronic device can pre-set a first preset model for determining flicker information. Before training the first preset model, the viewing angles of the image sensor and the event-based vision sensor can be fixed in a real environment, pointing towards a preset direction. Light sources with different numbers, color temperatures, intensities, positions, and flicker frequencies are set up. In various real-world environments, the image sensor acquires multiple images, and the event-based vision sensor acquires multiple pieces of raw information to determine brightness change information, serving as the first training data in the real-world environment. In a simulation environment, the real-world environment is simulated, and light sources with different numbers, color temperatures, intensities, positions, and flicker frequencies are set up. Multiple images and multiple pieces of brightness change information are simulated, serving as the second training data in the simulation environment. The task of the first preset model is set to input brightness change information and output flicker information. The first preset model is then trained using the first and second training data.

[0079] For example, the flicker information of the image acquisition area in step S200, based on the brightness change information, can also be determined in the following way:

[0080] The brightness change information and the acquired image are input into the first preset model to obtain flicker information. At this time, during the training of the first preset model, the task is set to input brightness change information and the acquired image, and output flicker information.

[0081] For example, the flicker information of the image acquisition area in step S200, based on the brightness change information, can also be determined in the following way:

[0082] The brightness change information is processed using computer vision algorithms to obtain flicker information.

[0083] For example, the flicker information of the image acquisition area in step S200, based on the brightness change information, can also be determined in the following way:

[0084] The flicker information is obtained by processing the brightness change information and the acquired image using computer vision algorithms.

[0085] In one embodiment, the target strategy in step S300, based on the flashing information, can be determined in the following way:

[0086] If the proportion of the flickering area in the image acquisition area reflected by the flickering information is greater than or equal to a preset proportion, the target strategy of adjusting the exposure parameters is executed.

[0087] If the proportion of the flickering area reflected in the flickering information is less than a preset proportion, a target strategy for local compensation of the acquired image is executed.

[0088] In this embodiment, when the proportion of the flickering area in the image acquisition area reflected by the flickering information is greater than or equal to a preset proportion, the influence range of the light source flicker is large and difficult to eliminate by compensation, so a target strategy of adjusting exposure parameters is executed. When the proportion of the flickering area reflected by the flickering information is less than the preset proportion, the influence range of the light source flicker is small and not suitable for elimination by adjusting exposure parameters, so a target strategy of local compensation for the acquired image is executed. By executing the corresponding target strategy according to the proportion of the flickering area, the image quality can be guaranteed while eliminating the influence of flicker, thereby improving the image capture quality.

[0089] For example, the preset ratio can range from 30% to 70%. The preset ratio can be 40%, 50%, 60%, etc.

[0090] For example, since the event pixel in the event information is generated when the brightness of the corresponding pixel changes, and is not generated when the brightness of the corresponding pixel does not change, the flickering area can be determined based on the event information. That is, the flickering information may also include the flickering area.

[0091] In one embodiment, the target strategy for adjusting exposure parameters in the above steps is determined as follows:

[0092] Increase the exposure time in the exposure parameters.

[0093] In this embodiment, by increasing the exposure time, the flicker of the light source can be averaged out during image acquisition without affecting the acquired image, thereby improving the quality of image capture.

[0094] For example, increasing the exposure time in the exposure parameters in the above steps can be done by increasing the exposure time determined based on the image acquired by the image sensor. The increased exposure time can be a multiple of the flash time corresponding to the flash frequency. For example, if the flash time is 0.02s, the increased exposure time can be 0.04s, 0.06s, etc.

[0095] It is understandable that, in addition to increasing the exposure time, other parameters in the exposure settings can also be adjusted; this is not limited here.

[0096] In one embodiment, the target strategy for performing local compensation on the acquired image in the above steps can be determined in the following way:

[0097] The acquired image is processed using a compensation algorithm to obtain a locally compensated image.

[0098] In this embodiment, by processing the acquired image with a compensation algorithm to obtain a locally compensated image, the influence of flicker on the local image can be eliminated and the acquired image can be avoided as blurry, thereby improving the quality of image capture.

[0099] For example, the process of processing the acquired image with a compensation algorithm in the above steps to obtain a locally compensated image can be performed by processing the image located in the flickering area with a compensation algorithm to obtain a locally compensated image.

[0100] In one embodiment, the target strategy for performing local compensation on the acquired image in the above steps can also be determined in the following way:

[0101] The acquired image and flicker information are input into the second preset model to obtain the locally compensated image.

[0102] In this embodiment, the acquired image is processed by combining a pre-trained second preset model with flicker information. The image processing method is fast and consistent with the shooting environment to effectively eliminate the effects caused by flicker, thereby improving the quality of image capture.

[0103] For example, an electronic device can pre-set a pre-trained second preset model for local image processing. Before training the second preset model, the viewing angles of the image sensor and the event-based vision sensor can be fixed in a real environment, pointing towards a preset direction. Light sources with different numbers, color temperatures, intensities, positions, and flicker frequencies are set, allowing the event-based vision sensor to collect multiple raw information sets to determine the flicker frequency. The exposure time of the image sensor is adjusted to be the same as or close to the flicker time corresponding to the flicker frequency, and multiple images with alternating bright and dark stripes are collected as third training data. The exposure time of the image sensor is adjusted to be multiple times the flicker time, and multiple images without alternating bright and dark stripes are collected as fourth training data. The exposure time of the image sensor is adjusted to be multiple times less than the flicker time, and multiple images with alternating bright and dark stripes are collected, with the image brightness increased, as fifth training data. The task of the second preset model is set to input the collected images and flicker information, and output a locally compensated image. The second preset model is trained using the third, fourth, and fifth training data and the event information determined by the multiple raw information sets.

[0104] For example, the execution of the target strategy based on the flashing information in step S300 can also be determined in the following way:

[0105] If the flickering information indicates the presence of light source flickering in the image acquisition area, the target strategy is executed based on the flickering information.

[0106] For example, after determining the flicker information of the image acquisition area based on the brightness change information in step S200, the shooting method further includes:

[0107] If the flickering information indicates that there is no light source flickering in the image acquisition area, the target strategy will not be executed.

[0108] In one embodiment, such as Figure 3 As shown, after determining the flicker information of the image acquisition area based on the brightness change information in step S200, the shooting method further includes:

[0109] S400. Perform semantic analysis on the acquired images to obtain semantic information.

[0110] S410. Perform depth estimation on the acquired image to obtain depth information.

[0111] S420. Determine the position of the light source based on semantic information, depth information, and blink information.

[0112] In this embodiment, semantic analysis is performed on the acquired images to obtain semantic information, which reflects the objects at each location in the image acquisition area. Depth estimation is performed on the acquired images to obtain depth information, which reflects the depth at each location in the image acquisition area. Since the position of the light source determines the illumination of objects at different depths and locations, the position of the light source is determined based on the semantic information, depth information, and flicker information. Determining the light source position using semantic information, depth information, and flicker information allows for adjustments to the acquired images based on the light source position, thereby improving the quality of image capture.

[0113] For example, the semantic analysis of the acquired image in step S400 to obtain semantic information can be performed by processing the acquired image using a semantic segmentation algorithm. The semantic information reflects the objects at each location within the image acquisition area. For instance, the semantic information might indicate that a soccer ball is located in the first area of ​​the image acquisition area, a computer is located in the second area, and a television is located in the third area. The depth estimation of the acquired image in step S410 to obtain depth information can also be performed by processing the acquired image using a depth estimation algorithm.

[0114] In one embodiment, the determination of the light source position in step S420 based on semantic information, depth information, and flicker information is performed in the following manner:

[0115] The semantic information, depth information, and flicker information are input into the third preset model to obtain the position of the light source.

[0116] In this embodiment, the position of the light source is obtained by combining semantic information, depth information and flicker information through a pre-trained third preset model. The method of determining the position of the light source is fast and consistent with the shooting environment, so as to effectively adjust the acquired image and improve the quality of image shooting.

[0117] For example, an electronic device can pre-set a pre-trained third preset model to determine the position of the light source. Before training the third preset model, first training data and second training data can be collected, and corresponding semantic information and depth information can be obtained as sixth training data. The task of the third preset model is set to input semantic information, depth information, and flicker information, and output the position of the light source. The third preset model is trained using the first training data, the second training data, and the sixth training data.

[0118] In one embodiment, after determining the light source position based on semantic information, depth information, and flicker information in step S420, the imaging method further includes:

[0119] Based on the position of the light source and flicker information, color and / or brightness compensation is performed on the acquired image.

[0120] In this embodiment, since the position of the light source reflects the illumination situation, color and / or brightness compensation is performed on the acquired image based on the position of the light source and flicker information. By performing color and / or brightness compensation on the acquired image using the position of the light source and flicker information, image color restoration and reconstruction can be achieved, making the image more realistic and thus improving the quality of image capture.

[0121] For example, the step of performing color and / or brightness compensation on the acquired image based on the light source position and flicker information can be to strip away the light source and retain only the reflected light, and set the color and / or brightness separately, or to warm or cool the hue of different locations in the acquired image based on the light source position and flicker information.

[0122] In one embodiment, after determining the light source position based on semantic information, depth information, and flicker information in step S420, the imaging method further includes:

[0123] The white balance parameters are adjusted based on the position of the light source and flicker information.

[0124] In this embodiment, since the position of the light source reflects the illumination situation, the white balance parameters are adjusted based on the position of the light source and flicker information. Adjusting the white balance parameters by considering the position of the light source and flicker information allows for more realistic colors in subsequent image acquisition, thereby improving the quality of image capture.

[0125] For example, since the color temperature of the light source needs to be estimated during the white balance parameter setting process, the light source can be decoupled using the light source position and flicker information to obtain the true color temperature without estimation. Compared to the estimated color temperature, setting the white balance parameters with the true color temperature results in a more realistic image, thus improving the quality of image capture.

[0126] This disclosure provides a shooting method, such as... Figure 4 As shown, the method includes:

[0127] S500: Acquire raw information from the event-based visual sensor output.

[0128] S510. Decode the original information to obtain event information.

[0129] S520. Based on the time difference between multiple adjacent event information and two adjacent original information, determine the light source intensity and the flicker frequency of each position in the image acquisition area as flicker information.

[0130] S530. If the proportion of the flashing area in the image acquisition area reflected by the flashing information is greater than or equal to the preset proportion, increase the exposure time in the exposure parameters.

[0131] S540. When the proportion of the flashing area reflected by the flashing information is less than the preset proportion, the acquired image is processed by a compensation algorithm to obtain a locally compensated image.

[0132] S550. Perform semantic analysis on the acquired images to obtain semantic information.

[0133] S560. Perform depth estimation on the acquired image to obtain depth information.

[0134] S570. Determine the position of the light source based on semantic information, depth information, and blink information.

[0135] S580. Based on the position of the light source and flicker information, perform color and / or brightness compensation on the acquired image.

[0136] S590. Adjust the white balance parameters based on the position of the light source and flicker information.

[0137] In this embodiment, raw information from the output of an event-based visual sensor is acquired to obtain raw information related to brightness changes in the image acquisition area. The raw information is decoded to obtain event information, which is used as brightness change information to reflect brightness changes in the image acquisition area. Since the brightness change caused by the flicker of the light source is cyclical and gradual, the time difference between multiple adjacent event information and two adjacent raw information is obtained to determine the light source intensity and the flicker frequency at each location in the image acquisition area as flicker information. When the proportion of the flickering area reflected by the flicker information in the image acquisition area is greater than or equal to a preset proportion, the proportion of alternating bright and dark stripes in the acquired image is relatively large, and the exposure time in the exposure parameters is increased. When the proportion of the flickering area reflected by the flicker information is less than the preset proportion, the proportion of alternating bright and dark stripes in the acquired image is relatively small, and a compensation algorithm is used to process the acquired image to obtain a locally compensated image. Semantic analysis is performed on the acquired image to obtain semantic information, which reflects objects at different locations in the image acquisition area. Depth estimation is performed on the acquired image to obtain depth information, which reflects the depth at each location in the image acquisition area. Based on the semantic information, depth information, and flicker information, the position of the light source is analyzed and determined. Based on the light source location and flicker information, color and / or brightness compensation is performed on the acquired image to restore and reconstruct colors. White balance parameters are adjusted based on the light source location and flicker information to match the actual light source. By implementing a targeted strategy, the effects of flicker are eliminated, preventing alternating bright and dark stripes in the acquired image, thereby improving image quality. Simultaneously, flicker information from the image acquisition area is used to eliminate the effects of flicker, preventing flicker from localized light sources from producing alternating bright and dark stripes in the acquired image, further improving image quality.

[0138] In one exemplary embodiment, a shooting device is provided for implementing the method described above. (Reference) Figure 5 As shown, the imaging device may include an acquisition module 100, a determination module 200, and an execution module 300, wherein, during the implementation of the above method,

[0139] The acquisition module 100 is configured to acquire brightness change information of the image acquisition area.

[0140] The determination module 200 is configured to determine the flicker information of the image acquisition area based on the brightness change information.

[0141] The execution module 300 is configured to execute a target strategy based on flicker information, the target strategy being used to eliminate the effect of light source flicker on the image.

[0142] In one exemplary embodiment, a shooting device is provided, wherein an acquisition module 100 is configured to:

[0143] Obtain raw information from the output of event-based visual sensors.

[0144] The original information is decoded to obtain the event information.

[0145] In one exemplary embodiment, a shooting apparatus is provided, wherein a determining module 200 is configured to:

[0146] The time difference between multiple adjacent event information and two adjacent raw information is obtained to determine the light source intensity and the flicker frequency at each location in the image acquisition area.

[0147] In one exemplary embodiment, a shooting apparatus is provided, wherein a determining module 200 is configured to:

[0148] The brightness change information is input into the first preset model to obtain the flicker information.

[0149] In one exemplary embodiment, a shooting device is provided, wherein the execution module 300 is configured to:

[0150] If the proportion of the flickering area in the image acquisition area reflected by the flickering information is greater than or equal to a preset proportion, the target strategy of adjusting the exposure parameters is executed.

[0151] If the proportion of the flickering area reflected in the flickering information is less than a preset proportion, a target strategy for local compensation of the acquired image is executed.

[0152] In one exemplary embodiment, a shooting device is provided, wherein the execution module 300 is configured to:

[0153] Increase the exposure time in the exposure parameters.

[0154] In one exemplary embodiment, a shooting device is provided, wherein the execution module 300 is configured to:

[0155] The acquired image is processed using a compensation algorithm to obtain a locally compensated image.

[0156] In one exemplary embodiment, a shooting device is provided, wherein the execution module 300 is configured to:

[0157] The acquired image and flicker information are input into the second preset model to obtain the locally compensated image.

[0158] In one exemplary embodiment, a shooting device is provided, the device further comprising:

[0159] The analysis module is configured to perform semantic analysis on the acquired images to obtain semantic information.

[0160] The estimation module is configured to perform depth estimation on the acquired images to obtain depth information.

[0161] In one exemplary embodiment, a shooting apparatus is provided, wherein a determining module 200 is configured to:

[0162] The location of the light source is determined based on semantic information, depth information, and flicker information.

[0163] In one exemplary embodiment, a shooting apparatus is provided, wherein a determining module 200 is configured to:

[0164] The semantic information, depth information, and flicker information are input into the third preset model to obtain the position of the light source.

[0165] In one exemplary embodiment, a shooting device is provided, the device further comprising:

[0166] The adjustment module is configured to perform color and / or brightness compensation on the acquired image based on the light source position and flicker information.

[0167] In one exemplary embodiment, an imaging device is provided, wherein an adjustment module is configured to:

[0168] The white balance parameters are adjusted based on the position of the light source and flicker information.

[0169] In one exemplary embodiment, an electronic device is provided, such as a mobile phone, a laptop computer, a tablet computer, and a wearable device.

[0170] refer to Figure 6 As shown, the electronic device 400 may include one or more of the following components: processing component 402, memory 404, power supply component 406, multimedia component 408, audio component 410, input / output (I / O) interface 412, sensor component 414, and communication component 416.

[0171] Processing component 402 typically controls the overall operation of electronic device 400, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 402 may include one or more processors 420 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 402 may include one or more modules to facilitate interaction between processing component 402 and other components. For example, processing component 402 may include a multimedia module to facilitate interaction between multimedia component 408 and processing component 402.

[0172] Memory 404 is configured to store various types of data to support the operation of electronic device 400. Examples of this data include instructions for any application or method operating on electronic device 400, contact data, phonebook data, messages, pictures, videos, etc. Memory 404 can be implemented by any type of volatile or non-volatile storage terminal or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0173] Power supply component 406 provides power to various components of electronic device 400. Power supply component 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 400.

[0174] Multimedia component 408 includes a screen that provides an output interface between electronic device 400 and user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 408 includes a front-facing camera module and / or a rear-facing camera module. When electronic device 400 is in an operating mode, such as shooting mode or video mode, the front-facing camera module and / or rear-facing camera module may receive external multimedia data. Each front-facing camera module and rear-facing camera module may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0175] Audio component 410 is configured to output and / or input audio signals. For example, audio component 410 includes a microphone (MIC) configured to receive external audio signals when electronic device 400 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 404 or transmitted via communication component 416. In some embodiments, audio component 410 also includes a speaker for outputting audio signals.

[0176] I / O interface 412 provides an interface between processing component 402 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0177] Sensor assembly 414 includes one or more sensors for providing state assessments of various aspects of electronic device 400. For example, sensor assembly 414 may detect the on / off state of electronic device 400, the relative positioning of components such as the display and keypad of electronic device 400, changes in position of electronic device 400 or a component of electronic device 400, the presence or absence of user contact with electronic device 400, orientation or acceleration / deceleration of electronic device 400, and temperature changes of electronic device 400. Sensor assembly 414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 414 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 414 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0178] Communication component 416 is configured to facilitate wired or wireless communication between electronic device 400 and other terminals. Electronic device 400 can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G, 5G, or combinations thereof. In one exemplary embodiment, communication component 416 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 416 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0179] In an exemplary embodiment, the electronic device 400 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing terminals (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods shown in the above embodiments or combinations thereof.

[0180] In one exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 404 including instructions, which can be executed by a processor 420 of an electronic device 400 to perform the methods shown in the embodiments or combinations thereof. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage terminal, etc. When the instructions in the storage medium are executed by the processor of the terminal, the terminal is able to perform the methods shown in the embodiments or combinations thereof.

[0181] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0182] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0183] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0184] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0185] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A shooting method, characterized in that, The shooting method includes: Acquire brightness change information in the image acquisition area; Based on the brightness change information, determine the flicker information of the image acquisition area; Based on the flickering information, a target strategy is executed to eliminate the effect of light source flicker on the image.

2. The shooting method according to claim 1, characterized in that, The brightness change information includes event information; the acquisition of brightness change information of the image acquisition area includes: Acquire raw information from the output of event-based visual sensors; The original information is decoded to obtain the event information.

3. The shooting method according to claim 2, characterized in that, The flicker information includes light source intensity and flicker frequency; determining the flicker information of the image acquisition area based on the brightness change information includes: The time difference between adjacent event information and two adjacent original information is obtained to determine the light source intensity and the flicker frequency at each location in the image acquisition area.

4. The shooting method according to claim 1, characterized in that, Determining the flicker information of the image acquisition area based on the brightness change information includes: The brightness change information is input into a first preset model to obtain the flickering information.

5. The shooting method according to claim 1, characterized in that, The step of executing the target strategy based on the flashing information includes: If the flickering information reflects that the proportion of the flickering area in the image acquisition area is greater than or equal to a preset proportion, the target strategy of adjusting the exposure parameters is executed. If the flickering information indicates that the proportion of the flickering area is less than the preset proportion, the target strategy of locally compensating the acquired image is executed.

6. The shooting method according to claim 5, characterized in that, The target strategy for implementing the adjustment of exposure parameters includes: Increase the exposure time in the exposure parameters; and / or, The target strategy for performing local compensation on the acquired image includes: The acquired image is processed using a compensation algorithm to obtain a locally compensated image.

7. The shooting method according to claim 5, characterized in that, The target strategy for performing local compensation on the acquired image includes: The acquired image and the flickering information are input into the second preset model to obtain the locally compensated image.

8. The shooting method according to any one of claims 1 to 7, characterized in that, After determining the flicker information of the image acquisition area based on the brightness change information, the shooting method further includes: Semantic analysis is performed on the acquired images to obtain semantic information; Depth estimation is performed on the acquired images to obtain depth information; The position of the light source is determined based on the semantic information, the depth information, and the flickering information.

9. The shooting method according to claim 8, characterized in that, Determining the light source position based on the semantic information, the depth information, and the flicker information includes: The semantic information, the depth information, and the flickering information are input into a third preset model to obtain the position of the light source.

10. The shooting method according to claim 8, characterized in that, After determining the light source position based on the semantic information, the depth information, and the flicker information, the shooting method further includes: Based on the light source position and the flicker information, color and / or brightness compensation is performed on the acquired image; and / or, The white balance parameters are adjusted based on the position of the light source and the flicker information.

11. A shooting device, characterized in that, The imaging device includes: The acquisition module is configured to acquire brightness change information of the image acquisition area; A determining module is configured to determine flicker information of the image acquisition area based on the brightness change information; An execution module is configured to execute a target strategy based on the flickering information, the target strategy being used to eliminate the effect of light source flicker on the image.

12. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store the processor's executable instructions; The processor is configured to perform the shooting method as described in any one of claims 1 to 10.

13. A non-transitory computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the terminal, the terminal is able to perform the shooting method as described in any one of claims 1 to 10.

Images