Image generation method, apparatus and system, device, and storage medium

Abstract

An image generation method, apparatus and system, a device, and a storage medium. The image generation method comprises: controlling an optical filter to set an optical filtering degree to a first optical filtering degree in a first exposure period corresponding to a first pixel unit in a single image frame acquired by an image acquisition apparatus, and acquiring first pixel data of the first pixel unit acquired by the image acquisition apparatus in the first exposure period; controlling the optical filter to adjust the optical filtering degree from the first optical filtering degree to a second optical filtering degree in a second exposure period corresponding to a second pixel unit in the single image frame acquired by the image acquisition apparatus, and acquiring second pixel data of the second pixel unit acquired by the image acquisition apparatus in the second exposure period; and outputting the single image frame comprising the first pixel data and the second pixel data. The method can effectively improve the photographing effect.

Description

Image generation method, device, system, apparatus, and storage medium TECHNICAL FIELD

[0001] The present application relates to the technical field of images, in particular to an image generation method, an image generation device, an image generation system, an electronic device, and a computer readable storage medium. BACKGROUND

[0002] A neutral density filter (ND filter) is a gray filter and is a commonly used device in photography, mainly used to reduce the amount of light entering the lens, thereby reducing the exposure amount. The neutral density filter allows photographers to use longer shutter speeds or larger apertures, such as long exposure or shallow depth of field effects.

[0003] At present, the neutral density filter can only achieve global light reduction, that is, uniformly reducing the light intensity in the entire image area, which cannot meet the shooting requirements in some scenes, resulting in poor shooting effects.

[0004] SUMMARY

[0005] Therefore, one of the purposes of the present application is to provide an image generation method, an image generation device, an image generation system, an electronic device, and a computer readable storage medium.

[0006] In a first aspect, an image generation method is provided, comprising:

[0007] controlling an electronic filter to set a voltage value to a first voltage value during a first exposure period corresponding to a first pixel unit in a single frame image collected by an image collection device, and obtaining first pixel data of the first pixel unit collected by the image collection device in the first exposure period, wherein the electronic filter is configured to filter light entering a lens assembly of the image collection device;

[0008] controlling the electronic filter to adjust the voltage value from the first voltage value to a second voltage value during a second exposure period corresponding to a second pixel unit in the single frame image collected by the image collection device, and obtaining second pixel data of the second pixel unit collected by the image collection device in the second exposure period, wherein the start time of the second exposure period is located after the first exposure period, the first voltage value is different from the second voltage value, and the first pixel unit is different from the second pixel unit;

[0009] outputting the single frame image including the first pixel data and the second pixel data.

[0010] By setting the voltage value of the electronic filter to the first voltage value in the first exposure period of the single frame image, and adjusting the voltage value of the electronic filter from the first voltage value to the second voltage value in the second exposure period, the voltage of the electronic filter in different exposure periods of the same frame image is controlled to be different voltage values, so that the filtering degree of the electronic filter is different when different pixel units of the same frame image are collected, and different filtering degrees of different pixel units in the same frame image are achieved to adapt to scenes with light and dark contrasts and improve the shooting effect.

[0011] In a second aspect, an embodiment of the present application provides an image generation method, comprising:

[0012] obtaining first pixel data of a first pixel unit in a single frame image collected by an image collection device in a first exposure period, wherein a voltage value of an electronic filter of the image collection device is a first voltage value in the first exposure period, and the electronic filter is configured to filter light entering a lens assembly of the image collection device;

[0013] obtaining second pixel data of a second pixel unit in the single frame image collected by the image collection device in a second exposure period, wherein the voltage value of the electronic filter is adjusted from the first voltage value to a second voltage value in the second exposure period, a starting time of the second exposure period is after the first exposure period, the first voltage value is different from the second voltage value, and the first pixel unit is different from the second pixel unit;

[0014] outputting the single frame image including the first pixel data and the second pixel data.

[0015] By setting the voltage value of the electronic filter to the first voltage value in the first exposure period of the single frame image, and adjusting the voltage value of the electronic filter from the first voltage value to the second voltage value in the second exposure period, the voltage of the electronic filter in different exposure periods of the same frame image is controlled to be different voltage values, so that the filtering degree of the electronic filter is different when different pixel units of the same frame image are collected, and different filtering degrees of different pixel units in the same frame image are achieved to adapt to scenes with light and dark contrasts and improve the shooting effect.

[0016] In a third aspect, an embodiment of the present application provides an image generation method, comprising:

[0017] controlling a filter to set a filtering degree to a first filtering degree in a first exposure period corresponding to a first pixel unit in a single frame image collected by an image collection device, and obtaining first pixel data of the first pixel unit collected by the image collection device in the first exposure period, wherein the filter is configured to filter light entering a lens assembly of the image collection device.

[0018] controlling the filter to adjust the filtering degree from the first filtering degree to a second filtering degree in a second exposure period corresponding to a second pixel unit of the single-frame image, and acquiring second pixel data of the second pixel unit collected by the image collection device in the second exposure period, wherein a starting time of the second exposure period is after the first exposure period, the first filtering degree is different from the second filtering degree, and the first pixel unit is different from the second pixel unit;

[0019] outputting the single-frame image including the first pixel data and the second pixel data.

[0020] By setting the filtering degree of the filter to the first filtering degree in the first exposure period of the single-frame image and adjusting the filtering degree of the filter from the first filtering degree to the second filtering degree in the second exposure period, the filtering degree of the filter in different exposure periods of the same frame image is controlled to be different, so that the filtering degree of the filter is different when collecting different pixel units of the same frame image, and different filtering degrees of different pixel units in the same frame image are achieved to adapt to scenes with light and dark contrasts and improve the shooting effect.

[0021] In a fourth aspect, an embodiment of the present application provides an image generation method, including:

[0022] acquiring first pixel data of a first pixel unit in a single-frame image collected by an image collection device in a first exposure period, wherein a filtering degree of a filter of the image collection device in the first exposure period is a first filtering degree, and the filter is used to filter light entering a lens assembly of the image collection device;

[0023] acquiring second pixel data of a second pixel unit in the single-frame image collected by the image collection device in a second exposure period, wherein the filtering degree of the filter in the second exposure period is adjusted from the first filtering degree to a second filtering degree, a starting time of the second exposure period is after the first exposure period, the first filtering degree is different from the second filtering degree, and the first pixel unit is different from the second pixel unit;

[0024] outputting the single-frame image including the first pixel data and the second pixel data.

[0025] By setting the filtering degree of the filter to a first filtering degree in a first exposure period of a single frame image and adjusting the filtering degree of the filter from the first filtering degree to a second filtering degree in a second exposure period, the filtering degree of the filter in different exposure periods of the same frame image is controlled to be different filtering degrees, so that the filtering degree of the filter is different when different pixel units of the same frame image are collected, and different filtering degrees corresponding to different pixel units in the same frame image are achieved to adapt to scenes with light and dark contrasts and improve the shooting effect.

[0026] In a fifth aspect, an embodiment of the present application provides an image generation apparatus, comprising:

[0027] one or more processors; and

[0028] one or more memories for storing computer program instructions;

[0029] The one or more processors are configured to invoke the computer program instructions to perform the method of the first aspect, the second aspect, the third aspect, or the fourth aspect.

[0030] In a sixth aspect, an embodiment of the present application provides an image generation system, comprising a processor and an image sensor;

[0031] The processor is configured to control an electronic filter to set a voltage value to a first voltage value in a first exposure period corresponding to a first pixel unit in a single frame image collected by the image sensor, wherein the electronic filter is configured to filter light entering a lens assembly of the image sensor.

[0032] The image sensor is configured to collect first pixel data of the first pixel unit in the first exposure period.

[0033] The processor is further configured to control the electronic filter to adjust the voltage value from the first voltage value to a second voltage value in a second exposure period corresponding to a second pixel unit in the single frame image collected by the image sensor, wherein a starting time of the second exposure period is located after the first exposure period, the first voltage value is different from the second voltage value, and the first pixel unit is different from the second pixel unit.

[0034] The image sensor is further configured to collect second pixel data of the second pixel unit in the second exposure period.

[0035] The processor is further configured to output the single frame image comprising the first pixel data and the second pixel data.

[0036] In a seventh aspect, an embodiment of the present application provides an image generation system, comprising a processor and an image sensor;

[0037] The image sensor is configured to: collect first pixel data of a first pixel unit in a single frame image in a first exposure period, wherein a voltage value of an electronic filter of the image sensor is a first voltage value in the first exposure period, and the electronic filter is configured to filter light entering a lens assembly of the image sensor;

[0038] The image sensor is further configured to: collect second pixel data of a second pixel unit in the single frame image in a second exposure period, wherein the voltage value of the electronic filter is adjusted from the first voltage value to a second voltage value in the second exposure period, a starting time of the second exposure period is located after the first exposure period, the first voltage value is different from the second voltage value, and the first pixel unit is different from the second pixel unit.

[0039] The processor is configured to: output the single frame image including the first pixel data and the second pixel data.

[0040] In an eighth aspect, an embodiment of the present application provides an image generation system, including a processor and an image sensor; the processor is configured to: control an electronic filter to set a filtering degree to a first filtering degree in a first exposure period corresponding to a first pixel unit in a single frame image collected by the image sensor, wherein the electronic filter is configured to filter light entering a lens assembly of the image sensor.

[0041] The image sensor is configured to: collect first pixel data of the first pixel unit in the first exposure period.

[0042] The processor is further configured to: control the electronic filter to adjust the filtering degree from the first filtering degree to a second filtering degree in a second exposure period corresponding to a second pixel unit in the single frame image collected by the image sensor, wherein a starting time of the second exposure period is located after the first exposure period, the first filtering degree is different from the second filtering degree, and the first pixel unit is different from the second pixel unit.

[0043] The image sensor is configured to: collect second pixel data of the second pixel unit in the second exposure period.

[0044] The processor is further configured to output the single frame image including the first pixel data and the second pixel data.

[0045] In a ninth aspect, an embodiment of the present application provides an image generation system, including a processor and an image sensor.

[0046] The image sensor is configured to: collect first pixel data of a first pixel unit in a single frame image in a first exposure period, wherein a filtering degree of a filter of the image sensor in the first exposure period is a first filtering degree, and the filter is configured to filter light entering a lens assembly of the image sensor.

[0047] The image sensor is further configured to: collect second pixel data of a second pixel unit in the single frame image in a second exposure period, wherein the filtering degree of the filter is adjusted from the first filtering degree to a second filtering degree in the second exposure period, a starting time of the second exposure period is after the first exposure period, the first filtering degree is different from the second filtering degree, and the first pixel unit is different from the second pixel unit.

[0048] The processor is configured to: output the single frame image including the first pixel data and the second pixel data.

[0049] In a tenth aspect, an embodiment of the present application provides an electronic device, including the image generation apparatus in the fifth aspect, or including the image generation system in the sixth aspect, the seventh aspect, the eighth aspect, or the ninth aspect.

[0050] In an eleventh aspect, an embodiment of the present application provides a computer readable storage medium, which stores computer instructions, and the instructions are executed by a processor to implement the steps of the method in the first aspect, the second aspect, the third aspect, or the fourth aspect. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0052] FIG. 1 is a structural schematic diagram of an image generation system according to an embodiment of the present application.

[0053] FIG. 2 is a structural schematic diagram of an electronic device according to an embodiment of the present application.

[0054] FIG. 3 is a structural schematic diagram of another image generation system according to an embodiment of the present application.

[0055] FIG. 4 is a flow schematic diagram of a first image generation method according to an embodiment of the present application.

[0056] FIG. 5 is a schematic diagram of exposure periods of different pixel rows in a single frame image according to an embodiment of the present application.

[0057] Figure 6 is a schematic diagram of the correspondence between shooting scenes and pixel data provided in the embodiments of this application.

[0058] Figure 7 is a schematic diagram of communication between an unmanned aerial vehicle and a remote control device provided in an embodiment of this application.

[0059] Figure 8 is a schematic diagram of another image generation system and electronic filter provided in an embodiment of this application.

[0060] Figure 9 is a schematic diagram of a transmittance control curve provided in an embodiment of this application.

[0061] Figure 10 is a flowchart illustrating the second image generation method provided in an embodiment of this application.

[0062] Figure 11 is a flowchart illustrating the third image generation method provided in the embodiments of this application.

[0063] Figure 12 is a flowchart illustrating the fourth image generation method provided in the embodiments of this application.

[0064] Figure 13 is a schematic diagram of the structure of an image generation device provided in an embodiment of this application. Detailed Implementation

[0065] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0066] Currently, neutral density filters can only achieve global light reduction, that is, uniformly reduce the light intensity throughout the entire image area. This cannot meet the shooting needs of certain scenarios, resulting in poor image quality. For example:

[0067] When shooting scenes with high contrast, such as sunset scenes where the sky is brighter and the ground is darker, using a neutral density filter, which has a uniform global light reduction, will not only darken the bright areas but also make the dark areas even darker. This is not suitable for scenes with high contrast, resulting in a darker overall photo and poor shooting quality.

[0068] Based on this, embodiments of this application provide an image generation method and related equipment, which can control the filter to different filtering degrees when exposing different pixel units of the same frame image at different exposure times during the continuous exposure duration of a frame image. This effectively solves the limitations of traditional neutral density filters and enables the filter to have different filtering degrees when capturing different pixel units of the same frame image, so that different pixel units in the same frame image correspond to different filtering degrees, in order to adapt to scenes with contrast between light and dark and improve the shooting effect.

[0069] Referring to FIG. 1, FIG. 1 is a structural schematic diagram of an image generation system 100 provided in an embodiment of the present application. The image generation system 100 includes an image generation apparatus 10 and an image acquisition apparatus 20. It can be understood that the embodiment of the present application does not make any limitation on the specific form of the image generation system 100, such as the image generation system 100 being a camera, a video camera, a mobile phone, a monitoring camera, a scanner or other image capturing device, etc. Of course, in other possible implementation manners, the image generation system 100 can include only the image generation apparatus 10 without the image acquisition apparatus 20, for example, the image acquisition apparatus 20 can be detachably connected with the image generation apparatus 10, and after the image acquisition apparatus 20 is mounted to the image generation apparatus 10, the image generation apparatus 10 can perform data transmission with the image acquisition apparatus 20.

[0070] The application scenarios of the image generation system 100 are exemplarily described below.

[0071] The image generation system 100 described above can be applied to an electronic device. The electronic device includes at least one of a movable platform and a handheld device.

[0072] The movable platform includes at least one of an aircraft, a vehicle, a ship and a movable robot, but is not limited thereto.

[0073] The aircraft can include a rotorcraft, a fixed-wing aircraft or a hybrid fixed-wing-rotor aircraft, etc. The rotorcraft can be a single-rotor aircraft, a double-rotor aircraft, a triple-rotor aircraft, a quad-rotor aircraft, a hexa-rotor aircraft, an octo-rotor aircraft, a deca-rotor aircraft or a dodeca-rotor aircraft, etc. The aircraft can include, but is not limited to, a manned aircraft, a logistics aircraft, a aerial photography aircraft and an agricultural plant protection aircraft, an industry rescue aircraft. The above are only exemplarily described, and the embodiment of the present application does not make any specific limitation on the type of the aircraft. The aircraft includes an unmanned aerial vehicle (UAV) and a manned aircraft, etc. The aircraft carrying the image generation system 100 can be used for aerial photography, aerial reconnaissance, geographic mapping, environmental monitoring and security patrol, etc. Taking the movable platform including the unmanned aerial vehicle as an example, referring to FIG. 2, a schematic diagram of the unmanned aerial vehicle 200 carrying the image generation system 100 is shown.

[0074] The vehicle includes an unmanned vehicle and a manned vehicle, etc. The vehicle carrying the image generation system 100 can be used for entertainment shooting, automatic driving, environmental perception, security patrol and rescue, navigation and mapping, etc.

[0075] The ship includes an unmanned ship and a manned ship, etc. The ship carrying the image generation system 100 can be used for marine mapping, environmental monitoring, search and rescue and entertainment, etc.

[0076] The movable robot-mounted image generation system 100 can be used for daily shooting, industrial automation, warehouse logistics, medical care, and security patrol, etc.

[0077] Exemplarily, the electronic device includes a handheld device, which includes but is not limited to a handheld gimbal and a gimbal camera. In some embodiments, the handheld device can be moved by a person or an object in an external environment, and in other embodiments, the handheld device itself has a power device to drive its autonomous movement. The type and power source of the handheld device are not specifically limited in the embodiments of the present application.

[0078] In the handheld gimbal, the handheld gimbal is detachably connected with the image generation system 100, which is convenient for carrying and operation, has light weight and high stability, and ensures the smooth picture during mobile shooting.

[0079] The gimbal camera is used for professional photography, video production, and live broadcast, etc. In the gimbal camera, the gimbal and the image generation system 100 are integrated, which provides a highly integrated solution, is convenient for carrying and operation, and is suitable for various shooting environments.

[0080] The following exemplary describes each device in the image generation system 100:

[0081] Referring to FIG. 3, the image acquisition device 20 includes an image sensor 21 and a lens assembly 22.

[0082] The lens assembly 22 functions to converge and focus the light reflected or emitted from the object onto the surface of the image sensor 21, forming a clear image.

[0083] The image sensor 21 is a device that converts optical signals into electrical signals, and is a core element in imaging devices. It captures light through a photosensitive element and converts it into electrical signals, which represent the brightness and color information of the image. The photosensitive element includes but is not limited to a CMOS (Complementary Metal-Oxide-Semiconductor) sensor or a CCD (Charge-Coupled Device) sensor. When light passes through the lens assembly 22 to the image sensor 21, the pixel units on the sensor capture photons and generate corresponding electric charges, which are converted into voltage signals and output by the image sensor 21 as raw image data, containing the brightness and color information of each pixel point.

[0084] Referring to FIG. 3, the image generation device 10 includes a processor, such as an image signal processor 11, which is a dedicated processor responsible for processing raw data from the image sensor 21. It performs various processing and optimization on the image to achieve the desired visual effect, such as image denoising, white balance, color correction, edge enhancement, etc.

[0085] The processor can also include other types of processors, such as a Central Processing Unit (CPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic component, a discrete hardware component, or the like. The present embodiment does not make any limitation on this.

[0086] For example, referring to FIGS. 1 and 3, the image generation system 100 can also be connected with a filter 30. The filter 30 is used to reduce the amount of light entering the image sensor 21, thereby reducing the exposure amount; or the filter 30 is used to reduce the waveband of light entering the image sensor 21.

[0087] The filter 30 includes an electronic filter or a mechanical filter.

[0088] The electronic filter adjusts its optical properties through electrical signals to control the intensity of the transmitted light or the waveband of the light. For example, the electronic filter uses liquid crystal, variable optical material or electrochromic material technology, etc. to change the transmittance of the filter by adjusting the applied voltage or current. This change can be continuous or stepwise. The electronic filter can achieve precise and fast light intensity control, and is suitable for dynamic changing scenes. The transmittance of the electronic filter can be adjusted according to real-time needs, and is suitable for scenes requiring different exposure.

[0089] The mechanical filter changes the properties of the light path through physical structure, usually using neutral density filter, polarizer or other physical filter material. The mechanical filter moves the filter through mechanical devices (such as rotating devices, sliding devices, etc.) to change the path or angle of the light passing through the filter, thereby adjusting the light intensity or the waveband of the light.

[0090] The following illustrates the mounting mode of the filter 30:

[0091] In one possible implementation, the filter 30 is detachably connected with the image acquisition device 20, and the filter 30 can be flexibly used in different shooting scenes. For example, the detachable connection mode includes but is not limited to buckle connection, threaded connection, magnetic connection or slot connection, etc.

[0092] For example, the image acquisition device 20 is provided with contact points for connecting the filter 30, which can be physical connection points, ensuring that the filter 30 is firmly docked with the image acquisition device 20 when installed. In a shooting scene where the filter 30 needs to be used, the filter 30 can be connected to the image acquisition device 20 through the contact points, so that the filter 30 coincides with the optical path of the lens assembly 22, thereby filtering the light entering the image acquisition device 20; in a shooting scene where the filter 30 does not need to be used, the user can detach the filter 30 from the image acquisition device 20, so that the optical path of the lens assembly 22 is not affected by the filter 30.

[0093] In another possible implementation, the filter 30 is integrated with the image acquisition device 20, such as the filter 30 and the image acquisition device 20 being connected through a mechanical structure; in a shooting scene where the filter 30 needs to be used, the filter 30 can be controlled to move from a first position to a second position through the mechanical structure; in a shooting scene where the filter 30 does not need to be used, the filter 30 can be controlled to move from the second position to the first position through the mechanical structure. Wherein, the filter 30 in the first position is offset from the optical path of the lens assembly 22 of the image acquisition device 20, and the filter 30 in the second position coincides with the optical path of the lens assembly 22.

[0094] The mechanical structure includes but is not limited to a rotating mechanism, a translating mechanism, an electric / motor-driven mechanism, etc. For example, the filter 30 can be rotated within a certain angle through a rotating mechanism, so as to be switched into or moved out of the optical path of the lens assembly 22, such as the rotating mechanism being a rotating disc, and the filter 30 being installed thereon. For another example, the filter 30 can be translated along a certain direction through a translating mechanism, thereby changing its position in the optical path of the lens assembly 22. For yet another example, an electric motor or a mechanical force can be utilized to drive the filter 30 to change position, realizing automatic control.

[0095] For example, the image acquisition device 20 includes an image sensor 21, and the filter 30 is closer to the object side than the image sensor 21. For example, the filter 30 is arranged in front of the image sensor 21.

[0096] With the lens assembly 22 as a reference, referring to FIG. 3, the filter 30 can be arranged in front of the lens assembly 22 and exposed to the external environment, which enables the filter 30 to directly adjust the light entering the lens.

[0097] Alternatively, the filter 30 can be arranged between the lens assembly 22 and the image sensor 21; that is, the light passes through the lens assembly 22, then enters the filter 30, and then reaches the image sensor 21.

[0098] Alternatively, the lens assembly 22 includes at least two lenses, and the filter 30 can be arranged between any two lenses. In this way, the filter 30 can further filter the light after the light is adjusted by the partial lens, and more accurate light control can be achieved.

[0099] The arrangement position of the filter 30 can be selected according to actual needs, and the embodiment is not limited in this regard.

[0100] In some embodiments, the processor in the image generation system 100 can output filter function opening indication information in response to determining that the filter 30 is in an available state, where the filter function opening indication information is used to indicate that the function of using the filter 30 to filter the light entering the lens assembly 22 of the image capture device 20 is turned on or to indicate that the user selects to turn on the function of using the filter 30 to filter the light entering the lens assembly 22 of the image capture device 20.

[0101] In one possible implementation, in the case where the filter 30 is detachably connected to the image capture device 20, the processor determines that the filter 30 is in an available state in response to detecting that the filter 30 is connected to the image capture device 20 through the contact point, that is, the filter 30 coincides with the optical path of the lens assembly 22, and then the filter function opening indication information can be output.

[0102] In another possible implementation, in the case where the filter 30 is integrated with the image capture device 20, the processor determines that the filter 30 is in an available state in response to detecting that the filter 30 moves from a first position to a second position, and then the filter function opening indication information can be output; where the filter 30 in the first position is offset from the optical path of the lens assembly 22 of the image capture device 20, and the filter 30 in the second position coincides with the optical path of the lens assembly 22.

[0103] The filter function opening indication information can be visual information, auditory information, or somatosensory information, but is not limited thereto.

[0104] For example, the image generation system 100 can include an output device, such as a visual output device (e.g., a display), an auditory output device (e.g., a speaker), or a somatosensory output device (e.g., a vibration motor), etc., so that the processor can output the filter function opening indication information through the output device.

[0105] For example, the image generation system 100 includes a communication module, and the processor can output the filter function opening indication information to an external device through the communication module, so as to remind the user with the help of the external device.

[0106] In some embodiments, the image generation method as shown in FIG. 4 is provided, which can be executed by the processor in the image generation system as described above, and the method comprises:

[0107] In S401, the filter is controlled to set the filtering degree to a first filtering degree in a first exposure period corresponding to the first pixel unit in the single-frame image acquisition by the image acquisition device, and first pixel data of the first pixel unit acquired by the image acquisition device in the first exposure period is obtained; wherein the filter is used to filter the light entering the lens assembly of the image acquisition device.

[0108] In S402, the filter is controlled to adjust the filtering degree from the first filtering degree to a second filtering degree in a second exposure period corresponding to the second pixel unit in the single-frame image acquisition by the image acquisition device, and second pixel data of the second pixel unit acquired by the image acquisition device in the second exposure period is obtained; wherein the starting time of the second exposure period is after the first exposure period, the first filtering degree is different from the second filtering degree, and the first pixel unit is different from the second pixel unit.

[0109] In S403, a single-frame image including the first pixel data and the second pixel data is output.

[0110] In the embodiment, the filtering degree of the filter is dynamically adjusted in the first exposure period corresponding to the first pixel unit and the second exposure period corresponding to the second pixel unit in the single-frame image, the filter is set to different first filtering degree and second filtering degree, the filtering degree of the filter in different exposure periods of the same frame image is controlled to be different filtering degree, so that the filtering degree of the filter is different when collecting pixel data of different pixel units in the same frame image, and different filtering degrees corresponding to different pixel units in the same frame image are achieved to adapt to the scene with light and dark contrast and improve the shooting effect.

[0111] For example, the first pixel unit and the second pixel unit are two adjacent pixel units.

[0112] For example, the pixel unit includes at least one of the following: a pixel point, a pixel row, or a pixel area. In one case, the first pixel unit and the second pixel unit can be the same kind of pixel point, pixel row, or pixel area, for example, the first pixel unit and the second pixel unit can be different pixel points, different pixel rows, or different pixel areas. In another case, the first pixel unit can be one kind of pixel point, pixel row, or pixel area, and the second pixel unit is another kind of pixel point, pixel row, or pixel area, for example, the first pixel unit is a pixel row, and the second pixel unit is a pixel area different from the first pixel unit.

[0113] For example, the first pixel unit and the second pixel unit correspond to different side regions in the same frame image.

[0114] In a possible implementation, the first pixel unit corresponds to an upper part region in the same frame image, and the second pixel unit corresponds to a lower part region in the same frame image.

[0115] In another possible implementation, the first pixel unit corresponds to a left side region in the same frame image, and the second pixel unit corresponds to a right side region in the same frame image.

[0116] For example, the first pixel unit and the second pixel unit correspond to different pixel rows in the same frame image. For example, the first pixel unit and the second pixel unit correspond to two adjacent different pixel rows.

[0117] Referring to FIG. 5, the total exposure period of a single frame image, that is, the exposure start time of different pixel rows is different. For example, the image sensor in the image acquisition device cooperates with the rolling shutter to perform exposure control and pixel value reading in units of pixel rows in the total exposure period of a single frame image, and each pixel row starts exposure at different time points.

[0118] In a possible implementation, the first pixel unit and the second pixel unit include adjacent different pixel rows in the same frame image; for example, the first pixel unit and the second pixel unit can be a bright-dark boundary position in the same frame image.

[0119] In another possible implementation, the first pixel unit corresponds to a pixel row in an upper part region in the same frame image, and the second pixel unit corresponds to a pixel row in a lower part region in the same frame image.

[0120] For example, the light filtering degree is related to the amount of light entering the lens assembly of the image acquisition device. The amount of light refers to the total intensity of the light entering the image sensor after passing through the light filter, and the amount of light directly affects the brightness and exposure degree of the image. The greater the light filtering degree of the light filter, the less the intensity of the light that passes through, and the less the amount of light entering the lens assembly of the image acquisition device; the smaller the light filtering degree of the light filter, the more light that is allowed to pass through, and the more the amount of light entering the lens assembly of the image acquisition device. By adjusting the light filtering degree, the amount of light entering the lens assembly can be dynamically adjusted according to the ambient light conditions, so as to optimize the exposure effect and ensure that images with appropriate brightness can be captured under various lighting conditions.

[0121] For example, the filtering degree is related to the wavelength band of the light entering the lens assembly of the image acquisition device. The wavelength band of the light refers to the spectral range of the light entering the lens assembly of the image acquisition device after passing through the filter; the light contains multiple wavelength bands, including visible light, infrared light, ultraviolet light, etc., and different wavelength bands of light have different effects on the color and details of the image. The greater the filtering degree of the filter, the more wavelength bands of light are limited to enter the lens assembly, and the fewer the wavelength bands of the light entering the lens assembly of the image acquisition device; the smaller the filtering degree of the filter, the more wavelength bands of light can pass through the filter to enter the lens assembly, and the more the wavelength bands of the light entering the lens assembly of the image acquisition device. By adjusting the filtering degree of the filter, the spectral composition of the light can be controlled, thereby achieving a specific color effect or reducing the influence of a specific wavelength band of light, improving the color restoration and contrast of the image, or by adjusting the spectral range entering the lens assembly of the image acquisition device, the amount of light entering the lens assembly of the image acquisition device can also be indirectly affected, and the fewer the wavelength bands of the light entering the lens assembly of the image acquisition device, the less the amount of light entering the lens assembly of the image acquisition device.

[0122] The first filtering degree is different from the second filtering degree, and at least one of the following is possible: the amount of light entering the lens assembly of the image acquisition device is different, and the number of wavelength bands of the light entering the lens assembly of the image acquisition device is different.

[0123] In some embodiments, the image generation method provided by the embodiments of the present application can be applied to the following shooting scene: the related parameters of the region of the current shooting scene corresponding to the first pixel unit are different from the related parameters of the region of the current shooting scene corresponding to the second pixel unit.

[0124] The related parameters include brightness or object category. The brightness parameter refers to the light intensity or brightness level of different regions in the current shooting scene. Different regions in the current shooting scene can exhibit different brightness due to different light sources, reflective properties of objects, or changes in ambient light. The object category parameter refers to the object type in different regions of the current shooting scene, including but not limited to natural scenery, buildings, people, vehicles, etc. Objects belonging to different object categories have different reflection and absorption properties of light, resulting in differences in filtering requirements.

[0125] The scheme of the embodiments of the present application can adapt to shooting scenes where the brightness of different regions in the shooting scene is inconsistent. Different filtering degrees can be applied to different shooting regions to achieve a shooting effect where the exposure effect for different brightness regions is appropriate, thereby enabling the shooting of higher dynamic range of picture content. In addition, the embodiments of the present application can also be applied to shooting scenes where different object categories exist in the scene. Different filtering degrees can be applied to different object categories to achieve a shooting effect where the exposure effect for different object categories is appropriate, thereby improving the shooting effect.

[0126] As shown in FIG. 6, the first pixel data corresponding to the first pixel unit is collected for the first region of the current shooting scene, and the second pixel data corresponding to the second pixel unit is collected for the second region of the current shooting scene.

[0127] In a first possible case, the first region and the second region are high-contrast scenes with different brightness.

[0128] For example, the first region is a sky region, and the second region is a ground region. The sky region is usually brighter than the ground region. By using the image generation method of the embodiment of the present application, different light filtering degrees can be applied to different regions during shooting, so that the brighter sky region transmits less light, and the darker ground region transmits more light, thereby achieving the effect of adaptively adjusting the light filtering degree based on the actual situation of different shooting regions in the same frame of image, to adapt to the shooting scene with brightness difference, and achieve better exposure and shooting effect.

[0129] In a second possible case, the first region and the second region respectively contain different object categories, and objects belonging to different object categories have different reflection and absorption characteristics of light.

[0130] For example, when shooting a portrait, the first region is a background region, and the second region is a portrait region. It is usually desired that the exposure of the person is softer, and the background can be appropriately dimmed. By adjusting the light filtering degree of the light filter in the portrait region and the background region, the subject of the person can be ensured to be clear and bright, and the background will not distract attention due to overexposure.

[0131] For another example, when shooting a natural landscape, the first region contains rocks, leaves, etc., and the second region contains water. The light reflection characteristics of different object categories such as water, leaves, and rocks are quite different. By dynamically adjusting the light filtering degree of the light filter corresponding to different regions, the performance of different object categories in the image can be ensured to be more real and natural.

[0132] For still another example, when shooting a building, the first region contains the upper half of the building (the building material is glass), and the second region contains the lower half of the building (the building material is stone). Different building materials (such as glass, metal, and stone) have different reflection effects on light. By adjusting the light filtering degree of different regions, each part of the building can be clearly presented in the same image, without losing details due to over-brightness or over-darkness in some regions.

[0133] For example, during the process of collecting the same frame of image, by precisely controlling the exposure time period of the image collecting device for collecting different pixel units in the same frame of image, and adjusting the filtering degree of the filter, the control manner makes each pixel unit be able to receive appropriate light quantity according to the illumination condition of the scene, avoids the situation of overexposure or underexposure, and can directly realize high dynamic range imaging (HDR) at the physical layer without post-processing. Through the HDR imaging at the physical layer, the exposure and filtering degree of different pixel units are precisely controlled, which ensures that the image can retain more details in highlight and shadow parts. For example, when shooting a high-contrast scene, the sky and the ground, the bright part and the dark part can be evenly exposed, thereby generating a high-quality image with rich details and real colors. Obviously, the HDR imaging at the physical layer not only improves the image quality, but also reduces the complexity and time cost of post-processing, and more efficiently generates high-quality images.

[0134] In some embodiments, in the actual shooting scene, the illumination condition or the position of the object belonging to different object categories may change over time, weather, location or other factors, resulting in changes in the brightness distribution or object category distribution of each region in the current shooting scene when shooting the same frame of image. For example, when shooting a sunset scene, the distribution of different regions in the shooting scene will also change accordingly as the sun gradually sets; for example, when shooting city scenery, the shadow of high-rise buildings will change as the sun moves, and the distribution of dark and bright regions will also change accordingly. Or there is a need to quickly switch the shooting scene in some cases, and the distribution of different regions in different shooting scenes also differs. In this case, adjusting the first filtering degree to the second filtering degree at the fixed pixel unit position obviously cannot adapt to such changes, and it is necessary to dynamically adjust the pixel position that needs to adjust the first filtering degree to the second filtering degree, or dynamically adjust the position of the filtering degree.

[0135] Based on this, the embodiments of the present application realize that the position of at least one of the first exposure time period and the second exposure time period in the total exposure time period of the single frame of image is changeable. The position of the first exposure time period in the total exposure time period of the single frame of image reflects the time distribution of the first exposure time period in the total exposure time period of the single frame of image. The position of the second exposure time period in the total exposure time period of the single frame of image reflects the time distribution of the second exposure time period in the total exposure time period of the single frame of image.

[0136] The changing process can be automatic, semi-automatic or manually performed by the user, thereby improving the shooting effect. Automatic changing refers to that the processor in the image generation system automatically adjusts the distribution of at least one of the first exposure period and the second exposure period in the total exposure period of the single frame image completely autonomously. Semi-automatic changing is a mechanism of adding user intervention on the basis of automatic adjustment, and the user can fine-tune the adjustment result of the processor according to his own needs. User manual changing is that the user manually adjusts the position distribution of at least one of the first exposure period and the second exposure period in the total exposure period of the single frame image based on personal experience.

[0137] Correspondingly, the positions of the first pixel units affected by the first exposure period and the second pixel units affected by the second exposure period in the single frame image are also changed according to the position distribution of the first exposure period and / or the second exposure period in the total exposure period of the single frame image. The position of the first pixel unit in the single frame image reflects the spatial distribution of the first pixel unit in the single frame image, and the position of the second pixel unit in the single frame image reflects the spatial distribution of the second pixel unit in the single frame image.

[0138] For example, in the first shooting scene, the starting position of the second exposure period is located at the 1 / 2 position in the total exposure period of the same frame image, and correspondingly, the filtering degree of the upper 1 / 2 area of the image is the first filtering degree, and the filtering degree of the lower 1 / 2 area of the image is the second filtering degree. After switching from the first shooting scene to the second shooting scene, or after the shooting content / illumination condition of the first shooting scene changes, the starting position of the second exposure period is located at the 1 / 3 position in the total exposure period of the same frame image, and correspondingly, the filtering degree of the upper 1 / 3 area of the image is the first filtering degree, and the filtering degree of the lower 2 / 3 area of the image is the second filtering degree. The dividing position of the filtering degree changes with the change of the shooting scene.

[0139] The embodiment adjusts the dividing position of the filtering degree automatically to cope with the change of the shooting scene, simplifies the operation process, can adjust according to the real-time shooting requirements, ensures that better exposure effect can be obtained under different illumination conditions or object categories, and can be applied to more diversified shooting scenes, and is not limited by the fixed dividing position of the filtering degree.

[0140] Exemplarily, considering that the filter (such as the neutral density filter described above) in the prior art is generally set to a fixed filter degree, the user needs to manually replace the filter according to the filter requirement of the shooting scene, which is tedious and inefficient; and in the scene of remotely controlling shooting, the operation of manually replacing the filter needs to spend a lot of time cost. In the image generation method provided in the present application, the filter can be adjusted to the first filter degree or the second filter degree, that is, the filter degree of the filter is variable and can be switched between the first filter degree and the second filter degree, so that the user does not need to manually replace the filter, and the user operation can be reduced to a certain extent; and it can be applied to the scene of remotely controlling shooting, effectively enhancing the practicability and flexibility of remote shooting, so that the user can more efficiently perform the shooting operation in the remote environment.

[0141] In a possible implementation, the position of at least one of the first exposure period and the second exposure period in the total exposure period of the single-frame image can be changed with the change of the current shooting scene, and the change process can be automatic or semi-automatic, thereby improving the shooting effect and reducing the user operation to a certain extent.

[0142] Exemplarily, the position of at least one of the first exposure period and the second exposure period in the total exposure period of the single-frame image can be changed with the change of the relevant parameter of the current shooting scene. The relevant parameter includes brightness or object category, but is not limited thereto, and can also be dynamically adjusted according to other relevant parameters, such as light source position, shooting angle, scene complexity, and the like. In the embodiment, the processor dynamically adjusts the time distribution of the first exposure period and the second exposure period in the total exposure period of the single-frame image according to the change of the brightness or object category of the current shooting scene, which can more flexibly cope with complex shooting scenes and ensure that better shooting effects can be obtained under different lighting conditions or object categories. This real-time adjustment reduces the need for manual intervention, is more intelligent or automatic, improves the shooting efficiency, and is suitable for rapidly changing shooting scenes.

[0143] For example, in a sunrise scene, the sun gradually rises from the horizon, and the brightness of the scene gradually increases. At the beginning of the shooting, the area below the horizon is dark (i.e., the dark area), and the brightness of the sky is high (i.e., the bright area). Assuming that the total exposure period of the same frame image is 30 ms, at the beginning of the shooting, the first exposure period is set to 1-14 ms in 30 ms, which is used to capture the bright area in the current shooting scene. The second exposure period is set to 15-30 ms in 30 ms, which is used to capture the dark area in the current shooting scene, wherein the filtering degree of the filter corresponding to the first exposure period is the first filtering degree, and the filtering degree of the filter corresponding to the second exposure period is switched from the first filtering degree to the second filtering degree, and the first filtering degree is greater than the second filtering degree.

[0144] As time goes on, the sun gradually rises, the brightness near the horizon increases rapidly, the light distribution of the whole scene changes, the bright area gradually expands, and the dark area shrinks. Therefore, the first exposure period can be adjusted to 1-20 ms in 30 ms, which increases the range of the bright area that needs to be dimmed, and the second exposure period is adjusted to 21-30 ms in 30 ms, which reduces the range of the dark area that does not need to be dimmed. By dynamically adjusting the distribution position of the second exposure period or the boundary position of different filtering degrees, the shooting scene with different brightness distributions can be adapted, and the shooting effect can be improved.

[0145] For another example, when shooting a multi-object scene, different object categories may need different exposure processing. For example, in a street scene, vehicles, pedestrians, and buildings may exist in the picture at the same time, and the reflection light intensity of these object categories is different. Assuming that the first area in the current shooting scene contains vehicles and buildings, and the second area contains pedestrians, the light of vehicles and buildings is strong, and the reflection light of pedestrians is weak.

[0146] Assuming that the total exposure period of the same frame image is 30 ms, at the beginning of the shooting, the first exposure period is set to 1-14 ms in 30 ms, which is used to capture the first area in the current shooting scene. The second exposure period is set to 15-30 ms in 30 ms, which is used to capture the second area in the current shooting scene, wherein the filtering degree of the filter corresponding to the first exposure period is the first filtering degree, and the filtering degree of the filter corresponding to the second exposure period is switched from the first filtering degree to the second filtering degree, and the first filtering degree is greater than the second filtering degree.

[0147] If the number of vehicles in the current shooting scene decreases, the number of pedestrians increases, or the focus of shooting shifts from vehicles to pedestrians, i.e., the first area decreases and the second area expands, the distribution position of the second exposure period or the demarcation position of different light filtering degrees can be adjusted when the change of the object categories is detected. For example, the first exposure period is adjusted to 1-8 ms, the range of the area requiring light reduction is reduced, the second exposure period is adjusted to 9-30 ms, the range of the area not requiring light reduction is increased, and the details of pedestrians in the picture are highlighted. By dynamically adjusting the distribution position of the second exposure period or the demarcation position of different light filtering degrees, the shooting scene of different object category distributions can be adapted, and the shooting effect can be improved.

[0148] As can be seen from the above examples, the change of the brightness or the object category of the current shooting scene mainly causes the difference in light intensity in different areas in the shooting scene, thereby forming different bright-dark demarcation positions, which reflect the distribution of the bright area and the dark area in the shooting scene.

[0149] The brightness in the shooting scene directly affects the distribution of the bright area and the dark area. For example, when shooting a landscape under the setting sun, as the sun gradually sets, the bright area will gradually decrease, and the dark area will expand. At this time, the bright-dark demarcation line will move over time.

[0150] Different object categories can have different light reflection or light absorption characteristics, which also affect the brightness distribution in the shooting scene. When the object categories in the shooting scene change, these objects can cause new bright-dark demarcation lines to appear or cause the original demarcation lines to shift. For example, when shooting a city landscape, a glass curtain wall building can reflect sunlight to form a highlight area, while an adjacent brick wall building forms a dark area.

[0151] Therefore, the processor in the image generation system can analyze the brightness change or the object category distribution of the current shooting scene, determine the change of the bright-dark demarcation position of the current shooting scene, and then change the position of the first exposure period and / or the second exposure period in the total exposure period of the single frame image based on the change of the bright-dark demarcation position of the current shooting scene. The embodiment realizes intelligent perception of the distribution change of the bright and dark areas in the current shooting scene, and adjusts the position distribution of the first exposure period and / or the second exposure period in the total exposure period of the single frame image in real time to adapt to the scene change and ensure the image quality.

[0152] Exemplarily, the position of the second exposure period in the total exposure period of a single frame image is changeable, and the position of the second exposure period in the total exposure period of different frame images is different to adapt to the change of the current shooting scene. Exemplarily, when the light and dark distribution of the current shooting scene changes, the position of the second exposure period in the total exposure period of different frame images is different. For example, at sunset, as the light gradually weakens, the light and dark demarcation position in the shooting scene may move downward, and at this time, the position of the second exposure period in the total exposure period may move downward frame by frame to ensure that different frame images can accurately capture scene details under specific lighting conditions. By dynamically adjusting the position of the second exposure period in the total exposure period, it can be ensured that each frame image can obtain a better exposure effect under changing lighting conditions. This flexible adjustment mechanism can adapt to different changes in shooting scenes and reduce the problem of inaccurate exposure caused by inconsistent lighting conditions, thereby improving the overall quality and detail performance of the image.

[0153] In another possible implementation, considering that in some photography scenes, the user wants to accurately control the details of each exposure period in order to capture excellent pictures under complex lighting conditions, at least one of the first exposure period and the second exposure period can be set by the user. Specifically, the user can manually adjust the distribution of at least one of the first exposure period and the second exposure period in the total exposure period according to his own experience and understanding of the current shooting scene to achieve a specific shooting effect. This embodiment allows the user to manually set the distribution position of the exposure period, which can achieve a more personalized shooting experience and meet the needs of professional photography and creative photography.

[0154] Taking an image generation system as an example, a camera, the user can enter a manual adjustment mode through the setting interface of the camera. The setting interface will display the current exposure period distribution, including the position of at least one of the first exposure period and the second exposure period in the total exposure period. Based on personal experience, the user manually adjusts the position and time distribution of the exposure period. For example, in a high-contrast scene, the user may shorten the first exposure period to reduce the range of areas that need to be dodged, avoid overexposure of bright areas, and extend the second exposure period to increase the range of areas that do not need to be dodged to ensure that more details of dark areas are preserved. After adjustment, the user can view the adjusted effect through the real-time preview function. When the user is satisfied with the adjusted effect, the user can save the setting and apply it to the subsequent shooting process.

[0155] For example, in some remote control shooting scenarios, the user can also set the position of at least one of the first exposure period and the second exposure period in the total exposure period of a single frame image through the remote control device, so as to meet the personalized shooting needs of the user.

[0156] For example, referring to FIG. 7, in a UAV aerial photography scenario, the user can control the flight and shooting of the UAV 200 through the remote control device 400, and the remote control device 400 can also include a display. The user can view the image shot by the UAV 200 in real time through the display on the remote control device 400, and when the current shooting scene changes, the user can adjust the position of at least one of the first exposure period and the second exposure period in the total exposure period of a single frame image through the remote control device 400.

[0157] For example, in order to improve the shooting efficiency of the user, a plurality of filter options can be pre-set, each of the plurality of filter options corresponding to the position of the first exposure period and / or the second exposure period in the total exposure period of a single frame image. In other words, the plurality of filter options are pre-set based on different brightness or different object categories in the shooting scene. For example, different filter options can be different gears. The first gear is 1 / 2 gear, the upper 1 / 2 area of the image corresponds to a filter degree, and the lower 1 / 2 area of the image corresponds to another filter degree. The second gear is 1 / 3 gear, the upper 1 / 3 area of the image corresponds to a filter degree, and the lower 2 / 3 area of the image corresponds to another filter degree. The third gear is 1 / 4 gear, the upper 1 / 4 area of the image corresponds to a filter degree, and the lower 3 / 4 area of the image corresponds to another filter degree. The user can select one from the plurality of gear options.

[0158] When the user encounters different light conditions or object types in the actual shooting process, the user can select one of the pre-set plurality of filter options according to the actual needs. This not only speeds up the operation, but also improves the shooting efficiency, so that different shooting scenes can be quickly adapted to. In this way, the user can more flexibly cope with various shooting needs, so as to ensure that high-quality images can be shot in different shooting environments.

[0159] In some embodiments, in the process of generating each frame image, the processor can automatically determine the position of the first exposure period and / or the second exposure period in the total exposure period of a single frame image based on the relevant parameters of the current shooting scene. The relevant parameters include brightness or object category, but are not limited thereto. Other relevant parameters such as light source position, shooting angle, scene complexity, etc. can also be dynamically adjusted. The automatic determination process can effectively reduce user operation and improve shooting efficiency.

[0160] For example, the brightness or object category of the current shooting scene determines the distribution of light intensity in the scene, thereby forming a light-dark boundary position, which reflects the distribution of bright and dark regions in the shooting scene. Therefore, the processor can automatically determine the position of at least one of the first exposure period and the second exposure period in the total exposure period of the single frame image based on the light-dark boundary position of the current shooting scene; and the light-dark boundary position of the current shooting scene can be determined by analyzing the brightness or object category distribution of different regions of the current shooting scene. According to the embodiment, the position of the two exposure periods in the total exposure period of the single frame image is automatically adjusted according to the light-dark boundary position of the shooting scene, which can more intelligently, efficiently or accurately capture key details in the shooting scene, especially in scenes with frequent light changes or complex object categories, so that the final image can more accurately reflect important features in the shooting scene, improving shooting efficiency and shooting effect.

[0161] In a possible implementation, the distribution boundary of the bright region and the dark region in the shooting scene can be determined by light measurement on the current shooting scene, for example, by light measurement on the current shooting scene to determine the image brightness of each region in the scene to determine the distribution boundary of the bright region and the dark region. For example, the light-dark boundary position condition can be set in advance, which includes that the brightness difference between different regions at both ends of the light-dark boundary position is greater than a preset brightness difference value, which is used to determine that the brightness difference between different regions is significant enough to represent the boundary between the bright region and the dark region in the image.

[0162] In the embodiment, the light-dark distribution boundary is determined by light measurement on the current shooting scene, which can accurately determine the light-dark boundary of the current shooting scene, thereby quickly adjusting the position of at least one of the first exposure period and the second exposure period in the total exposure period of the single frame image, improving the shooting effect.

[0163] In this embodiment, considering that the light-dark difference of different shooting scenes is different, different light-dark boundary position conditions can be set for different shooting scenes, that is, the preset brightness difference values corresponding to different shooting scenes are different. For example, for high-contrast scenes (such as shadows under strong light), the preset brightness difference value may need to be set higher, and for scenes with soft light, the preset brightness difference value may be lower, thereby adapting to different shooting scenes.

[0164] In a possible implementation, in the case that the current frame image and the historical image frame of the current frame image both correspond to the current shooting scene, the light-dark boundary position of the historical image frame usually has high consistency with the boundary position of the current scene, and the distribution boundary of the bright region and the dark region in the current shooting scene can be determined by the light-dark boundary position of the historical image frame.

[0165] Therefore, the processor can automatically determine the position of at least one of the first exposure period and the second exposure period in the total exposure period of the single-frame image based on the bright-dark division position of the historical image frame of the single-frame image captured by the image capturing device. In this embodiment, by referring to the bright-dark division position of the historical image frame, the processor can early, quickly and accurately infer the bright-dark division of the current shooting scene, so as to quickly adjust the position of at least one of the first exposure period and the second exposure period in the total exposure period of the single-frame image, and improve the shooting effect.

[0166] Here, the determination process of the bright-dark division position in the historical image frame is exemplarily described:

[0167] In a possible implementation, the processor can determine the bright-dark division position in the historical image frame based on the luminance difference between different pixel units in the historical image frame.

[0168] For example, the bright-dark division position condition can be preset, and the bright-dark division position condition includes that the luminance difference between different pixel units at both ends of the bright-dark division position is greater than a preset luminance difference value, and the preset luminance difference value is used to determine that the luminance difference between different pixel units is significant enough to represent the division between the bright area and the dark area in the image. Then the processor can traverse all pixel units in the historical image frame, calculate the luminance difference between adjacent pixel units, which can be checking the luminance difference of each pair of adjacent pixel units in the image one by one in row or column, if the luminance difference of a pair of adjacent pixel units is greater than the preset luminance difference value, it is determined that the pair of adjacent pixel units is located at the bright-dark division position, and the processor can take the boundary between the pair of adjacent pixel units as the bright-dark division position of the historical image frame, to accurately identify the bright-dark division position in the historical image frame. That is, the luminance difference between different pixel units at both ends of the bright-dark division position in the historical image frame is greater than the preset luminance difference value.

[0169] Among them, considering that the bright-dark difference of different shooting scenes is different, different bright-dark division position conditions can be set for different shooting scenes, that is, the preset luminance difference values corresponding to different shooting scenes are different. For example, for high-contrast scenes (such as shadows under strong light), the preset luminance difference value may need to be set higher, and for scenes with soft light, the preset luminance difference value may be lower, so as to adapt to different shooting scenes.

[0170] In another possible implementation, the processor can generate a brightness curve based on the brightness of each pixel unit in the historical image frame, which can be to expand the image by row or by column, record the brightness value of each pixel unit, and thus form a continuous curve, for example, in a row of pixel points, the x-axis (horizontal axis) represents the position of the pixel point in the image, and the y-axis (vertical axis) represents the brightness value of the corresponding pixel point; the shape of the brightness curve can reveal the light and dark distribution in the image, especially in the part of the curve where there is a sharp change, which usually means the existence of light and dark boundary. Therefore, the processor determines the light and dark boundary position in the historical image frame based on the relationship between the peak value and the valley value in the brightness curve, the peak value corresponds to the area with higher brightness, and the valley value corresponds to the area with lower brightness; for example, if at a certain point, the brightness curve rapidly decreases from an obvious peak value to a valley value, this change point can be considered as the light and dark boundary position, which realizes accurate identification of the light and dark boundary position in the historical image frame.

[0171] In the image shooting scene, the historical image frame includes a viewfinder image or a preview image. The viewfinder image or the preview image is an image observed by the user through the image acquisition device before formal shooting, which is used to determine the shooting angle, composition and other shooting parameters. Since the viewfinder image or the preview image directly reflects the lighting conditions, object layout and light and dark distribution of the current shooting scene, it can be used as an important reference to determine the light and dark boundary position of the current shooting scene.

[0172] In the video shooting scene, the use of the historical image frame can be divided into two cases, depending on whether the current single frame image is the first frame image in the video. In response to the single frame image being the first frame, the historical image frame includes a viewfinder image or a preview image, which is usually an image seen by the user through the viewfinder or the display before starting to record the video. Since the viewfinder image or the preview image is highly related to the current shooting scene and can reflect the lighting changes and object layout in the current shooting scene, it can provide a reference for the first frame image. The processor can use the light and dark boundary position in the viewfinder image or the preview image to adjust the boundary position of different filter degrees of the first frame image to ensure good lighting effect when the video starts to be recorded.

[0173] Alternatively, in response to the single frame image being a non-first frame, the historical image frame includes an image frame captured by the image capturing device before the single frame image is captured. That is, one or more video frames before the current single frame image can be regarded as historical image frames, which can provide illumination information and scene change data in a continuous video stream. The processor can analyze the light-dark boundary position in the historical image frames, and automatically adjust the position of the first exposure period and / or the second exposure period in the overall exposure period of the current single frame image based on the analysis result, to adapt to the dynamic changes of the scene, so as to ensure the picture quality in the entire video recording process.

[0174] Further, if the filter is also in use during the process of capturing the historical image frame, it means that the light-dark condition of the historical image frame has been subjected to the filtering process, and cannot truly reflect the actual light-dark condition of the current shooting scene. Therefore, in order to improve the exposure period determination accuracy, the processor can first perform brightness recovery processing on the historical image frame, so as to obtain more accurate scene illumination information, and then determine the position of at least one of the first exposure period and the second exposure period in the total exposure period of the single frame image based on the light-dark boundary position of the historical image frame after the brightness recovery processing. In the embodiment, the brightness recovery processing can eliminate the influence of the filter on the brightness of the historical image frame, so that the image after the brightness recovery processing can more accurately reflect the actual illumination condition and the light-dark boundary position of the current shooting scene, which provides a reliable basis for the subsequent position adjustment of the exposure period, and ensures that the adjusted exposure setting is more in line with the actual needs of the scene.

[0175] In the brightness recovery processing, the processor can apply the inverse operation of the filter to compensate for the brightness attenuation caused by the filter. This means that the processor can calculate the amount of light attenuated in the historical image frame based on the known filter parameters (such as the filtering degree, wavelength range, etc.), and then add the light amount back to the historical image frame to restore the original brightness.

[0176] In the embodiment of the present application, since the filter is adjusted to different filtering degrees in different exposure periods of a frame image, different light attenuation will be caused to different pixel units of the frame image. In order to further accurately recover the brightness of the image, the processor can perform different brightness compensation on different pixel units of the historical image frame, for example, the processor can specifically perform greater brightness recovery on the pixel units with more serious light attenuation, and perform smaller recovery on the pixel units with less light attenuation, to accurately recover the brightness of the historical image frame.

[0177] In some embodiments, in actual shooting scenes, the lighting conditions or objects belonging to different object categories can change over time, weather, location or other factors; for example, when shooting a sunset landscape, as the sun gradually sets, the brightness of different areas in the shooting scene will also change accordingly, and the filtering requirements for different areas will also change accordingly; for example, when shooting cityscapes, as the sun moves, the brightness of the dark and bright areas of high-rise buildings will also change accordingly, and the filtering requirements for different areas will also change accordingly. Or there is a need to quickly switch between shooting scenes, and the brightness of different areas in different shooting scenes also differs, resulting in different filtering requirements. In this case, the fixed first and second filtering degrees obviously cannot adapt to such changes.

[0178] Based on this, the embodiments of the present application realize that at least one of the first and second filtering degrees is also changeable. This change process can be automatic, semi-automatic or manually performed by the user, thereby improving the shooting effect. Automatic change means that the processor in the image generation system completely autonomously adjusts at least one of the first and second filtering degrees. Semi-automatic change is a mechanism that adds user intervention on the basis of automatic adjustment, and the user can fine-tune the adjustment results of the processor according to his own needs. User manual change is that the user manually adjusts at least one of the first and second filtering degrees based on personal experience.

[0179] In the embodiments, the user does not need to manually replace the filter, and the function of changing the filtering degree of such a filter can adapt to changes in the current shooting scene, greatly simplifying user operations, especially in scenes where shooting conditions change frequently, such as the movement of high-rise building shadows when shooting cityscapes, which can significantly improve shooting efficiency; and can be applied to remote control shooting scenes, greatly enhancing the practicality and flexibility of remote shooting, such as the use of aircraft, the image generation system can control the change of the filtering degree of the filter, without the user controlling the aircraft to fly back from a distance, replacing the filter for the aircraft, and then controlling the aircraft to fly to the shooting position for shooting, which can significantly improve the shooting efficiency and shooting experience.

[0180] In one possible implementation, at least one of the first and second filtering degrees can change with changes in the current shooting scene, and the change process can be automatic or semi-automatic, thereby improving the shooting effect and reducing user operations to some extent.

[0181] For example, the processor can change at least one of the first filter degree and the second filter degree based on a change in a relevant parameter of a current shooting scene. The relevant parameter can include brightness or object category, but is not limited thereto. The first filter degree and the second filter degree can also be dynamically adjusted according to other relevant parameters, such as light source position, shooting angle, scene complexity, and the like. In this embodiment, the processor dynamically adjusts at least one of the first filter degree and the second filter degree based on a change in brightness or object category of the current shooting scene. This can more flexibly cope with complex shooting scenes and ensure that a better filtering effect is obtained under different lighting conditions or different object categories. This adjustment reduces the need for manual intervention and improves shooting efficiency, and is suitable for rapidly changing shooting scenes.

[0182] In one example, when shooting a landscape, different regions in the picture can have different brightness. For example, a mountain in the distance can be in a region with strong light (bright region), while a forest in the near distance can be in the shade (dark region). When starting to shoot, in order to capture the details of the entire picture, the first filter degree corresponding to the bright region can be set to be relatively high to ensure that the mountain is not overexposed, and the second filter degree corresponding to the dark region can be set to be relatively low to ensure that the forest is moderately exposed. If the lighting conditions change, for example, the clouds cover the sun and the light in the entire scene becomes uniform, the filter can be adjusted at this time so that the first filter degree and the second filter degree are closer to each other to adapt to the new lighting conditions and ensure that the picture is evenly exposed.

[0183] In another example, when shooting a portrait, the background region can be brighter than the foreground region. When starting to shoot, in order to ensure that the details of the person are clear, the first filter degree corresponding to the background region can be set to be moderate to reduce the intensity of the background light to ensure that the background is not overexposed, and the second filter degree of the foreground region can be set to be relatively low to ensure that the person is clearly visible. If the background light conditions change during shooting, for example, the background light suddenly becomes further stronger, the filter degree needs to be adjusted at this time. The first filter degree can be increased to reduce the amount of light in the background region, or the second filter degree can be reduced to adapt to the new lighting conditions and ensure that the exposure effect of the person and the background is good.

[0184] In another possible implementation, considering that in some photography scenes, the user wants to be able to accurately control the brightness details of different pixel units in the picture to capture excellent pictures under complex lighting conditions, at least one of the first filter degree and the second filter degree can be set by the user. Specifically, the user can set at least one of the first filter degree and the second filter degree according to his / her experience and understanding of the current shooting scene to achieve a specific shooting effect. This embodiment allows the user to manually set the filter degree, which can achieve a more personalized shooting experience and meet the needs of professional photography and creative photography.

[0185] Exemplarily, in some remote control shooting scenarios, the user can also set at least one of the first filter degree and the second filter degree through the remote control device to meet the personalized shooting needs of the user.

[0186] Exemplarily, in order to improve the shooting efficiency of the user, a plurality of filter options can be pre-set, each of the plurality of filter options corresponds to a first filter degree and / or a second filter degree, in other words, the plurality of filter options are pre-set based on different brightness or different object categories in the shooting scene. For example, different filter options can be different gears, the first gear is 1 / 2 gear, the upper 1 / 2 area of the image corresponds to a filter degree ND8, and the lower 1 / 2 area of the image corresponds to another filter degree ND2, the second gear is 1 / 3 gear, the upper 1 / 3 area of the image corresponds to a filter degree ND8, and the lower 2 / 3 area of the image corresponds to another filter degree ND4, the third gear is 1 / 4 gear, the upper 1 / 4 area of the image corresponds to a filter degree ND8, and the lower 3 / 4 area of the image corresponds to another filter degree ND2, wherein the larger the value of the ND suffix, the greater the filter degree, the smaller the amount of light or the total number of light wave bands entering the lens assembly of the image sensor, and the user can select one from the plurality of gear options.

[0187] When the user encounters different light conditions or object types in the actual shooting process, the user can select one of the pre-set plurality of filter options according to the actual filter needs, which not only speeds up the operation, but also improves the shooting efficiency, so as to quickly adapt to different shooting scenes. In this way, the user can more flexibly cope with various shooting needs, so as to ensure that high-quality images can be shot in different shooting environments.

[0188] In some embodiments, in the process of generating each frame of image, the processor can automatically determine at least one of the first filter degree and the second filter degree based on the related parameters of the current shooting scene. The related parameters include brightness or object category, but are not limited thereto, and can also be dynamically adjusted according to other related parameters, such as light source position, shooting angle, scene complexity, etc. The automatic determination process can effectively reduce user operation and improve shooting efficiency.

[0189] The first filter degree is related to a relevant parameter of a region corresponding to the first pixel unit in the current shooting scene, and the second filter degree is related to a relevant parameter of a region corresponding to the second pixel unit in the current shooting scene. The relevant parameter includes brightness or object category, but is not limited thereto. The processor can select different filter degrees for different pixel units based on the relevant parameters of different regions to adapt to the light distribution or object distribution in the scene. In this embodiment, the filter degree is associated with the relevant parameter of different regions in the current shooting scene, so that the filtering effect of different regions in the current shooting scene can be more accurately controlled. This fine control can ensure that different pixel units in the image obtain better exposure effect, avoid overexposure or underexposure, and thus improve the overall quality and detail performance of the image.

[0190] For example, the brightness or object category of the current shooting scene determines the distribution of light intensity in the scene, thereby forming a bright-dark boundary position, which reflects the distribution of bright regions and dark regions in the shooting scene. The region corresponding to the first pixel unit in the current shooting scene can be one of the bright regions and the dark regions in the current shooting scene, and the region corresponding to the second pixel unit in the current shooting scene can be the other of the bright regions and the dark regions in the current shooting scene.

[0191] In a possible implementation, the brightness of the region corresponding to the first pixel unit in the current shooting scene is greater than the brightness of the region corresponding to the second pixel unit in the current shooting scene, for example, the first pixel unit corresponds to a bright region in the current shooting scene, and the second pixel unit corresponds to a dark region in the current shooting scene, which means that the region corresponding to the first pixel unit receives more light and is prone to overexposure. To avoid this situation, the processor sets the first filter degree to be greater than the second filter degree, that is, during the first exposure period corresponding to the first pixel unit, the filter will reduce more light entering the image sensor, thereby preventing the high-brightness region in the image from losing details.

[0192] In another possible implementation, the brightness of the region corresponding to the first pixel unit in the current shooting scene is less than the brightness of the region corresponding to the second pixel unit in the current shooting scene, for example, the first pixel unit corresponds to a dark region in the current shooting scene, and the second pixel unit corresponds to a bright region in the current shooting scene, which means that the region corresponding to the first pixel unit receives less light and is prone to underexposure. To compensate for this deficiency, the processor sets the first filter degree to be less than the second filter degree, that is, during the first exposure period corresponding to the first pixel unit, the filter will allow more light to enter the image sensor, thereby improving the brightness of the region, so that the details of the dark region can be clearly presented.

[0193] The two implementation manners above respectively set the light filtering degree automatically to optimize the image quality under different brightness conditions. In the first implementation manner, the processor sets a larger light filtering degree for the area with higher brightness to prevent overexposure; in the second implementation manner, the processor sets a smaller light filtering degree for the area with lower brightness to prevent underexposure. The intelligent adjustment not only improves the dynamic range of the image, but also ensures the balance of the image details and quality under different shooting scenes.

[0194] The first pixel unit and the second pixel unit respectively correspond to different side regions of the current shooting scene based on the brightness distribution of the current shooting scene and the distribution of the objects belonging to different object categories.

[0195] For example, the first pixel unit corresponds to the upper region in the current shooting scene, and the second pixel unit corresponds to the lower region in the current shooting scene.

[0196] For example, when shooting a landscape scene, the light is usually concentrated in the upper region (such as the sky) of the scene, while the lower region (such as the ground or buildings) may be in the shade. At this time, the brightness of the upper region corresponding to the first pixel unit is higher than the brightness of the lower region corresponding to the second pixel unit, and the processor can set the first light filtering degree to be greater than the second light filtering degree.

[0197] For another example, there may be special cases of shooting angles. For example, when shooting at a low angle, the light reflected by the ground may cause the brightness of the lower region to be higher than that of the upper region. At this time, the brightness of the upper region corresponding to the first pixel unit is lower than the brightness of the lower region corresponding to the second pixel unit, and the processor can set the first light filtering degree to be less than the second light filtering degree to ensure that the light distribution of the entire scene is more uniform, thereby avoiding the situation that some regions in the image are too dark or too bright.

[0198] For example, the first pixel unit corresponds to the left region in the current shooting scene, and the second pixel unit corresponds to the right region in the current shooting scene.

[0199] For example, when shooting a city night scene, the left region of the street may be illuminated by neon lights, while the right region is relatively dark. The first pixel unit corresponds to the left neon light region, and a higher light filtering degree is required due to the extremely high brightness. The second pixel unit corresponds to the right dark region, and a lower light filtering degree is required due to the lower brightness to ensure that the levels and details of the dark part of the night scene can be fully displayed.

[0200] In some embodiments, the light filter includes an electronic light filter, and the light filtering degree of the electronic light filter is achieved by adjusting an electrical parameter of the electronic light filter. The electrical parameter includes at least one of voltage, current, or capacitance. The electronic light filter has higher flexibility and accuracy, and can quickly respond to changes in the shooting scene to optimize the image quality.

[0201] Before setting the filtering degree of the electronic filter to the first filtering degree at the first exposure period corresponding to the first pixel unit of the single-frame image captured by the image capturing device, the processor can determine a filtering degree control curve of the electronic filter within the total exposure period of the single-frame image, the filtering degree control curve at least describing that the filtering degree of the electronic filter is the first filtering degree at the first exposure period and the second filtering degree at the second exposure period within the total exposure period of the single-frame image. In this embodiment, the combination of the electronic filter and the processor in the image generation system can make the filtering process automatic and accurate, thereby reducing the manual operation of the user, which is particularly beneficial for complex shooting scenes, such as a situation where the light changes rapidly or the scene content is complex, and can significantly improve the shooting efficiency.

[0202] For example, the processor can set the filtering degree of the electronic filter to the first filtering degree at the first exposure period and the second filtering degree at the second exposure period within the total exposure period of the single-frame image based on the filtering degree control curve. Alternatively, the processor can send the filtering degree control curve to the electronic filter, so that the relevant control elements (such as microprocessors, central processing units, etc.) in the electronic filter set the filtering degree of the electronic filter to the first filtering degree at the first exposure period and the second filtering degree at the second exposure period based on the filtering degree control curve.

[0203] For example, in order to avoid the sudden jump of the filter between different filtering degrees, causing the fluctuation of the image quality, the processor can perform smoothing processing on the filtering degree control curve. Such smoothing processing can make the filter gradually change its filtering degree during the adjustment process, thereby avoiding the obvious brightness mutation in the transition area of the image. For example, if there is a significant difference between the first exposure period and the second exposure period, the smooth filtering degree control curve can form a natural transition between the two, so that the electronic filter can be smoothly adjusted from the first filtering degree to the second filtering degree.

[0204] For example, if the electrical parameter is voltage, the filtering degree control curve can be realized by adjusting the voltage of the electronic filter. Within the total exposure period of the single-frame image, the change of the voltage can be performed according to the preset curve, thereby adjusting the filtering degree. For example, when transitioning from the first filtering degree (corresponding to a higher voltage) to the second filtering degree (corresponding to a lower voltage), the voltage will smoothly decrease, avoiding the abrupt brightness change in the image.

[0205] In some embodiments, the filter is a mechanical filter, and the degree of filtering is achieved by adjusting the physical position or angle of the filter, changing the type or thickness of the filter material, or by selecting different filters. For example, the degree of filtering can be changed by moving or rotating the mechanical filter to different angles in the optical path of the lens assembly during the total exposure period of a single frame image, thereby changing the amount of light entering the image sensor. For another example, the mechanical filter can be composed of multiple layers of filters with different materials or thicknesses, and the degree of filtering can be changed by adjusting the combination of these filters with different materials or thicknesses during the total exposure period of a single frame image.

[0206] In some embodiments, the degree of filtering of the filter affects the intensity of light entering the camera sensor, thereby affecting the brightness and contrast of the image. Changes in the current shooting scene can cause changes in the light conditions, such as movement of the light source position or changes in the brightness of the scene. Therefore, the filter needs to adjust its degree of filtering according to these changes to ensure the quality of different frame images taken in the current shooting scene; that is, the degree of filtering of the filter during the exposure period of different frame images is different. The exposure period can be any one or more of the first exposure period, the second exposure period, or the total exposure period of different frame images. This embodiment dynamically adjusts the degree of filtering of the filter in different frame images, so that the image can maintain a better exposure effect under different lighting conditions, reduces the decline in image quality caused by inconsistent lighting, and improves the overall shooting effect.

[0207] For example, different shooting scenes can exhibit different brightness distributions, object types, and light source positions. These parameters directly affect the light conditions in different regions of the scene. Therefore, the degree of filtering of the filter during the exposure period of different frame images is different when the relevant parameters of the current shooting scene change. The relevant parameters include brightness or object category, but are not limited thereto, and can also be dynamically adjusted according to other relevant parameters, such as light source position, shooting angle, scene complexity, etc. to adapt to changes in the current shooting scene.

[0208] For example, different environmental brightnesses can cause the same degree of filtering to produce different exposure effects in different frame images. To solve this problem, the filter can be individually adjusted for the specific brightness conditions of each frame image. For example, in a darker scene, the filter can need to reduce the degree of filtering to increase the amount of light entering; while in a scene with higher brightness, the filter can need to increase the degree of filtering to avoid overexposure.

[0209] The different frame images at least include a first frame image and a second frame image; in the case that the ambient brightness when the first frame image is collected is different from the ambient brightness when the second frame image is collected, the filtering degree of the filter in the exposure period of the first frame image is different from the filtering degree of the filter in the exposure period of the second frame image, so that the filtering degree of the filter in different frame images is flexibly adjusted based on the ambient brightness.

[0210] It should be noted that the present application is exemplified by the image sensor collecting pixels row by row from top to bottom, and in other implementation manners, the image sensor can also collect pixels row by row from bottom to top, or collect pixels column by column from left to right or from right to left, which is not limited in the present application.

[0211] It should be noted that the present application is exemplified by two pixel units and two different filtering degrees, and in actual application, more filtering degrees can be divided according to actual conditions, for example, at least three different pixel units are divided, and the corresponding filtering degrees in the exposure period of each pixel unit are different.

[0212] In an exemplary embodiment, the image generation system is connected with an electronic filter for exemplary description:

[0213] Please refer to FIG. 8, the image generation system 100 includes a processor (including an image signal processor 11 and a microprocessor 12), an image sensor 21 and a lens assembly 22, wherein the electronic filter 31 is connected in front of the lens assembly 22.

[0214] When the image signal processor 11 in the image generation system 100 detects that the electronic filter 31 is in an available state, that is, the electronic filter 31 coincides with the optical path of the lens assembly 22, the function of filtering the light of the lens assembly 22 entering the image sensor 21 using the filter is started, and prompt information that the filtering function has been started is output; or the filtering function starting option is output for user confirmation, so that the user selects to start the function of filtering the light of the lens assembly 22 entering the image sensor 21 using the filter.

[0215] The subject matter in the current shooting scene passes through the electronic filter 31 and the lens assembly 22 to form an image on the image sensor 21.

[0216] The image signal processor 11 obtains a historical image frame of a current frame image to be shot for analysis, and obtains the light-dark boundary position of the historical image frame. The historical image frame and the current frame image to be shot both correspond to the current shooting scene, so the light-dark boundary position of the historical image frame usually has high consistency with the boundary position of the current scene, and can be used as a reference for the current frame image.

[0217] If the electronic filter 31 is also in an available state when the historical image frame is obtained, the historical image frame needs to be subjected to a brightness recovery process first, the process of which is to inversely apply the parameters of the filter, and in the brightness recovery process, the image signal processor 11 can compensate for the brightness attenuation caused by the electronic filter 31; and then the analysis process of the bright-dark boundary position is performed.

[0218] The image sensor 21 sends a synchronization signal of each frame of image to the image signal processor 11 and the microprocessor 12 when capturing each frame of image. For example, the synchronization signal is a frame synchronization signal (VSYNC), which can be used to control the exposure time of the image sensor 21, i.e. the exposure start time and the exposure duration of a frame of image.

[0219] The image signal processor 11 can determine the positions of the first exposure period and the second exposure period in the total exposure period of the single frame of image based on the determined bright-dark boundary position and the synchronization signal of the single frame of image; and the image signal processor 11 can generate a transmittance control curve based on the bright-dark boundary position divided by the brightness of the bright region and the brightness of the dark region of the historical image frame, the transmittance control curve at least describing the transmittance of the electronic filter 31 (such as an electronic ND filter) in different exposure periods in the total exposure period of the single frame of image, it is to be noted that the greater the transmittance, the smaller the filtering degree, the more light entering the lens assembly 22 of the image sensor 21, and the smaller the transmittance, the greater the filtering degree, the less light entering the lens assembly 22 of the image sensor 21. The transmittance of the electronic filter 31 (such as an electronic ND filter) can be changed by adjusting the voltage value thereof.

[0220] The mapping relationship between the voltage value of the electronic filter 31 and the filtering degree (such as transmittance) can be calibrated in advance.

[0221] For example, the horizontal axis of the filtering degree control curve is the total exposure period of the single frame of image, the vertical axis of the curve is the transmittance, and in order to make the image more natural, the transmittance control curve is subjected to smoothing processing so as to make the change of the transmittance more gentle, as shown in FIG. 9. The area formed by the transmittance control curve and the time axis describes the exposure amount of each pixel row, and it can be seen that the exposure amount of the first row of pixels is less than that of the eighth row of pixels under the action of the transmittance control curve.

[0222] Assuming that the single frame image has 3000 rows, and the readout time interval of every two rows is dt, the transmittance control curve sent by the image signal processor 11 to the microcontroller 12 is similar to: (0, 2V), (1000*dt, 2V), (2000*dt, 10V), (3000*dt, 10V), that is, the voltage of the electronic filter 31 is 2V between 0-2000 rows, and the voltage of the electronic filter 31 is 10V between 2000-3000 rows.

[0223] After obtaining the transmittance control curve of the single frame image, the image signal processor 11 can send the transmittance control curve to the microcontroller 12. The microcontroller 12 can send a first voltage value to the electronic filter 31 in the first exposure period of the image sensor 21 according to the transmittance control curve of the current frame image and the synchronization signal of the image sensor 21, so that the electronic filter 31 is in a first filtering degree; and send a second voltage value to the electronic filter 31 in the second exposure period of the image sensor 21, so that the electronic filter 31 is switched from the first filtering degree to a second filtering degree.

[0224] In the above example, the electronic filter 31 will exhibit a relatively low transmittance in the 0-2000th row, and the transmittance of the electronic filter 31 will be higher after the 2000th row, thereby facilitating the balancing of the brighter and darker areas in the image.

[0225] In some embodiments, referring to FIG. 10, the embodiment of the present application also provides another image generation method, which can be executed by the processor in the image generation system described above, and the method comprises the following steps:

[0226] In S100, first pixel data of a first pixel unit in a single frame image collected by an image collection device in a first exposure period is obtained, wherein the filtering degree of a filter of the image collection device in the first exposure period is a first filtering degree, and the filter is used for filtering light entering a lens assembly of the image collection device.

[0227] In S101, second pixel data of a second pixel unit in the single frame image collected by the image collection device in a second exposure period is obtained, wherein the filtering degree of the filter is adjusted from the first filtering degree to a second filtering degree in the second exposure period, the starting time of the second exposure period is located after the first exposure period, the first filtering degree is different from the second filtering degree, and the first pixel unit is different from the second pixel unit.

[0228] In S102, the single frame image comprising the first pixel data and the second pixel data is output.

[0229] In the embodiment, the light filtering degree of the filter is dynamically adjusted in the first exposure period corresponding to the first pixel unit and the second exposure period corresponding to the second pixel unit, and the light filtering degree of the filter is set as different first light filtering degree and second light filtering degree, so that the filter is in different light filtering degrees in different exposure periods to adapt to the shooting scene with light and dark contrast and improve the shooting effect.

[0230] The parts not mentioned in the embodiments of the present application can refer to the specific description of the foregoing embodiments, which will not be described here.

[0231] In some embodiments, referring to FIG. 11, the embodiments of the present application also provide another image generation method, which can be executed by the processor in the image generation system described above, and the method comprises:

[0232] In S110, the voltage value of the electronic filter is set as a first voltage value in a first exposure period corresponding to a first pixel unit in a single frame image collected by an image collection device, and first pixel data of the first pixel unit collected by the image collection device in the first exposure period is obtained, wherein the electronic filter is used to filter the light entering a lens assembly of the image collection device.

[0233] In S111, the voltage value of the electronic filter is adjusted from the first voltage value to a second voltage value in a second exposure period corresponding to a second pixel unit in the single frame image collected by the image collection device, and second pixel data of the second pixel unit collected by the image collection device in the second exposure period is obtained, wherein the starting time of the second exposure period is after the first exposure period, the first voltage value is different from the second voltage value, and the first pixel unit is different from the second pixel unit.

[0234] In S112, a single frame image including the first pixel data and the second pixel data is output.

[0235] The electronic filter adjusts its optical characteristics through voltage, thereby controlling the intensity of the transmitted light or the wave band of the light; the electronic filter can realize accurate and rapid light intensity control and is suitable for dynamic changing scenes.

[0236] In the embodiment, the voltage value of the electronic filter is dynamically adjusted in the first exposure period corresponding to the first pixel unit and the second exposure period corresponding to the second pixel unit, and the voltage value of the electronic filter is set as different first voltage value and second voltage value, so that the electronic filter is in different light filtering degrees in different exposure periods corresponding to the pixel data of different pixel units collected by the image collection device to adapt to the shooting scene with light and dark contrast and improve the shooting effect.

[0237] Portions of the present embodiments not mentioned can refer to the specific description of the foregoing embodiments, which will not be repeated here

[0238] In some embodiments, referring to FIG. 12, the present embodiments also provide another image generation method, which can be executed by the processor in the image generation system described above, and the method comprises:

[0239] In S120, first pixel data of a first pixel unit in a single-frame image collected by the image collection device in a first exposure period is obtained, wherein the voltage value of the electronic filter of the image collection device is a first voltage value in the first exposure period, and the electronic filter is used to filter the light entering the lens assembly of the image collection device.

[0240] In S121, second pixel data of a second pixel unit in the single-frame image collected by the image collection device in a second exposure period is obtained, wherein the voltage value of the electronic filter is adjusted from the first voltage value to a second voltage value in the second exposure period, the starting time of the second exposure period is after the first exposure period, the first voltage value is different from the second voltage value, and the first pixel unit is different from the second pixel unit.

[0241] In S122, the single-frame image comprising the first pixel data and the second pixel data is output.

[0242] In the present embodiment, the voltage value of the electronic filter is dynamically adjusted in the first exposure period corresponding to the first pixel unit of the single-frame image and the second exposure period corresponding to the second pixel unit, and the voltage value of the electronic filter is set to different first voltage value and second voltage value, so that the electronic filter is in different filtering degrees in different exposure periods corresponding to the pixel data of different pixel units collected by the image collection device, so as to adapt to the shooting scene with light and dark contrast and improve the shooting effect.

[0243] Portions of the present embodiments not mentioned can refer to the specific description of the foregoing embodiments, which will not be repeated here

[0244] Based on the same concept as the above method, referring to FIG. 13, the present embodiments also provide an image generation device 10, comprising:

[0245] one or more processors 13; and

[0246] one or more memories 14 for storing computer program instructions;

[0247] The one or more processors 13 are configured to invoke the computer program instructions to execute the above method.

[0248] The processor 13 can be an image signal processor, a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0249] The memory 14 can include at least one type of storage medium, including a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, and the like), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like.

[0250] For the device embodiment, since it basically corresponds to the method embodiment, the relevant part is described with reference to the method embodiment.

[0251] Based on the same concept as the above method, the embodiments of the present application also provide an image generation system, comprising a processor and an image sensor;

[0252] The processor is configured to control the light filter to set the light filtering degree to a first light filtering degree during a first exposure period corresponding to a first pixel unit in the image sensor for collecting a single frame of image, wherein the light filter is configured to filter the light entering a lens assembly of the image sensor.

[0253] The image sensor is configured to collect first pixel data of the first pixel unit during the first exposure period.

[0254] The processor is further configured to control the light filter to adjust the light filtering degree from the first light filtering degree to a second light filtering degree during a second exposure period corresponding to a second pixel unit in the image sensor for collecting a single frame of image, wherein the starting time of the second exposure period is located after the first exposure period, the first light filtering degree is different from the second light filtering degree, and the first pixel unit is different from the second pixel unit.

[0255] The image sensor is configured to collect second pixel data of the second pixel unit during the second exposure period.

[0256] The processor is further configured to output the single-frame image including the first pixel data and the second pixel data.

[0257] The image generation system further includes an electronic filter.

[0258] Based on the same concept as the above method, the embodiments of the present application further provide an image generation system including a processor and an image sensor.

[0259] The image sensor is configured to: collect first pixel data of a first pixel unit in a single-frame image in a first exposure period, wherein a light filtering degree of a filter of the image sensor in the first exposure period is a first light filtering degree, and the filter is configured to filter light entering a lens assembly of the image sensor.

[0260] The image sensor is further configured to: collect second pixel data of a second pixel unit in the single-frame image in a second exposure period, wherein the light filtering degree of the filter is adjusted from the first light filtering degree to a second light filtering degree in the second exposure period, a starting time of the second exposure period is after the first exposure period, the first light filtering degree is different from the second light filtering degree, and the first pixel unit is different from the second pixel unit.

[0261] The processor is configured to output the single-frame image including the first pixel data and the second pixel data.

[0262] The image generation system further includes the filter.

[0263] Based on the same concept as the above method, the embodiments of the present application further provide an image generation system including a processor and an image sensor.

[0264] The processor is configured to control the electronic filter to set the voltage value to a first voltage value in a first exposure period corresponding to the first pixel unit in the image sensor collecting the single-frame image, wherein the electronic filter is configured to filter light entering a lens assembly of the image sensor.

[0265] The image sensor is configured to collect the first pixel data of the first pixel unit in the first exposure period.

[0266] The processor is further configured to control the electronic filter to adjust the voltage value from the first voltage value to a second voltage value in a second exposure period corresponding to a second pixel unit in the image sensor collecting the single-frame image, wherein a starting time of the second exposure period is after the first exposure period, the first voltage value is different from the second voltage value, and the first pixel unit is different from the second pixel unit.

[0267] The image sensor is further configured to collect the second pixel data of the second pixel unit in the second exposure period.

[0268] The processor is further configured to output the single-frame image including the first pixel data and the second pixel data.

[0269] Based on the same concept as the above method, the embodiments of the present application further provide an image generation system including a processor and an image sensor.

[0270] The image sensor is configured to collect first pixel data of first pixel units in the single-frame image in a first exposure period, wherein a voltage value of an electronic filter of the image sensor is a first voltage value in the first exposure period, and the electronic filter is configured to filter light entering a lens assembly of the image sensor.

[0271] The image sensor is further configured to collect second pixel data of second pixel units in the single-frame image in a second exposure period, wherein the voltage value of the electronic filter is adjusted from the first voltage value to a second voltage value in the second exposure period, a starting time of the second exposure period is after the first exposure period, the first voltage value is different from the second voltage value, and the first pixel units are different from the second pixel units.

[0272] The processor is configured to output the single-frame image including the first pixel data and the second pixel data.

[0273] In some embodiments, the embodiments of the present application further provide an electronic device including the above-mentioned image generation system.

[0274] The electronic device includes at least one of a movable platform and a handheld device.

[0275] The movable platform includes at least one of an aircraft, a vehicle, a ship, or a movable robot.

[0276] The handheld device includes at least one of a handheld gimbal and a gimbal camera.

[0277] In some embodiments, a non-transitory computer-readable storage medium including computer instructions is also provided, for example, a memory including computer instructions, the above-mentioned computer instructions being executable by a processor of an image generation device to complete the above-mentioned method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0278] A non-transitory computer readable storage medium, when the computer instructions in the storage medium are executed by a processor of an image generation device, enable the image generation device to perform the above method. It should be noted that, in this paper, relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or sequence between the entities or operations. The terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0279] The above describes the method and related device provided by the embodiments of the present application in detail, the principles and implementation manners of the present application are described by applying specific examples, and the above embodiment description is only used to help understand the method and core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed, and the above description should not be understood as the limitation of the present application.

Claims

1. An image generation method, characterized in that, include: The electronic filter is controlled to set the voltage value to a first voltage value during the first exposure period corresponding to the first pixel unit in a single frame image acquired by the image acquisition device, and to acquire the first pixel data of the first pixel unit acquired by the image acquisition device during the first exposure period. The electronic filter is used to filter the light entering the lens assembly of the image acquisition device. The electronic filter is controlled to adjust the voltage value from the first voltage value to the second voltage value during the second exposure period corresponding to the second pixel unit in the single frame image acquired by the image acquisition device, and the second pixel data of the second pixel unit acquired by the image acquisition device during the second exposure period is obtained. The start time of the second exposure period is after the first exposure period, the first voltage value is different from the second voltage value, and the first pixel unit is different from the second pixel unit. The output is the single-frame image that includes the first pixel data and the second pixel data.

2. An image generation method, characterized in that, include: The image acquisition device acquires first pixel data of the first pixel unit in a single frame image acquired during the first exposure period, wherein the voltage value of the electronic filter of the image acquisition device during the first exposure period is a first voltage value, and the electronic filter is used to filter the light entering the lens assembly of the image acquisition device. The second pixel data of the second pixel unit in the single frame image acquired by the image acquisition device during the second exposure period is obtained, wherein the voltage value of the electronic filter is adjusted from the first voltage value to the second voltage value during the second exposure period, the start time of the second exposure period is after the first exposure period, the first voltage value is different from the second voltage value, and the first pixel unit is different from the second pixel unit; The output is the single-frame image that includes the first pixel data and the second pixel data.

3. An image generation method, characterized in that, include: The control filter sets the filtering degree to a first filtering degree during the first exposure period corresponding to the first pixel unit in a single frame image acquired by the image acquisition device, and obtains the first pixel data of the first pixel unit acquired by the image acquisition device during the first exposure period. The filter is used to filter the light entering the lens assembly of the image acquisition device. The filter is controlled to adjust the filtering degree from the first filtering degree to the second filtering degree during the second exposure period corresponding to the second pixel unit in the single frame image acquired by the image acquisition device, and the second pixel data of the second pixel unit acquired by the image acquisition device during the second exposure period is obtained. The start time of the second exposure period is after the first exposure period, the first filtering degree is different from the second filtering degree, and the first pixel unit is different from the second pixel unit. The output is the single-frame image that includes the first pixel data and the second pixel data.

4. The method according to claim 3, characterized in that, The parameters of the current shooting scene area corresponding to the first pixel unit are different from the parameters of the current shooting scene area corresponding to the second pixel unit.

5. The method according to claim 4, characterized in that, The relevant parameters include brightness or object category.

6. The method according to any one of claims 3 to 5, characterized in that, The positions of the first exposure period and / or the second exposure period within the total exposure period of the single frame image are changeable.

7. The method according to claim 6, characterized in that, The positions of the first exposure period and / or the second exposure period within the total exposure period of the single frame image may change as the current shooting scene changes.

8. The method according to claim 7, characterized in that, The positions of the first exposure period and / or the second exposure period within the total exposure period of the single frame image may change as relevant parameters of the current shooting scene change.

9. The method according to claim 8, characterized in that, The relevant parameters include brightness or object category.

10. The method according to claim 8 or 9, characterized in that, The positions of the first exposure period and / or the second exposure period within the total exposure period of the single frame image can change as the position of the light-dark boundary of the current shooting scene changes.

11. The method according to claim 6, characterized in that, The positions of the first exposure period and / or the second exposure period within the total exposure period of the single frame image are set by the user.

12. The method according to claim 11, characterized in that, The positions of the first exposure period and / or the second exposure period within the total exposure period of the single frame image are set by the user via a remote control device.

13. The method according to claim 11, characterized in that, The position of the first exposure period and / or the second exposure period within the total exposure period of the single frame image is selected by the user from a variety of preset filter options, wherein each of the preset filter options corresponds to a position of the first exposure period and / or the second exposure period within the total exposure period of the single frame image.

14. The method according to claim 6, characterized in that, The positions of the first exposure period and / or the second exposure period within the total exposure period of the single frame image are automatically determined based on relevant parameters of the current shooting scene.

15. The method according to claim 14, characterized in that, The positions of the first exposure period and / or the second exposure period within the total exposure period of the single frame image are automatically determined based on the light-dark boundary position of the current shooting scene.

16. The method according to claim 15, characterized in that, The position of the first exposure period and / or the second exposure period within the total exposure period of the single frame image is related to the brightness boundary position of the historical image frame of the single frame image acquired by the image acquisition device. Both the single frame image and the historical image frame correspond to the current shooting scene.

17. The method according to claim 16, characterized in that, The brightness difference between different pixel units at both ends of the brightness-dark boundary in the historical image frame is greater than a preset brightness difference value.

18. The method according to claim 17, characterized in that, Different shooting scenarios require different preset brightness difference values.

19. The method according to claim 16, characterized in that, In the image capture scenario, the historical image frames include viewfinder images or preview images; In a video shooting scenario, in response to the single-frame image being the first frame, the historical image frames include the viewfinder image or the preview image; or, in response to the single-frame image not being the first frame, the historical image frames include image frames captured by the image acquisition device before capturing the single-frame image.

20. The method according to claim 16, characterized in that, If the filter is in use during the acquisition of the historical image frame, the position of the first exposure period and / or the second exposure period within the total exposure period of the single frame image is related to the brightness boundary position of the historical image frame after brightness restoration processing.

21. The method according to any one of claims 3 to 20, characterized in that, The first degree of filtering and / or the second degree of filtering can be changed.

22. The method according to claim 21, characterized in that, The first level of filtering and / or the second level of filtering may change as the current shooting scene changes.

23. The method according to claim 22, characterized in that, The first level of filtering and / or the second level of filtering may change as relevant parameters of the current shooting scene change.

24. The method according to claim 21, characterized in that, The first filtering degree and / or the second filtering degree are set by the user.

25. The method according to claim 24, characterized in that, The first filtering degree and / or the second filtering degree are set by the user via a remote control device.

26. The method according to claim 24, characterized in that, The first filtering degree and / or the second filtering degree are selected by the user from a variety of preset filtering options, wherein each of the preset filtering options corresponds to a first filtering degree and / or a second filtering degree.

27. The method according to claim 21, characterized in that, The first filtering degree and / or the second filtering degree are automatically determined based on relevant parameters of the current shooting scene.

28. The method according to claim 23 or 27, characterized in that, The relevant parameters include brightness or object category.

29. The method according to claim 21, characterized in that, The first degree of filtering is related to the relevant parameters of the area of ​​the current shooting scene corresponding to the first pixel unit, and the second degree of filtering is related to the relevant parameters of the area of ​​the current shooting scene corresponding to the second pixel unit.

30. The method according to claim 29, characterized in that, The brightness of the area corresponding to the current shooting scene of the first pixel unit is greater than the brightness of the area corresponding to the current shooting scene of the second pixel unit, and the first filtering degree is greater than the second filtering degree.

31. The method according to claim 29, characterized in that, The brightness of the area corresponding to the current shooting scene of the first pixel unit is less than the brightness of the area corresponding to the current shooting scene of the second pixel unit, and the first filtering degree is less than the second filtering degree.

32. The method according to claim 30 or 31, characterized in that, The first pixel unit and the second pixel unit respectively correspond to different side regions of the current shooting scene.

33. The method according to claim 32, characterized in that, The first pixel unit corresponds to the upper region of the current shooting scene, and the second pixel unit corresponds to the lower region of the current shooting scene.

34. The method according to any one of claims 3 to 33, characterized in that, The degree of filtering is related to the amount of light entering the image acquisition device.

35. The method according to claim 34, characterized in that, The greater the filtering degree of the filter, the less light enters the lens assembly of the image acquisition device; the smaller the filtering degree of the filter, the more light enters the lens assembly of the image acquisition device.

36. The method according to any one of claims 3 to 33, characterized in that, The degree of filtering is related to the wavelength of light entering the filter.

37. The method according to claim 36, characterized in that, The greater the filtering degree of the filter, the fewer wavelengths of light enter the lens assembly of the image acquisition device; the smaller the filtering degree of the filter, the more wavelengths of light enter the lens assembly of the image acquisition device.

38. The method according to any one of claims 3 to 37, characterized in that, The filter may be an electronic filter or a mechanical filter.

39. The method according to any one of claims 3 to 37, characterized in that, The filter includes an electronic filter, and the filtering degree of the filter is achieved by adjusting the electrical parameters of the electronic filter.

40. The method according to claim 39, characterized in that, The electrical parameters include at least one of voltage, current, and capacitance.

41. The method according to claim 39, characterized in that, Before the control filter sets the filter level to the first filter level during the first exposure period corresponding to the first pixel unit in a single frame image acquired by the image acquisition device, the control filter further includes: A filtering degree control curve of the electronic filter is determined during the total exposure period of the single frame image. The filtering degree control curve is used to describe at least the filtering degree of the electronic filter during the first exposure period as a first filtering degree and the filtering degree during the second exposure period as a second filtering degree during the total exposure period of the single frame image.

42. The method according to any one of claims 3 to 41, characterized in that, The first pixel unit and / or the second pixel unit includes at least one of the following: a pixel point, a pixel row, or a pixel region.

43. The method according to claim 42, characterized in that, The first pixel unit and the second pixel unit correspond to different side regions in the single frame image.

44. The method according to claim 43, characterized in that, The first pixel unit corresponds to the upper part of the single-frame image, and the second pixel unit corresponds to the lower part of the single-frame image.

45. The method according to claim 42, characterized in that, The first pixel unit and the second pixel unit comprise adjacent, distinct rows of pixels in the single-frame image.

46. ​​The method according to any one of claims 3 to 45, characterized in that, The filter has different filtering levels during the exposure periods of different frame images.

47. The method according to claim 46, characterized in that, When the relevant parameters of the current shooting scene change, the filter has different filtering levels during the exposure time of different frame images.

48. The method according to claim 47, characterized in that, The different frame images include at least a first frame image and a second frame image; The ambient brightness when the first frame image is acquired is different from the ambient brightness when the second frame image is acquired, and the filtering degree of the filter during the exposure period of the first frame image is different from the filtering degree during the exposure period of the second frame image.

49. The method according to any one of claims 3 to 45, characterized in that, The second exposure period is located at different positions within the total exposure period of different frame images.

50. The method according to claim 49, characterized in that, When the brightness distribution of the current shooting scene changes, the second exposure time period will be located at a different position within the total exposure time period of different frame images.

51. The method according to any one of claims 3 to 50, characterized in that, The filter is detachably connected to the image acquisition device; or the filter is integrated with the image acquisition device.

52. The method according to any one of claims 3 to 50, characterized in that, The image acquisition device includes an image sensor, and the filter is closer to the object side than the image sensor.

53. The method according to claim 52, characterized in that, The image acquisition device also includes a lens assembly; The filter is positioned in front of the lens assembly; or... The filter is disposed between the lens assembly and the image sensor; or... The lens assembly includes at least two lenses, and the filter is disposed between any two lenses.

54. The method according to any one of claims 51 to 53, characterized in that, Also includes: In response to determining that the filter is in an available state, a filter function activation indication message is output, wherein the filter function activation indication message is used to indicate that the function of using the filter to filter the light entering the lens assembly of the image acquisition device is enabled, or to indicate that the user selects to enable the function of using the filter to filter the light entering the lens assembly of the image acquisition device.

55. The method according to claim 54, characterized in that, Also includes: When the filter is detachably connected to the image acquisition device, in response to detecting that the filter is connected to the image acquisition device through a contact point, it is determined that the filter is in a usable state; or, When the filter is integrated with the image acquisition device, in response to detecting that the filter moves from a first position to a second position, it is determined that the filter is in an available state; wherein, the filter in the first position is offset from the optical path of the lens assembly of the image acquisition device, and the filter in the second position is aligned with the optical path of the lens assembly.

56. An image generation method, characterized in that, include: The image acquisition device acquires the first pixel data of the first pixel unit in a single frame image acquired during the first exposure period, wherein the filtering degree of the filter of the image acquisition device during the first exposure period is the first filtering degree, and the filter is used to filter the light entering the lens assembly of the image acquisition device. The second pixel data of the second pixel unit in the single frame image acquired by the image acquisition device during the second exposure period is obtained, wherein the filtering degree of the filter is adjusted from the first filtering degree to the second filtering degree during the second exposure period, the start time of the second exposure period is after the first exposure period, the first filtering degree is different from the second filtering degree, and the first pixel unit is different from the second pixel unit; The output is the single-frame image that includes the first pixel data and the second pixel data.

57. An image generation apparatus, characterized in that, include: One or more processors; as well as One or more memories used to store computer program instructions; One or more of the processors are configured to invoke the computer program instructions to perform the method according to any one of claims 1 to 56.

58. An image generation system, characterized in that, Including processors and image sensors; The processor is configured to: control the electronic filter to set the voltage value to a first voltage value during the first exposure period corresponding to the first pixel unit in the single frame image acquired by the image sensor, wherein the electronic filter is configured to filter the light entering the lens assembly of the image sensor; The image sensor is used to: acquire first pixel data of the first pixel unit during the first exposure period; The processor is further configured to: control the electronic filter to adjust the voltage value from the first voltage value to the second voltage value during the second exposure period corresponding to the second pixel unit in the single frame image acquired by the image sensor, wherein the start time of the second exposure period is after the first exposure period, the first voltage value is different from the second voltage value, and the first pixel unit is different from the second pixel unit; The image sensor is also used to: acquire second pixel data of the second pixel unit during the second exposure period; The processor is also configured to output the single-frame image including the first pixel data and the second pixel data.

59. An image generation system, characterized in that, Including processors and image sensors; The image sensor is used to: acquire first pixel data of a first pixel unit in a single frame image during a first exposure period, wherein the voltage value of the electronic filter of the image sensor is a first voltage value during the first exposure period, and the electronic filter is used to filter the light entering the lens assembly of the image sensor; The image sensor is also used to: acquire second pixel data of the second pixel unit in the single frame image during the second exposure period, wherein the voltage value of the electronic filter is adjusted from the first voltage value to the second voltage value during the second exposure period, the start time of the second exposure period is after the first exposure period, the first voltage value is different from the second voltage value, and the first pixel unit is different from the second pixel unit; The processor is configured to: output the single-frame image including the first pixel data and the second pixel data.

60. An image generation system, characterized in that, Including processors and image sensors; The processor is configured to: control the filter to set the filtering degree to a first filtering degree during the first exposure period corresponding to the first pixel unit in the single frame image acquired by the image sensor, wherein the filter is configured to filter the light entering the lens assembly of the image sensor; The image sensor is used to: acquire first pixel data of the first pixel unit during the first exposure period; The processor is further configured to: control the filter to adjust the filtering degree from the first filtering degree to the second filtering degree during the second exposure period corresponding to the second pixel unit in the single frame image acquired by the image sensor, wherein the start time of the second exposure period is after the first exposure period, the first filtering degree is different from the second filtering degree, and the first pixel unit is different from the second pixel unit; The image sensor is used to: acquire second pixel data of the second pixel unit during the second exposure period; The processor is also configured to output the single-frame image including the first pixel data and the second pixel data.

61. The image generation system according to claim 60, characterized in that, The image generation system also includes the filter.

62. An image generation system, characterized in that, Including processors and image sensors; The image sensor is used to: acquire first pixel data of a first pixel unit in a single frame image during a first exposure period, wherein the filtering degree of the filter of the image sensor is a first [value missing] during the first exposure period. The degree of light filtering, wherein the filter is used to filter the light entering the lens assembly of the image sensor; The image sensor is also used to: acquire second pixel data of the second pixel unit in the single frame image during the second exposure period, wherein the filtering degree of the filter is adjusted from the first filtering degree to the second filtering degree during the second exposure period, the start time of the second exposure period is after the first exposure period, the first filtering degree is different from the second filtering degree, and the first pixel unit is different from the second pixel unit; The processor is configured to: output the single-frame image including the first pixel data and the second pixel data.

63. The image generation system according to claim 62, characterized in that, The image generation system also includes the filter.

64. An electronic device, characterized in that, It includes the image generation apparatus as described in claim 57, or the image generation system as described in any one of claims 58 to 63.

65. The electronic device according to claim 64, characterized in that, The electronic device includes at least one of a mobile platform and a handheld device.

66. The electronic device according to claim 65, characterized in that, The mobile platform includes at least one of the following: aircraft, vehicle, vessel, or mobile robot; and / or The handheld device includes a handheld gimbal and / or a gimbal camera.

67. A computer-readable storage medium storing computer instructions thereon, characterized in that, When executed by the processor, this instruction implements the steps of the method as described in any one of claims 1 to 56.

Images