Image processing method and related equipment

Abstract

The invention provides an image processing method and a related device. The method comprises the following steps: acquiring at least one piece of brightness event information of a target position detected by an event-based visual sensor EVS; according to the at least one piece of brightness event information and an exposure time window of a target sensor unit, a compensation parameter is determined, and the target sensor unit is a sensor unit used for obtaining an optical signal of the target position in a complementary metal oxide semiconductor (CIS) image sensor; acquiring a first output signal corresponding to the target sensor unit; and compensating the first output signal according to the compensation parameter to obtain a second output signal. According to the technical scheme, the position and time of the brightness change in the shot scene are detected through the EVS; and based on the time when the brightness change occurs and the exposure time, determining a compensation parameter to compensate a signal output by a target sensor unit used for acquiring the target position in the CIS. In this way, the stripe phenomenon can be reduced or even avoided.

Description

Technical Field

[0001] The present application relates to the field of image processing, and more specifically, to a method for processing an image and related devices. Background Art

[0002] Pulse width modulation (PWM) dimming is a common technology for controlling the brightness of light sources. PWM dimming is widely used in many fields such as lighting systems and display screens. PWM dimming controls the average brightness of the light source by quickly turning the light source on and off with different duty cycles.

[0003] In the process of turning the light source on and off, the actual brightness of the light source changes at a high frequency. However, due to the visual characteristics of the human eye, this brightness change cannot be detected by the human eye. However, when using a camera to shoot a device that uses PWM dimming technology (such as a desk lamp, display screen) or an object illuminated by a lighting system that uses PWM dimming, the exposure time does not completely match the PWM dimming cycle, which will cause stripes or color bands to appear in the captured image. This phenomenon can be called "striping phenomenon."

[0004] The appearance of streaks will greatly reduce the quality of the acquired image. Therefore, how to avoid the streaks is an urgent problem to be solved in the industry. Summary of the invention

[0005] The present application provides a method and related device for processing an image, which can improve image quality and reduce or even avoid the occurrence of stripe phenomenon.

[0006] In a first aspect, an embodiment of the present application provides a method for processing an image, comprising: obtaining at least one brightness event information of a target position detected by an event-based visual sensor EVS, each brightness event information in the at least one brightness event information being used to indicate the type of brightness change event occurring at the target position and the time when the brightness change event occurs, the type of brightness change event including brightness increase or brightness decrease; determining a compensation parameter based on the at least one brightness event information and an exposure time window of a target sensor unit, wherein the target sensor unit is a sensor unit in a complementary metal oxide semiconductor image sensor CIS for acquiring a light signal at the target position; obtaining a first output signal corresponding to the target sensor unit; and compensating the first output signal based on the compensation parameter to obtain a second output signal.

[0007] In the above technical solution, the position (i.e., the target position) and time at which the brightness change occurs in the captured scene are detected by the EVS. Based on the time at which the brightness change occurs and the exposure time, compensation parameters are determined to compensate the signal output by the target sensor unit in the CIS for obtaining the target position. In this way, the occurrence of the streak phenomenon can be reduced or even avoided.

[0008] In combination with the first aspect, in a possible implementation of the first aspect, the compensation parameter is determined based on the at least one brightness event information and the exposure time window of the target sensor unit, including: determining a reference time window based on the at least one brightness event information and the exposure time window, wherein the brightness value of the target position within the reference time window is different from the brightness value outside the reference time window; determining the compensation parameter based on the reference time window and the exposure time window.

[0009] In combination with the first aspect, in a possible implementation method of the first aspect, determining a reference time window based on the at least one brightness event information and the exposure time window includes: determining N reference brightness event information from the at least one brightness event information based on the exposure time window and the moment indicated by the at least one brightness event information, wherein any one of the N reference moments is within the range of the exposure time window, the N reference moments are respectively moments indicated by the N reference brightness event information, and N is a positive integer greater than or equal to 1; determining the reference time window based on the N reference brightness event information and the exposure time window.

[0010] In combination with the first aspect, in a possible implementation method of the first aspect, when N is equal to 1, the reference time window is determined based on the N reference brightness event information and the exposure time window, including: when the type of the brightness change event indicated by the N reference brightness event information is the brightness reduction, the starting time of the reference time window is determined to be the first time, and the end time of the reference time window is the end time of the exposure time window, and the first time is the time indicated by the N reference brightness event information; when the type of the brightness change event indicated by the N reference brightness event information is the brightness increase, the starting time of the reference time window is determined to be the starting time of the exposure time window, and the end time of the reference time window is the first time.

[0011] In combination with the first aspect, in a possible implementation of the first aspect, when N is equal to 2, the reference time window is determined based on the N reference brightness event information and the exposure time window, including: determining second brightness event information and third brightness event information from the N reference brightness event information, wherein the type of brightness change event indicated by the second brightness event information is the brightness rise, and the type of brightness change event indicated by the third brightness event information is the brightness decrease, the second moment is earlier than the third moment, the second moment is the moment indicated by the second brightness event information, and the third moment is the moment indicated by the third brightness event information; determining the starting moment of the reference time window as the second moment, and the ending moment of the reference time window as the third moment.

[0012] In combination with the first aspect, in a possible implementation manner of the first aspect, the reference time window, the exposure time window, and the compensation parameter satisfy the following relationship: F=T ref / T exp , where F is the compensation parameter, T ref is the length of the reference time window, T exp is the duration of the exposure time window.

[0013] In combination with the first aspect, in a possible implementation of the first aspect, the compensation parameter, the first output signal and the second output signal satisfy the following relationship: P'=P / (F+c), where P' is the second output signal, P is the first output signal, F is the compensation parameter, and c is the correction parameter.

[0014] In a second aspect, an embodiment of the present application provides an electronic device, which includes a photographic system and a processor, wherein the photographic system includes an event-based visual sensor EVS and a complementary metal oxide semiconductor image sensor CIS, wherein the EVS detects brightness changes at a target position and determines at least one brightness event information of the target position, each brightness event information in the at least one brightness event information is used to indicate a brightness change event that occurs at the target position and a time when the brightness change event occurs, and the brightness change event includes an increase in brightness or a decrease in brightness; the processor is used to obtain the at least one brightness event information from the EVS, and determine a compensation parameter based on the at least one brightness event information and an exposure time window of a target sensor unit, wherein the target sensor unit is a sensor unit in the CIS for obtaining a light signal at the target position; the processor is also used to obtain a first output signal corresponding to the target sensor unit, and compensate the first output signal based on the compensation parameter to obtain a second output signal.

[0015] In combination with the second aspect, in a possible implementation of the second aspect, the processor is specifically used to: determine a reference time window based on the at least one brightness event information and the exposure time window, wherein the brightness value of the target position within the reference time window is different from the brightness value outside the reference time window; determine the compensation parameter based on the reference time window and the exposure time window.

[0016] In combination with the second aspect, in a possible implementation method of the second aspect, the processor is specifically used to: determine N reference brightness event information from the at least one brightness event information according to the exposure time window and the moment indicated by the at least one brightness event information, wherein any reference moment among the N reference moments is later than the start moment of the exposure time window and earlier than the end moment of the exposure time window, and the N reference moments are respectively the moments indicated by the N reference brightness event information, and N is a positive integer greater than or equal to 1; determine the reference time window according to the N reference brightness event information and the exposure time window.

[0017] In combination with the second aspect, in a possible implementation method of the second aspect, the processor is specifically used to: when it is determined that N is equal to 1 and the type of brightness change event indicated by the N reference brightness event information is the brightness reduction, determine the starting moment of the reference time window to be the first moment, the end moment of the reference time window to be the end moment of the exposure time window, and the first moment is the moment indicated by the N reference brightness event information; when it is determined that N is equal to 1 and the type of brightness change event indicated by the N reference brightness event information is the brightness increase, determine the starting moment of the reference time window to be the starting moment of the exposure time window, and the end moment of the reference time window to be the first moment.

[0018] In combination with the second aspect, in a possible implementation of the second aspect, the processor is specifically used to: when determining that N is equal to 2, determine the second brightness event information and the third brightness event information from the N reference brightness event information, wherein the type of brightness change event indicated by the second brightness event information is the brightness rise, the type of brightness change event indicated by the third brightness event information is the brightness decrease, the second moment is earlier than the third moment, the second moment is the moment indicated by the second brightness event information, and the third moment is the moment indicated by the third brightness event information; determine the starting moment of the reference time window to be the second moment, and the ending moment of the reference time window to be the third moment.

[0019] In combination with the second aspect, in a possible implementation manner of the second aspect, the reference time window, the exposure time window and the compensation parameter satisfy the following relationship: F=T ref / T exp , where F is the compensation parameter, T refis the length of the reference time window, T exp is the duration of the exposure time window.

[0020] In combination with the second aspect, in a possible implementation of the second aspect, the compensation parameter, the first output signal and the second output signal satisfy the following relationship: P'=P / (F+c), where P' is the second output signal, P is the first output signal, F is the compensation parameter, and c is the correction parameter.

[0021] In combination with the second aspect, in a possible implementation of the second aspect, the photography system also includes a first lens module and a second lens module, wherein the EVS detects the brightness change of the target position through the first lens module; and the CIS obtains the light signal of the target position through the second lens module.

[0022] In combination with the second aspect, in a possible implementation of the second aspect, the photographic system also includes a third lens module and a spectrometer, the spectrometer obtains incident light from the third lens module, and divides the incident light into a first outgoing light and a second outgoing light, the EVS detects the brightness change of the target position through the first outgoing light; the CIS detects the brightness change of the target position through the second outgoing light.

[0023] In combination with the second aspect, in a possible implementation of the second aspect, the photographic system also includes a fourth lens module, wherein the CIS includes multiple EVS areas, the EVS includes multiple EVS modules, and the multiple EVS modules are respectively located in the multiple EVS areas; the EVS detects the brightness change of the target position through the fourth lens module; and the CIS detects the photoelectric signal of the target position through the fourth lens module.

[0024] In combination with the second aspect, in a possible implementation of the second aspect, the photography system also includes a fifth lens module, wherein the EVS is located in front of the CIS, and the EVS detects the brightness change of the target position through the fifth lens module; the CIS detects the photoelectric signal of the target position through the fifth lens module.

[0025] In a third aspect, an embodiment of the present application provides a chip system, comprising: a logic circuit, which is used to couple with an input / output interface and transmit data through the input / output interface to execute a method such as the first aspect or any possible implementation method of the first aspect.

[0026] In a fourth aspect, an embodiment of the present application provides a computer-readable medium, which stores a program code. When the computer program code runs on a computer, the computer executes a method as in the first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a schematic diagram of a sensor array of an image sensor.

[0028] Figure 2 Schematic diagram of two PWM signals.

[0029] Figure 3 This is a schematic diagram of the shooting process.

[0030] Figure 4 This is another schematic diagram of the shooting process.

[0031] Figure 5 It is a schematic deconstruction block diagram of an electronic device provided according to an embodiment of the present application.

[0032] Figure 6 It is a schematic deconstruction block diagram of a photography system provided according to an embodiment of the present application.

[0033] Figure 7 It is a schematic deconstruction block diagram of another photography system provided according to an embodiment of the present application.

[0034] Figure 8 It is a schematic deconstruction block diagram of another photography system provided according to an embodiment of the present application.

[0035] Fig. 9 It is a schematic deconstruction block diagram of another photography system provided according to an embodiment of the present application.

[0036] Fig.10 It is a schematic flowchart of a method for processing an image provided according to an embodiment of the present application.

[0037] Fig.11 The shooting area of ​​the EVS is shown.

[0038] Fig.12 It is a schematic flowchart of a method for processing an image provided according to an embodiment of the present application.

[0039] Fig.13 A schematic diagram showing the signals corresponding to the target sensor. DETAILED DESCRIPTION

[0040] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0041] In the embodiments of the present application, words such as "exemplary" and "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present concepts in a concrete way.

[0042] In the embodiments of the present application, “corresponding” and “relevant” may sometimes be used interchangeably. It should be noted that when the distinction between them is not emphasized, the meanings they intend to express are consistent.

[0043] The network architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. A person of ordinary skill in the art can appreciate that with the evolution of the network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0044] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear at different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0045] In the present application, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: including the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0046] Figure 1 is a schematic diagram of a sensor array of an image sensor.

[0047] like Figure 1The sensor array 100 shown includes 6 photoelectric sensor rows, a total of 6×6 photoelectric sensors 101. The 6 photoelectric sensor rows are photoelectric sensor rows 110 to 160. Each photoelectric sensor row includes 6 photoelectric sensors. The photoelectric sensor 101 can be a photoelectric conversion element such as a photodiode, which is not limited in the embodiment of the present application.

[0048] During the shooting process, the photoelectric sensors in the sensor array 100 are exposed row by row to obtain a captured image.

[0049] The following uses a light emitting diode (LED) lamp as an example to introduce PWM dimming.

[0050] Connect the PWM signal to the LED driving circuit of the LED lamp. The LED driving circuit usually includes a switching element (such as a metal-oxide-semiconductor field effect transistor (MOSFET). The PWM signal can control the switching element in the LED driving circuit to be in an on or off state. When the switching element is in the on state, the LED lamp beads in the LED lamp light up; when the switching element is in the off state, the LED lamp beads are off. Due to the afterglow effect, when the LED lamp beads flicker at a very high frequency, the human eye will not perceive this flicker. The longer the LED lamp beads are on, the brighter the brightness recognized by the human eye; the shorter the LED lamp beads are on, the darker the brightness recognized by the human eye. Therefore, the brightness of the LED lamp can be adjusted by adjusting the duty cycle of the PWM signal.

[0051] Figure 2 Two PWM signals are shown.

[0052] The duty cycle of the PWM signal is the ratio of the pulse width to the pulse period. When the PWM signal is at a high level, the switch element of the LED drive circuit is in the on state, and the LED lamp bead is off; when the PWM signal is at a low level, the switch element of the LED drive circuit is in the off state, and the LED lamp bead is off.

[0053] like Figure 2 As shown, the pulse period of PWM signal 1 is equal to the pulse period of PWM signal 2, but the pulse width of PWM signal 1 is greater than the pulse width of PWM signal 2. Therefore, the brightness of the LED lamp driven by PWM signal 1 is higher than the brightness of the LED lamp driven by PWM signal 2.

[0054] Figure 3 This is a schematic diagram of the shooting process.

[0055] like Figure 3As shown, the photoelectric sensors in the sensor array capture the display object displayed in the display using the PWM dimming technology by exposing the display line by line. Figure 3 In the shooting process shown, the exposure duration of the photosensor is the same as the pulse period of the PWM signal. When the PWM signal is at a high level, the display element (such as an LED) in the display screen is lit; when the PWM signal is at a low level, the display element is turned off. Therefore, when the exposure duration of the photosensor is the same as the pulse period of the PWM signal, the duration of the light received by the photosensor is the same as the duration of the display element being lit. In other words, the duration of the light received by each photosensor in the sensor array is the same as the pulse width of the PWM signal.

[0056] The function of a photoelectric sensor is to convert received photons into electrical signals. The more photons a photoelectric sensor receives, the stronger the electrical signal it generates. Since the photoelectric sensor receives light for the same length of time, the strength of the electrical signal output by the photoelectric sensor is also the same, so the brightness of the pixels in the output image is also the same, and no stripes will occur.

[0057] It is understandable that if the exposure time of the photoelectric sensor is a positive integer multiple of the pulse period of the PWM signal, then the time the photoelectric sensor receives light is also the same (i.e., a positive integer multiple of the pulse width). In this way, the brightness of the pixels in the output image is also the same, and no stripe phenomenon will occur.

[0058] However, when the exposure time of the photoelectric sensor is not a positive integer multiple of the pulse period of the PWM signal, the time duration of the light received by the photoelectric sensor is not exactly the same, which will cause a stripe phenomenon.

[0059] Figure 4 This is another schematic diagram of the shooting process.

[0060] like Figure 4 As shown, the photoelectric sensors in the sensor array shoot the display object displayed in the display using PWM dimming technology by exposure line by line. When the PWM signal is at a high level, the display element (such as LED) in the display screen is lit; when the PWM signal is at a low level, the display element is extinguished. Therefore, when the exposure time of the photoelectric sensor is different from the pulse period of the PWM signal, the time when some photoelectric sensors receive light is not exactly the same as the time when the display element is lit, and different photoelectric sensors receive light for different times.

[0061] exist Figure 4In the shooting process shown, the exposure time of the photosensor is shorter than the pulse period of the PWM signal. The time duration of the photosensor for acquiring the light signal at position 1 (hereinafter referred to as photosensor 1) is shorter than the time duration of the photosensor for acquiring the light signal at position 2 (hereinafter referred to as photosensor 2). Therefore, the intensity of the electrical signal output by photosensor 1 is smaller than the intensity of the electrical signal output by photosensor 2. In this case, in the shooting result, the brightness of the pixel corresponding to position 1 is smaller than the brightness of the pixel corresponding to position 2. In this way, stripes will appear in the shooting result.

[0062] Although setting the exposure time to a positive integer multiple of the pulse period of the PWM signal can solve the problem of stripes, there may still be problems in practical applications. For example, setting the exposure time to an integer multiple of the pulse period of the PWM signal limits the frame rate of the image. The duration of a frame is positively correlated with the exposure time. The longer the exposure time, the longer the duration of a frame. The duration of a frame is the reciprocal of the frame rate. For example, if the frame rate is 15 frames per second (FPS), the duration of a frame is 66.7 milliseconds (ms). It can be seen that increasing the exposure time limits the shooting frame rate. In addition, in some scenes, an object may be displayed by multiple display screens or illuminated by multiple different light sources. For example, a large-screen TV wall is usually composed of multiple display screens, and the pulse periods of PWM signals of different display screens may not be the same. Therefore, when taking pictures, it cannot be guaranteed that the exposure time and the pulse period of the PWM signal are positive integer multiples.

[0063] An image processing method according to an embodiment of the present application can reduce or even avoid the occurrence of stripe phenomenon without adjusting the exposure time to a positive integer multiple of the pulse period of a PWM signal.

[0064] The following first introduces an electronic device that supports the method for processing images provided in an embodiment of the present application.

[0065] Figure 5 is a schematic deconstruction block diagram of an electronic device provided according to an embodiment of the present application. Figure 5 The electronic device 500 shown includes a camera system 510 and a processor 520. The camera system 510 includes an event-based vision sensor (EVS) 511 and a complementary metal oxide semiconductor image sensor (CIS) 512. Figure 5The electronic device 500 shown can be any electronic device with a photographic system. For example, the electronic device 500 can be a mobile terminal (e.g., a mobile phone, a tablet computer, a laptop computer, etc.), a photographic device (e.g., a digital camera, a digital video camera, a sports camera, etc.), etc., or other electronic devices with a photographic system (e.g., a surveillance camera, a traffic camera, etc.). The photographic system 510 can be a built-in photographic system of the electronic device 500, or a photographic system connected to the electronic device 500 by wire or wireless means. For example, in some embodiments, the electronic device 500 can be a laptop computer or a desktop computer, the photographic system 510 can be an external camera connected to the laptop computer or desktop computer via a wired transmission medium, and the processor 520 is a processor in the laptop computer or desktop computer. Processor 520 may be a chip, for example, a central processor unit (CPU), an image signal processor (ISP), a graphic processing unit (GPU), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a digital signal processor (DSP), a micro controller unit (MCU), a programmable logic device (PLD), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or other integrated chips. Processor 520 may also be a system on chip (SoC).

[0066] It is understandable that the electronic device 500 may further include a memory (not shown in the figure), which may be used to store the image output by the processor. The memory may also be used to store instructions for implementing the method of the technical solution of the present application.

[0067] The EVS 511 can detect brightness change events within the shooting range, determine the location of the brightness change event, the time of the brightness change event, and the type of the brightness change event. The EVS 511 can send the location of the brightness change event, the time of the brightness change event, and the type of the brightness change event to the processor 520.

[0068] The CIS 512 may acquire an optical signal within a shooting range, perform photoelectric conversion, obtain an electrical signal corresponding to the optical signal, and send the electrical signal to the processor 520 .

[0069] The processor 520 may compensate the signal from the CIS 512 according to the brightness event information from the EVS 511 and output a compensated output signal.

[0070] The embodiments of the present application support photographic systems with different architectures. Figure 6 is a schematic deconstruction block diagram of a photographic system provided according to an embodiment of the present application. Figure 6 The camera system 600 shown can be applied to Figure 5 In the electronic device 500 shown. In other words, Figure 5 The camera system 510 shown may be Figure 6 The photography system 600 is shown implemented.

[0071] like Figure 6 The photographic system 600 shown includes: an EVS 610 , a CIS 620 , a lens module 630 and a lens module 640 .

[0072] The lens module 630 may include one or more lenses. Similarly, the lens module 640 may also include one or more lenses.

[0073] In some embodiments, the lens module 630 and / or the lens module 640 may further include other elements, such as a prism, a variable aperture assembly, etc.

[0074] like Figure 6 As shown, the lens module 630 corresponds to the EVS 610. The EVS 610 detects brightness change events within the shooting range through the lens module 630. The lens module 640 corresponds to the CIS 620. The CIS 620 obtains light signals within the shooting range through the lens module 640.

[0075] Figure 7 is a schematic deconstruction block diagram of another photographic system provided according to an embodiment of the present application. Figure 7 The camera system 700 shown can be applied to Figure 5 In the electronic device 500 shown. In other words, Figure 5 The camera system 510 shown may be composed of Figure 7 The photography system 700 is shown implemented.

[0076] like Figure 7 The photographic system 700 shown includes: an EVS 710 , a CIS 720 , a lens module 730 and a beam splitter 740 .

[0077] The lens module 730 may include one lens or multiple lenses, which is not limited in the present embodiment. In some embodiments, the lens module 730 may also include other components, such as a prism, a variable aperture assembly, etc.

[0078] like Figure 7 As shown, the beam splitter 740 obtains the incident light 750 from the lens module 730, and then divides the incident light 750 into an outgoing light 751 and an outgoing light 752. The EVS 710 detects brightness change events within the shooting range through the outgoing light 751, and the CIS 720 obtains the light signal within the shooting range through the outgoing light 752.

[0079] Figure 8 is a schematic deconstruction block diagram of another photographic system provided according to an embodiment of the present application. Figure 8 The camera system 800 shown can be applied to Figure 5 In the electronic device 500 shown. In other words, Figure 5 The camera system 510 shown may be composed of Figure 8 The photography system 800 is shown implemented.

[0080] like Figure 8 The photographic system 800 shown includes: an EVS 810 , a CIS 820 and a lens module 830 .

[0081] The lens module 830 may include one lens or multiple lenses, which is not limited in the present embodiment. In some embodiments, the lens module 830 may also include other elements, such as a prism, a variable aperture assembly, etc.

[0082] like Figure 8 As shown, EVS 810 is located in front of CIS 820. In this way, EVS 810 and CIS 820 can share lens module 830. In other words, EVS 810 can detect brightness change events in the shooting area through lens module 830, and CIS 820 can also obtain light signals in the shooting range through lens module 830.

[0083] Fig. 9 is a schematic deconstruction block diagram of another photographic system provided according to an embodiment of the present application. Fig. 9 The photographic system 900 shown in (a) can be applied to Figure 5 In the electronic device 500 shown. In other words, Figure 5 The camera system 510 shown may be composed of Fig. 9 The photographic system 900 shown in (a) is implemented.

[0084] like Fig. 9The photographic system 900 shown in (a) includes: an EVS 910, a CIS 920 and a lens module 930.

[0085] The lens module 930 may include one lens or multiple lenses, which is not limited in the present embodiment. In some embodiments, the lens module 930 may also include other elements, such as a prism, a variable aperture assembly, and the like.

[0086] The CIS 920 may include a plurality of EVS areas. The EVS 910 may include a plurality of EVS modules. The plurality of EVS modules are respectively located in the plurality of EVS areas.

[0087] Fig. 9 (b) in FIG. 9 shows two EVS areas in the CIS 920 , namely, an EVS area 921 and an EVS area 922 . Fig. 9 (b) in FIG. 9 also shows two EVS modules in EVS 910, namely, EVS module 911 and EVS module 912. Fig. 9 As shown in (b) in FIG. 8 , the EVS module 911 is located in an EVS area 921 in the CIS 920 , and the EVS module 912 is located in an EVS area 922 in the CIS 920 .

[0088] In this way, the EVS 910 and the CIS 920 can share the lens module 930. In other words, the EVS 910 can detect brightness change events in the shooting area through the lens module 930, and the CIS 920 can also obtain light signals in the shooting range through the lens module 930.

[0089] Fig.10 FIG. 1 is a schematic flow chart of a method for processing an image according to an embodiment of the present application. Fig.10 The method shown can be implemented by an electronic device (e.g. Figure 5 An electronic device 500) or a component in an electronic device (eg Figure 5 For ease of description, it is assumed in the following embodiments that Fig.10 The method shown is implemented by a processor in an electronic device.

[0090] Combine the following Fig.11 right Fig.10 The method shown is introduced.

[0091] Fig.11 The shooting area of ​​EVS is shown. The shooting area of ​​EVS can be divided into multiple locations. Fig.11 The shooting area shown includes 36 positions in total. Fig.11The position corresponding to the coordinates (x, y) shown is called position xy. For example, the position corresponding to the coordinates (0, 0) can be called position 00, the position corresponding to the coordinates (0, 1) can be called position 01, and so on.

[0092] In some embodiments, a position in the EVS shooting area may correspond to a sensor in the CIS. In other embodiments, a position in the EVS shooting area may correspond to multiple sensors in the CIS. For ease of description, one or more sensors corresponding to a position in the EVS shooting area may be referred to as a sensor unit. In other words, a position in the EVS shooting area corresponds to a sensor unit in the CIS, which may include one or more sensors. The sensor can convert the acquired light signal into an electrical signal, so the sensor can also be called a photoelectric sensor.

[0093] 1001. The processor obtains brightness event information of a position in the EVS shooting area.

[0094] Taking position 00 as an example, EVS can detect whether a brightness change event occurs at position 00. If EVS detects that a brightness change event occurs at position 00, then EVS can determine the brightness change event information of position 00 and send the brightness change event information of position 00 to the processor. The brightness change event information of position 00 is used to indicate the type of brightness change event that occurs at position 00 and the time when the functional brightness change event occurs. The type of brightness change event includes brightness increase or brightness decrease. For example, in some embodiments, the brightness change event information may include the following (x, y; t; TYP), where x, y represent the coordinates of the brightness change event, t represents the time when the brightness change event occurs, and TYP represents the type of brightness change event. For example, if x=0 and y=0 in the brightness change event information, the brightness change event information is the brightness change event information for position 00. t can be an absolute moment or a relative moment. The absolute moment is the current moment of the internal clock of the electronic device when the brightness change event occurs. The relative moment is the moment relative to a reference moment when the brightness change event occurs. For example, assuming that the moment t0 when the shutter is pressed is taken as the reference moment, then t means that a brightness change event occurs at moment t starting from t0. TYP can indicate the type of brightness change event through one or more bits. For example, assuming that the length of TYP is 1 bit, if the value of TYP is 0, then it means that the type of brightness change event that occurs is brightness increase; if the value of TYP is 1, then it means that the type of brightness change event that occurs is brightness decrease. For example, assuming that a brightness change event information is: (0,0; t1; 0), then it means that EVS detects that a brightness increase event occurs at position 00 at moment t1; assuming that a brightness change event information is: (0,0; t2; 1), then it means that EVS detects that a brightness decrease event occurs at position 00 at moment t2.

[0095] 1002. The processor determines, based on the acquired brightness change event information, whether the moment when the brightness change event occurs is within the range of the exposure time window of the sensor unit corresponding to the position.

[0096] If it is determined that the time when the brightness change event occurs is within the exposure time window of the sensor unit, step 1003 is executed; if the time when the brightness change event occurs is outside the exposure time window of the sensor unit, step 1004 is executed.

[0097] 1003, determine compensation parameters.

[0098] 1004, skip the calculation of compensation parameters.

[0099] 1005 , determining whether all brightness change events within the exposure time of the image capture have been checked to determine whether corresponding compensation parameters need to be determined.

[0100] If the judgment result is no (i.e., there is still unchecked brightness change event information within the exposure time of the image capture), then continue to obtain the next brightness change event information, and determine whether it is necessary to determine the corresponding brightness compensation parameters based on the acquired brightness change event information. If the judgment result is yes (i.e., all brightness change event information within the exposure time of the image capture has been checked), then the processing flow ends.

[0101] Fig.10 The overall process of the image processing method provided by the embodiment of the present application is shown. Fig.12 The specific process of determining the compensation parameters corresponding to a certain position is introduced.

[0102] Fig.12 is a schematic flow chart of a method for processing an image according to an embodiment of the present application. Fig.10 Similar to the method shown, Fig.12 The method shown may be executed by an electronic device, or may be executed by a component (such as a processor) in the electronic device.

[0103] 1201, obtain at least one brightness event information of a target position detected by the EVS.

[0104] by Fig.11 Taking the shooting area shown in the figure as an example, the target position can be as follows Fig.11 Any of the 36 positions shown.

[0105] Each brightness event information in the at least one brightness event information is used to indicate the type of brightness change event occurring at the target position and the time when the brightness change event occurs. The type of the brightness change event includes brightness increase or brightness decrease.

[0106] 1202. Determine a compensation parameter according to the at least one brightness event information and an exposure time window of a target sensor unit.

[0107] The target sensor unit may include one or more sensors. The target sensor unit is a sensor unit corresponding to the target position in the CIS. The target sensor unit is used to obtain an optical signal at the target position and convert the obtained optical signal into an electrical signal.

[0108] 1203. Acquire a first output signal corresponding to the target sensor unit.

[0109] 1204. Compensate the first output signal according to the compensation parameter to obtain a second output signal.

[0110] Fig.12 The method shown can compensate the output signal of the photoelectric sensor to obtain a compensated output signal. By compensating the output signal, the brightness value corresponding to the output signal can be increased, which can reduce or even avoid the occurrence of stripe phenomenon.

[0111] Optionally, in some embodiments, determining the compensation parameter based on the at least one brightness event information and the exposure time of the target sensor unit may include: determining a reference time window based on the at least one brightness event information and the exposure time window, wherein the brightness value of the target position within the reference time window is different from the brightness value outside the reference time window; and determining the compensation parameter based on the reference time window and the exposure time window.

[0112] For the convenience of description, in the following examples, T exp The exposure time window of the target sensor unit is represented by tr, and the start time of the exposure time window is represented by ts. exp , tr and ts satisfy the following relationship:

[0113] T exp =tr-ts, (Formula 1.1)

[0114] Optionally, in some embodiments, ts may also be referred to as a shutter time, and tr may also be referred to as a read out time.

[0115] For the sake of convenience of description, if the type of brightness change event indicated by a brightness event information is an increase in brightness, then the moment indicated by the brightness event information can be represented by ton; if the type of brightness change event indicated by a brightness event information is a decrease in brightness, then the moment indicated by the brightness event information can be represented by toff.

[0116] For the convenience of description, if the moment of occurrence of a brightness change event indicated by a brightness event information is within the range of the exposure time window (that is, the moment of occurrence of the brightness change event indicated by the brightness event information is not earlier than the start moment of the exposure time window and not later than the end moment of the exposure time window), then the brightness event information can be called reference brightness event information. The moment indicated by the reference brightness event information can be called the reference moment.

[0117] In some embodiments, all brightness event information in the at least one brightness event information may be the reference brightness event information. In other embodiments, only part of the at least one brightness event information may be the reference brightness event information. For ease of description, it is assumed below that the at least one brightness event information includes N reference brightness event information. It is understood that N can be a positive integer greater than or equal to 1 and less than the total number of brightness event information of the target position.

[0118] In some embodiments, the value of N may be 1. In other words, there is only one reference brightness event information in the at least one brightness event information. If the type of brightness change event indicated by the reference brightness event information is brightness increase, then the starting time of the reference time window is ton and the ending time is tr. Assuming that T ref represents the length of the reference time window, then T ref , ton and tr satisfy the following relationship:

[0119] T ref =tr-ton, (Formula 1.2).

[0120] It can be understood that, if ton=ts, then the duration of the reference time window is the same as the duration of the exposure time window; if ton=tr, then the duration of the reference time window is equal to 0.

[0121] In some other embodiments, there is only one reference brightness event information in the at least one brightness event information, and the type of brightness change event indicated by the reference brightness event information is brightness reduction, then the starting time of the reference time window is ts, and the ending time is toff. In this case, T ref , ts and toff satisfy the following relationship:

[0122] T ref =toff-ts, (Formula 1.3).

[0123] It can be understood that, if toff=tr, the duration of the reference time window is the same as the duration of the exposure time window; if toff=ts, the duration of the reference time window is equal to 0.

[0124] In some other embodiments, the value of N may be 2. In other words, the at least one luminance event information includes two reference luminance event information. For ease of description, the following will refer to these two reference luminance event information as reference luminance event information 1 and reference luminance event information 2 respectively. The moment indicated by reference luminance event information 1 is referred to as reference moment 1, the type of the luminance change event indicated by reference luminance event information 1 is referred to as reference type 1, the moment indicated by reference luminance event information 2 is reference moment 2, and the type of the luminance change event indicated by reference luminance event information 2 is referred to as reference type 2.

[0125] In some embodiments, the reference type 1 is an increase in luminance, the reference type 2 is a decrease in luminance, and the reference moment 1, the reference moment 2, the start moment of the exposure time window, and the end moment of the exposure time window satisfy the following relationship: ts≤t1<t2≤tr, where ts is the start moment of the exposure time window, t1 is the reference moment 1, t2 is the reference moment 2, and tr is the end moment of the exposure time window. Then, Tref, t1, and t2 satisfy the following relationship:

[0126] Tref = t2 - t1, (Equation 1.4).

[0127] In some other embodiments, the reference type 1 is a decrease in luminance, the reference type 2 is an increase in luminance, and the reference moment 1, the reference moment 2, the start moment of the exposure time window, and the end moment of the exposure time window satisfy the following relationship: ts≤t1<t2≤tr, where ts is the start moment of the exposure time window, t1 is the reference moment 1, t2 is the reference moment 2, and tr is the end moment of the exposure time window. Then, Tref, t1, t2, tr, and ts satisfy the following relationship:

[0128] Tref = (t1 - ts) + (tr - t2), (Equation 1.5).

[0129] In some embodiments, if N is a positive integer greater than or equal to 2, and the types of the luminance events indicated by the N reference luminance event information are the same, then the duration of the reference time window can be calculated in the manner of Equation 1.2 or Equation 1.3, where ton in Equation 1.2 is the moment indicated by the first reference luminance event information among the N reference luminance event information, and toff in Equation 1.3 is the moment indicated by the last reference luminance event information among the N reference luminance event information. The so-called first reference luminance event information refers to the reference luminance event information with the earliest indicated moment among the N reference luminance event information. Correspondingly, the last reference luminance event information refers to the reference luminance event information with the latest indicated moment among the N reference luminance event information.

[0130] For example, assume that N equals 3, and the types of the brightness change events indicated by the 3 reference brightness event information are all brightness increases. The times when the brightness change events occur indicated by the 3 reference brightness event information are t1, t2, and t3 respectively, and ts, t1, t2, t3, and tr satisfy the following relationship: ts < t1 < t2 < t3 < tr. Then the duration of the reference time window can satisfy the following relationship:

[0131] Tref = tr - t1, (Equation 1.6).

[0132] Another example, assume that N equals 3, and the types of the brightness change events indicated by the 3 reference brightness event information are all brightness decreases. The times when the brightness change events occur indicated by the 3 reference brightness event information are t1, t2, and t3 respectively, and ts, t1, t2, t3, and tr satisfy the following relationship: ts < t1 < t2 < t3 < tr. Then the duration of the reference time window can satisfy the following relationship:

[0133] T ref = t3 - ts, (Equation 1.7).

[0134] In some embodiments, if N is greater than 3 and the types of the brightness change events indicated by the N reference brightness event information include brightness increases and brightness decreases, then the difference between the time indicated by the last reference brightness event information and the time indicated by the first reference brightness event information can be used as the duration of the reference time window. For example, assume that N equals 3, and the types of the brightness change events indicated by the 3 reference brightness event information include two brightness increases and one brightness decrease. The times when the brightness change events occur indicated by the 3 reference brightness event information are t1, t2, and t3 respectively, and ts, t1, t2, t3, and tr satisfy the following relationship: ts < t1 < t2 < t3 < tr. Then the duration of the reference time window can satisfy the following relationship:

[0135] T ref = t3 - t1, (Equation 1.8).

[0136] After determining the reference time window, the compensation parameter can be determined according to the reference time window and the exposure time window.

[0137] In some embodiments, the reference time window, the exposure time window, and the compensation parameter satisfy the following relationship:

[0138] F = T ref / T exp , (Equation 1.9)

[0139] where F represents the compensation parameter, T ref represents the duration of the reference time window, T expIndicates the duration of the exposure time window.

[0140] Optionally, in some other embodiments, if N is a positive integer greater than or equal to 3, then the target sensor unit may not be compensated. Or, in other words, in this case, the compensation parameter may be set to 100%.

[0141] Optionally, in some other embodiments, if N is a positive integer greater than or equal to 2, then the target sensor unit may not be compensated. Or, in other words, in this case, the compensation parameter may be set to 100%.

[0142] Optionally, in some embodiments, if there is only one brightness change event before the start time of the exposure time window, and the type of the brightness change event is brightness increase, the compensation parameter may be set to 100%.

[0143] Optionally, in some embodiments, if there is only one brightness change event before the start time of the exposure time window, and the type of the brightness change event is brightness reduction, the compensation parameter may be set to 0%.

[0144] After the compensation parameters are obtained, the first output signal corresponding to the target sensor unit may be compensated using the compensation parameters to obtain a second output signal.

[0145] The target sensor unit performs photoelectric conversion on the acquired light signal to obtain the corresponding electrical signal. The electrical signal is converted into a digital signal after analog to digital (AD) conversion. The processor processes the digital signal to obtain the pixel value of the pixel that can form the picture.

[0146] Fig.13 A schematic diagram showing the signals corresponding to the target sensor.

[0147] like Fig.13 As shown, the target sensor unit acquires a first light signal at the target position. The first light signal is converted into a first electrical signal by the target sensor unit. The first electrical signal is subjected to AD conversion to obtain a first digital signal. The processor processes the digital signal to obtain a pixel value of a pixel in the output image.

[0148] In some embodiments, the processor may first compensate the first digital signal to obtain a second digital signal, and then perform a series of processing (e.g., color correction, gamma correction, etc.) on the second digital signal to obtain the pixel value in the output image. In other words, in this case, the first output signal corresponding to the target sensor unit is the first digital signal, and the second output signal is the second digital signal.

[0149] In other embodiments, the processor may first perform a series of processing (e.g., color correction, gamma correction, etc.) on the first digital signal to obtain a first pixel value, and then compensate the first pixel value using the compensation parameter to obtain a second pixel value. In this case, the first output signal corresponding to the target sensor unit is the first pixel value, and the second output signal is the second pixel value.

[0150] In other embodiments, the compensation parameter can also be used to compensate the first electrical signal obtained after photoelectric conversion. In other words, the first electrical signal is compensated by the compensation parameter to obtain a second electrical signal, and then the second electrical signal is AD converted and subsequently processed to obtain a pixel value of a pixel in the output image. In this case, the first output signal corresponding to the target sensor unit is the first electrical signal, and the second output signal is the second electrical signal.

[0151] In some embodiments, the compensation parameter, the first output signal, and the second output signal satisfy the following relationship:

[0152] P'=P / F, (Formula 2.1)

[0153] Wherein P' is the second output signal, P is the first output signal, and F is the compensation parameter.

[0154] In some embodiments, if the compensation parameter is 0, the value of P' may be a preset value or a value based on an output signal at another position. For example, the value of P' may be an output signal (or an output signal after compensation) of a sensor unit corresponding to an adjacent position of the target position or an average value of output signals (or output signals after compensation) of multiple adjacent unknown sensor units, wherein the adjacent position of the target position may be one of the upper, lower, left, right, upper left, upper right, lower left, or ranger of the target position.

[0155] In some other embodiments, the compensation parameter, the first output signal and the second output signal satisfy the following relationship:

[0156] P'=P / (F+c), (Formula 2.1)

[0157] Wherein P' is the second output signal, P is the first output signal, F is the compensation parameter, and c is a correction parameter. The correction parameter can be greater than 0 and less than 1. In this way, when F is equal to 0, the second output signal can also be determined according to the first output signal. The correction parameter can be a preset constant or a parameter value calculated according to the compensation parameters of adjacent positions.

[0158] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in a processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0159] It should be noted that the processor in the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to be executed, or a combination of hardware and software modules in the decoding processor to be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0160] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0161] According to the method provided in the embodiments of the present application, the present application also provides a computer program product, which includes: a computer program code, when the computer program code is run on a computer, the computer executes each step in the above embodiments.

[0162] According to the method provided in the embodiments of the present application, the present application also provides a computer-readable medium, which stores a program code. When the program code runs on a computer, the computer executes each step in the above embodiments.

[0163] According to the method provided in an embodiment of the present application, an embodiment of the present application provides a chip system, which includes a logic circuit, which is used to couple with an input / output interface and transmit data through the input / output interface to execute the various steps in the above embodiments.

[0164] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0165] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0166] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0167] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0168] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0169] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0170] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for processing an image, characterized in that: include: Acquire at least one brightness event information of a target position detected by an event-based vision sensor EVS, wherein each brightness event information of the at least one brightness event information is used to indicate the type of brightness change event occurring at the target position and the time when the brightness change event occurs, and the type of the brightness change event includes brightness increase or brightness decrease; determining a compensation parameter according to the at least one brightness event information and an exposure time window of a target sensor unit, wherein the target sensor unit is a sensor unit in a complementary metal oxide semiconductor image sensor CIS for acquiring an optical signal at the target position; Acquire a first output signal corresponding to the target sensor unit; The first output signal is compensated according to the compensation parameter to obtain a second output signal.

2. The method according to claim 1, characterized in that The determining of the compensation parameter according to the at least one brightness event information and the exposure time window of the target sensor unit comprises: Determine a reference time window according to the at least one brightness event information and the exposure time window, wherein a brightness value of the target position within the reference time window is different from a brightness value outside the reference time window; The compensation parameter is determined according to the reference time window and the exposure time window.

3. The method according to claim 2, characterized in that The determining a reference time window according to the at least one brightness event information and the exposure time window comprises: According to the exposure time window and the moment indicated by the at least one brightness event information, determine N reference brightness event information from the at least one brightness event information, wherein any one of the N reference moments is within the range of the exposure time window, the N reference moments are respectively moments indicated by the N reference brightness event information, and N is a positive integer greater than or equal to 1; The reference time window is determined according to the N reference brightness event information and the exposure time window.

4. The method according to claim 3, characterized in that When N is equal to 1, determining the reference time window according to the N reference brightness event information and the exposure time window includes: In a case where the type of the brightness change event indicated by the N reference brightness event information is the brightness reduction, determining that the starting moment of the reference time window is the first moment, the ending moment of the reference time window is the ending moment of the exposure time window, and the first moment is the moment indicated by the N reference brightness event information; When the type of brightness change event indicated by the N reference brightness event information is the brightness increase, the starting time of the reference time window is determined to be the starting time of the exposure time window, and the ending time of the reference time window is determined to be the first time.

5. The method according to claim 3, characterized in that: When N is equal to 2, determining the reference time window according to the N reference brightness event information and the exposure time window includes: Determine second brightness event information and third brightness event information from the N reference brightness event information, wherein the type of brightness change event indicated by the second brightness event information is the brightness rise, the type of brightness change event indicated by the third brightness event information is the brightness drop, the second moment is earlier than the third moment, the second moment is the moment indicated by the second brightness event information, and the third moment is the moment indicated by the third brightness event information; The starting time of the reference time window is determined to be the second time, and the ending time of the reference time window is determined to be the third time.

6. The method according to any one of claims 2 to 5, characterized in that The reference time window, the exposure time window and the compensation parameter satisfy the following relationship: F=T ref / T exp , Wherein, F is the compensation parameter, T ref is the duration of the reference time window, T exp is the duration of the exposure time window.

7. The method according to any one of claims 1 to 6, characterized in that The compensation parameter, the first output signal and the second output signal satisfy the following relationship: P'=P / (F+c), Wherein P' is the second output signal, P is the first output signal, F is the compensation parameter, and c is the correction parameter.

8. An electronic device, characterized in that: The electronic device comprises a photographic system and a processor, wherein the photographic system comprises an event-based visual sensor EVS and a complementary metal oxide semiconductor image sensor CIS, wherein The EVS detects a brightness change at a target position, and determines at least one brightness event information at the target position, wherein each brightness event information in the at least one brightness event information is used to indicate a brightness change event occurring at the target position and a time when the brightness change event occurs, and the brightness change event includes a brightness increase or a brightness decrease; The processor is used to obtain the at least one brightness event information from the EVS, and determine a compensation parameter according to the at least one brightness event information and an exposure time window of a target sensor unit, wherein the target sensor unit is a sensor unit in the CIS for obtaining a light signal at the target position; The processor is further configured to obtain a first output signal corresponding to the target sensor unit, and compensate the first output signal according to the compensation parameter to obtain a second output signal.

9. The electronic device according to claim 8, characterized in that: The processor is specifically used for: Determine a reference time window according to the at least one brightness event information and the exposure time window, wherein a brightness value of the target position within the reference time window is different from a brightness value outside the reference time window; The compensation parameter is determined according to the reference time window and the exposure time window.

10. The electronic device according to claim 9, characterized in that: The processor is specifically used for: According to the exposure time window and the moment indicated by the at least one brightness event information, determine N reference brightness event information from the at least one brightness event information, wherein any reference moment of the N reference moments is later than the start moment of the exposure time window and earlier than the end moment of the exposure time window, and the N reference moments are respectively the moments indicated by the N reference brightness event information, and N is a positive integer greater than or equal to 1; The reference time window is determined according to the N reference brightness event information and the exposure time window.

11. The electronic device according to claim 10, characterized in that: The processor is specifically used for: In a case where it is determined that N is equal to 1 and the type of the brightness change event indicated by the N reference brightness event information is the brightness reduction, determining that the starting moment of the reference time window is the first moment, the ending moment of the reference time window is the ending moment of the exposure time window, and the first moment is the moment indicated by the N reference brightness event information; When it is determined that N is equal to 1 and the type of brightness change event indicated by the N reference brightness event information is the brightness increase, the starting time of the reference time window is determined to be the starting time of the exposure time window, and the end time of the reference time window is determined to be the first time.

12. The electronic device according to claim 10, characterized in that: The processor is specifically used for: When it is determined that N is equal to 2, second brightness event information and third brightness event information are determined from the N reference brightness event information, wherein the type of brightness change event indicated by the second brightness event information is the brightness rise, the type of brightness change event indicated by the third brightness event information is the brightness drop, the second moment is earlier than the third moment, the second moment is the moment indicated by the second brightness event information, and the third moment is the moment indicated by the third brightness event information; The starting time of the reference time window is determined to be the second time, and the ending time of the reference time window is determined to be the third time.

13. The electronic device according to any one of claims 9 to 12, characterized in that: The reference time window, the exposure time window and the compensation parameter satisfy the following relationship: F=T ref / T exp , Wherein, F is the compensation parameter, T ref is the duration of the reference time window, T exp is the duration of the exposure time window.

14. The electronic device according to any one of claims 8 to 13, characterized in that: The compensation parameter, the first output signal and the second output signal satisfy the following relationship: P'=P / (F+c), Wherein P' is the second output signal, P is the first output signal, F is the compensation parameter, and c is the correction parameter.

15. The electronic device according to any one of claims 8 to 14, characterized in that: The photographic system further includes a first lens module and a second lens module, wherein: The EVS detects brightness changes at the target position through the first lens module; The CIS acquires the optical signal of the target position through the second lens module.

16. The electronic device according to any one of claims 8 to 14, characterized in that: The photographic system further includes a third lens module and a beam splitter, wherein the beam splitter obtains incident light from the third lens module and splits the incident light into a first outgoing light and a second outgoing light. The EVS detects a brightness change of the target position through the first emitted light; The CIS detects brightness changes at the target position through the second emitted light.

17. The electronic device according to any one of claims 8 to 14, characterized in that: The photographic system further includes a fourth lens module, wherein the CIS includes a plurality of EVS areas, the EVS includes a plurality of EVS modules, and the plurality of EVS modules are respectively located in the plurality of EVS areas; The EVS detects brightness changes at the target position through the fourth lens module; The CIS detects the photoelectric signal of the target position through the fourth lens module.

18. The electronic device according to any one of claims 8 to 14, characterized in that: The photographic system further comprises a fifth lens module, wherein the EVS is located in front of the CIS. The EVS detects brightness changes at the target position through the fifth lens module; The CIS detects the photoelectric signal of the target position through the fifth lens module.

19. A chip system, characterized in that: include: A logic circuit, the logic circuit is used to be coupled to an input / output interface, and transmit data through the input / output interface to execute the method according to any one of claims 1 to 8.

20. A computer readable medium, characterized in that The computer-readable medium stores a computer program code, and when the computer program code is executed on a computer, the computer is caused to perform the method according to any one of claims 1 to 8.

Images