Imaging method, system, apparatus, event camera, and storage medium of an event camera

Abstract

The application relates to an imaging method, system and device of an event camera, the event camera and a storage medium. The method comprises the following steps: in the case that it is determined that there is a relatively static object in the sensing range of the event camera, changing the preset light transmittance of a variable tint glass; the variable tint glass is arranged in front of a camera sensor of the event camera; acquiring event stream data captured by the camera sensor under multiple different light transmittances; obtaining event frames of the camera sensor under the various light transmittances based on the event stream data; and fusing the multiple event frames under the multiple different light transmittances to generate an environment perception image of the event camera, wherein the environment perception image contains an image of the relatively static object. By changing the light transmittance of the variable tint glass, the image containing the relatively static object can be formed, so that the completeness of the image generated by the event camera is improved.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to an imaging method, system, device, event camera, storage medium, and computer program product for an event camera. Background Technology

[0002] With the development of computer vision technology, a technology has emerged that uses event cameras as intelligent sensors. Event cameras are characterized by low power consumption, high frequency, and high dynamic range, thus having a wide range of applications. Their main principle is to capture changes in light intensity at the location of a target object, converting this into voltage changes at the sensing location, thereby triggering a pulse event signal at the corresponding pixel location. Therefore, in practical applications, event cameras primarily capture targets in a state of relative motion. If there is relative motion between the target object and the event camera's sensor, the event camera sensor will detect changes in light, generate an event signal, and thus produce a corresponding image.

[0003] However, in current event camera imaging methods, if the event camera moves but remains relatively stationary with respect to the target object, such as when the event camera and the target object move in the same direction and at the same speed, the event camera cannot generate a signal or corresponding image because the position and light intensity perceived by the target object on the event camera sensor do not change. Therefore, the image generated by existing event camera imaging methods has low integrity. Summary of the Invention

[0004] Therefore, it is necessary to provide an imaging method, system, device, event camera, computer-readable storage medium, and computer program product for an event camera that can improve the integrity of generated images, in order to address the above-mentioned technical problems.

[0005] In a first aspect, this application provides an imaging method for an event camera, the method comprising:

[0006] If it is determined that there is a relatively stationary object within the sensing range of the event camera, the transmittance of the preset photochromic glass is changed; the photochromic glass is placed in front of the camera sensor of the event camera.

[0007] Acquire event stream data captured by the camera sensor under various different transmittance conditions;

[0008] Based on the event stream data, the event frames of the camera sensor under various transmittance conditions are obtained;

[0009] By fusing multiple event frames under various different transmittances, an environmental perception image of the event camera is generated, which includes images of the relatively stationary objects.

[0010] In one embodiment, changing the transmittance of the preset photochromic glass includes: acquiring a plurality of preset input signals of different frequencies, and inputting the plurality of input signals of different frequencies to a preset change control unit; wherein the change control unit is signal-connected to the photochromic glass, and the plurality of input signals of different frequencies are used to trigger the change control unit to generate a plurality of corresponding control signals, and to use the control signals to change the transmittance of the photochromic glass.

[0011] In one embodiment, the method further includes: when the event camera is in an initial state, initializing the transmittance of the photochromic glass to the maximum transmittance within an adjustable range.

[0012] In one embodiment, fusing multiple event frames with different transmittances to generate an environmental perception image of the event camera includes: performing Gaussian filtering on the event frames corresponding to each transmittance to obtain denoised event frames; and performing weighted averaging on the denoised event frames to obtain the environmental perception image of the event camera.

[0013] In one embodiment, performing Gaussian filtering on the event frames corresponding to each of the transmittances to obtain denoised event frames includes: obtaining a current event frame and the transmittance corresponding to the current event frame; the current event frame is any one of the event frames corresponding to each of the transmittances; obtaining Gaussian filtering parameters corresponding to the current event frame based on the current event frame and the transmittance; and performing Gaussian filtering on the current event frame based on the Gaussian filtering parameters to obtain denoised event frames corresponding to the current event frame.

[0014] In one embodiment, before changing the transmittance of the preset photochromic glass when it is determined that there is a relatively stationary object within the perception range of the event camera, the method further includes: acquiring a current event stream; the current event stream being the event stream collected by the camera sensor during the current time period; acquiring a historical event stream corresponding to the current event stream; the historical event stream being the event stream collected by the camera sensor during the previous time period of the current time period; comparing the current event stream with the historical event stream, and determining the perception result of the relatively stationary object based on the comparison result.

[0015] In one embodiment, the imaging region corresponding to the current event stream includes multiple first imaging region blocks; the imaging region corresponding to the historical event stream includes multiple second imaging region blocks corresponding to each of the first imaging region blocks; the step of comparing the current event stream and the historical event stream, and determining the perception result of the relatively stationary object based on the comparison result, includes: obtaining a first event contained in the current first imaging region block, and a second event contained in the current second imaging region block corresponding to the current first imaging region block; the current first imaging region block is any one of the multiple first imaging region blocks; comparing the first event and the second event, and when the number of event differences between the first event and the second event is greater than a preset difference threshold, the current first imaging region block is taken as the target first imaging region block; if there is at least one target first imaging region block in the imaging region corresponding to the current event stream, it is determined that there is a relatively stationary object in the current event stream.

[0016] In one embodiment, comparing the current event stream with the historical event stream includes: acquiring historical location data corresponding to the event camera; performing correction processing on the historical event stream based on the historical location data to obtain a corrected historical event stream; and comparing the current event stream with the corrected historical event stream.

[0017] Secondly, this application also provides an imaging system for an event camera, comprising: a perception computing unit, a camera sensor, and photochromic glass disposed in front of the camera sensor; wherein:

[0018] The sensing and computing unit is used to change the light transmittance of the photochromic glass when it is determined that there is a relatively stationary object within the sensing range of the event camera.

[0019] The camera sensor is used to capture event stream data corresponding to multiple different transmittances of the photochromic glass, and send the event stream data under multiple different transmittances to the sensing computing unit.

[0020] The perception computing unit is also used to obtain event frames of the camera sensor under various transmittances based on the event stream data under different transmittances, and to fuse multiple event frames under various different transmittances to generate an environmental perception image of the event camera, wherein the environmental perception image includes the image of the relatively stationary object.

[0021] In one embodiment, the system further includes: a change control unit connected to the photochromic glass signal; the sensing and computing unit is further configured to acquire a plurality of preset input signals of different frequencies when it is determined that there is a relatively stationary object within the sensing range of the event camera, and input the plurality of input signals of different frequencies to the preset change control unit; the change control unit is configured to generate control signals corresponding to the plurality of input signals of different frequencies respectively, and use the control signals to change the light transmittance of the photochromic glass.

[0022] In one embodiment, the perception computing unit includes: a memory and a comparison unit; the memory is used to store the current event stream and the historical event stream; the current event stream is the event stream collected by the camera sensor in the current time period; the historical event stream is the event stream collected by the camera sensor in the previous time period of the current time period; the comparison unit is used to compare the current event stream and the historical event stream, and determine the perception result of the relatively stationary object based on the comparison result.

[0023] In one embodiment, the sensing computing unit further includes: an odometer; the odometer is used to store historical location data corresponding to the event camera; the comparison unit is further used to perform correction processing on the historical event stream based on the historical location data to obtain a corrected historical event stream; and to compare the current event stream with the corrected historical event stream.

[0024] In one embodiment, the change control unit includes: a trigger and a numerical control unit; the trigger is used to receive the plurality of input signals of different frequencies and generate trigger signals corresponding to the plurality of input signals of different frequencies respectively, and input the plurality of trigger signals to the numerical control unit; the numerical control unit is used to receive the plurality of trigger signals and generate control signals corresponding to the plurality of trigger signals respectively, and use the control signals to change the light transmittance of the photochromic glass.

[0025] Thirdly, this application also provides an imaging device for an event camera, the device comprising:

[0026] A transmittance changing module is used to change the transmittance of a preset photochromic glass when it is determined that there is a relatively stationary object within the sensing range of the event camera; the photochromic glass is located in front of the camera sensor of the event camera.

[0027] The event data acquisition module is used to acquire event stream data captured by the camera sensor under various different transmittance conditions;

[0028] An event frame generation module is used to obtain event frames of the camera sensor under various transmittances based on the event stream data.

[0029] The perception image generation module is used to fuse multiple event frames under different transmittances to generate an environmental perception image of the event camera, wherein the environmental perception image contains images of the relatively stationary objects.

[0030] Fourthly, this application also provides an event camera, which includes a camera sensor and a perception computing unit. When a photochromic glass is provided in front of the camera sensor, the perception computing unit is used to perform the following steps:

[0031] If it is determined that there is a relatively stationary object within the sensing range of the event camera, the transmittance of the preset photochromic glass is changed; the photochromic glass is placed in front of the camera sensor of the event camera.

[0032] Acquire event stream data captured by the camera sensor under various different transmittance conditions;

[0033] Based on the event stream data, the event frames of the camera sensor under various transmittance conditions are obtained;

[0034] By fusing multiple event frames under various different transmittances, an environmental perception image of the event camera is generated, which includes images of the relatively stationary objects.

[0035] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0036] If it is determined that there is a relatively stationary object within the sensing range of the event camera, the transmittance of the preset photochromic glass is changed; the photochromic glass is placed in front of the camera sensor of the event camera.

[0037] Acquire event stream data captured by the camera sensor under various different transmittance conditions;

[0038] Based on the event stream data, the event frames of the camera sensor under various transmittance conditions are obtained;

[0039] By fusing multiple event frames under various different transmittances, an environmental perception image of the event camera is generated, which includes images of the relatively stationary objects.

[0040] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0041] If it is determined that there is a relatively stationary object within the sensing range of the event camera, the transmittance of the preset photochromic glass is changed; the photochromic glass is placed in front of the camera sensor of the event camera.

[0042] Acquire event stream data captured by the camera sensor under various different transmittance conditions;

[0043] Based on the event stream data, the event frames of the camera sensor under various transmittance conditions are obtained;

[0044] By fusing multiple event frames under various different transmittances, an environmental perception image of the event camera is generated, which includes images of the relatively stationary objects.

[0045] The aforementioned imaging method, system, apparatus, event camera, storage medium, and computer program product for an event camera, by changing the transmittance of a preset photochromic glass when a relatively stationary object is determined to exist within the sensing range of the event camera; the photochromic glass is placed in front of the camera sensor of the event camera; event stream data captured by the camera sensor at various transmittances is acquired; based on the event stream data, event frames of the camera sensor at various transmittances are obtained; and multiple event frames at various transmittances are fused to generate an environmental perception image of the event camera, which includes an image of a relatively stationary object. This application can change the transmittance of the photochromic glass placed in front of the camera sensor of the event camera when the event camera senses a relatively stationary object, thereby enabling the camera sensor to capture event frames at different transmittances. These event frames can then be fused to form a perceived image containing a relatively stationary object. By changing the transmittance of the photochromic glass, this application can form an image containing a relatively stationary object, thereby improving the completeness of the image generated by the event camera. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the system architecture of an event camera imaging method in one embodiment;

[0047] Figure 2 This is a flowchart illustrating the imaging method of an event camera in one embodiment;

[0048] Figure 3 This is a schematic diagram of the system architecture of an event camera imaging method in another embodiment;

[0049] Figure 4 This is a flowchart illustrating the process of obtaining a denoised event frame in one embodiment;

[0050] Figure 5This is a flowchart illustrating the process of determining the perception result of a relatively stationary object in one embodiment.

[0051] Figure 6 This is a flowchart illustrating the process of determining the perception result based on the comparison result in one embodiment;

[0052] Figure 7 This is a schematic diagram of the imaging system of an event camera in one embodiment;

[0053] Figure 8 This is a schematic diagram of the imaging system of an event camera in one embodiment;

[0054] Figure 9 This is a schematic diagram of the structure of a sensing computing unit in one embodiment;

[0055] Figure 10 This is a schematic diagram of the structure of the change control unit in one embodiment;

[0056] Figure 11 This is a schematic diagram of the framework of a method for imaging a relatively stationary target using an event camera in one embodiment;

[0057] Figure 12 This is a schematic diagram of the structure of a sensing computing unit in one embodiment;

[0058] Figure 13 This is a schematic diagram of the structure of the change control unit in one embodiment;

[0059] Figure 14 This is a flowchart illustrating the reconstruction algorithm in one embodiment;

[0060] Figure 15 This is a schematic diagram illustrating the principle of a relatively stationary target imaging method using an event camera in one embodiment;

[0061] Figure 16 This is a structural block diagram of the imaging device of an event camera in one embodiment;

[0062] Figure 17 This is a diagram of the internal structure of an event camera in one embodiment. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0064] The imaging method for an event camera provided in this application embodiment can be applied to, for example... Figure 1In the system architecture shown, the camera sensor communicates with the perception computing unit via signals. The perception computing unit can change the transmittance of the photochromic glass through control signals. Specifically, when the perception computing unit determines that there is a relatively stationary object within the sensing range of the event camera, it can change the transmittance of the photochromic glass placed in front of the camera sensor. Then, after ambient reflected light passes through the photochromic glass with different transmittances, the camera sensor can capture event stream data at different transmittances and transmit it to the perception computing unit. The perception computing unit can then obtain event frames at various transmittances based on the event stream data and fuse these event frames to obtain an environmental perception image containing the relatively stationary object. The photochromic glass can be electrochromic glass, and the camera sensor can be implemented using a dynamic vision sensor.

[0065] In one embodiment, such as Figure 2 As shown, an imaging method for an event camera is provided, which can be applied to... Figure 1 Taking the sensory computing unit in the image as an example, the following steps are included:

[0066] Step S201: If it is determined that there is a relatively stationary object within the sensing range of the event camera, the transmittance of the preset photochromic glass is changed; the photochromic glass is placed in front of the camera sensor of the event camera.

[0067] In this context, a relatively stationary object refers to an object within the sensing range of the event camera that is relatively stationary relative to the camera. This relatively stationary object can include the event camera's sensor, meaning either both the camera sensor and the object are stationary, or the camera sensor and the object are moving in the same direction at the same speed. In the case of relative stationarity, since the position and light intensity of the object on the camera sensor do not change, the event camera cannot image it. The preset photochromic glass refers to photochromic glass pre-placed in front of the camera sensor of the event camera. The transmittance of this photochromic glass can be altered by the sensing and computing unit. By changing the transmittance of the photochromic glass, the amount of light intensity attenuation sensed by the camera sensor at different pixel positions can be made different, thereby enabling the event camera to image relatively stationary objects. Specifically, when the sensing and computing unit determines that a relatively stationary object exists within the sensing range of the event camera, it can change the transmittance of the photochromic glass placed in front of the camera sensor to achieve image formation of the relatively stationary object.

[0068] Step S202: Acquire event stream data captured by the camera sensor under various different transmittance conditions;

[0069] Step S203: Based on the event stream data, obtain the event frames of the camera sensor under various transmittance conditions.

[0070] Event stream data can be event stream data captured by the camera sensor based on received ambient reflected light. The camera sensor can capture event data by receiving ambient reflected light to generate event stream data. Since the transmittance of the photochromic glass changes, the camera sensor can also capture event stream data under various transmittance levels. An event frame is an image frame generated from the event stream data. After event conversion, the event stream data can generate corresponding image frames, i.e., event frames. Specifically, the camera sensor can capture event stream data corresponding to different transmittance levels and transmit the event stream data under different transmittance levels to the perception computing unit, allowing the perception computing unit to obtain event stream data under various transmittance levels. The perception computing unit can then perform event conversion on the aforementioned event stream data under various transmittance levels to generate event frame images from the camera sensor under various transmittance levels.

[0071] Step S204: Fuse multiple event frames with different transmittances to generate an environment-aware image of the event camera, which includes images of relatively stationary objects.

[0072] The environmental perception image is the final perceived environmental image generated by the event camera. After obtaining event frames with various transmittances, the perception computing unit can fuse these multiple event frames to reconstruct the image, thereby generating the environmental perception image of the event camera. Because the transmittance of the photochromic glass changes, reflected light from the environment attenuates as it enters the glass, causing a change in light intensity signal, which in turn generates an event signal from the event camera sensor. Furthermore, since the initial light intensity and wavelength differ at different locations, the transmittance of light of different wavelengths varies as it passes through the photochromic glass. Consequently, the amount of light intensity attenuation sensed by each pixel of the event camera sensor differs, resulting in different numbers of events. Therefore, by fusing multiple event frames with different transmittances, the generated environmental perception image can include relatively stationary objects.

[0073] In the aforementioned imaging method of the event camera, when the perception computing unit determines that a relatively stationary object exists within the perception range of the event camera, it changes the transmittance of a preset photochromic glass. The photochromic glass is placed in front of the camera sensor of the event camera. Event stream data captured by the camera sensor under various transmittances are acquired. Based on the event stream data, event frames of the camera sensor under various transmittances are obtained. Multiple event frames under various transmittances are fused to generate an environmental perception image of the event camera, which includes an image of a relatively stationary object. This application can change the transmittance of the photochromic glass placed in front of the camera sensor of the event camera when the event camera perceives a relatively stationary object, thereby enabling the camera sensor to capture event frames under different transmittances. These event frames can then be fused to form a perception image containing a relatively stationary object. By changing the transmittance of the photochromic glass, this application can form an image containing a relatively stationary object, thereby improving the completeness of the image generated by the event camera.

[0074] In one embodiment, step S201 may further include: acquiring a plurality of preset input signals of different frequencies, and inputting the plurality of input signals of different frequencies to a preset change control unit; wherein the change control unit is signal-connected to the photochromic glass, and the plurality of input signals of different frequencies are used to trigger the change control unit to generate a plurality of corresponding control signals, and to use the control signals to change the light transmittance of the photochromic glass.

[0075] In this embodiment, the change in light transmittance of the photochromic glass can be achieved through a change control unit, which can be connected to the photochromic glass signal, as shown below. Figure 3 As shown, in this embodiment, the sensing and computing unit is signal-connected to the change control unit, which is also signal-connected to the photochromic glass. Input signals of different frequencies are used by the sensing and computing unit to trigger the change control unit to output control signals that change the light transmittance of the photochromic glass. Different frequency input signals can be used to output different control signals to control different light transmittances of the photochromic glass, thereby achieving a change in light transmittance.

[0076] Specifically, in this embodiment, when the sensing and computing unit determines that there is a relatively stationary object within the sensing range of the event camera, it generates an input signal. This input signal corresponds to multiple pre-set different frequencies, and the multiple input signals of different frequencies are input to the change control unit connected to the photochromic glass. The change control unit can then output corresponding control signals according to the different frequency input signals, thereby changing the light transmittance of the photochromic glass through the control signals.

[0077] In this embodiment, the sensing and computing unit can input input signals of different frequencies to the change control unit, and the change control unit can control the light transmittance of the photochromic glass based on the different control signals to achieve the change of the light transmittance of the photochromic glass. Compared with directly using the sensing and computing unit to change the light transmittance of the photochromic glass, this embodiment can reduce the amount of computation of the sensing and computing unit through the change control unit.

[0078] In addition, the imaging method of the event camera may also include: when the event camera is in its initial state, initializing the transmittance of the photochromic glass to the maximum transmittance within an adjustable range.

[0079] If the event camera is running in its initial state—for example, after the event camera is started but before it detects a relatively stationary object—the transmittance of the photochromic glass can be initialized to the maximum transmittance within an adjustable range. Specifically, during the event camera's initialization, the transmittance of the photochromic glass can be set to the maximum transmittance within an adjustable range, and the system can detect the presence of a relatively stationary object. If such an object is detected, the transmittance of the photochromic glass can be changed from the initially set maximum transmittance to various other transmittance values.

[0080] In this embodiment, when the event camera is in its initial state, the transmittance of the photochromic glass can be initialized to the maximum transmittance within an adjustable range. This ensures that even when no relatively stationary object is detected, accurate imaging of non-relatively stationary objects can still be achieved, thereby ensuring the accuracy of the event camera's imaging.

[0081] In one embodiment, step S204 may further include: performing Gaussian filtering on the event frames corresponding to each transmittance to obtain denoised event frames; and performing weighted averaging on the denoised event frames to obtain the environmental perception image of the event camera.

[0082] The denoised event frame refers to the denoised event frame obtained after Gaussian filtering of the event frames corresponding to various transmittances. In this embodiment, after the perception computing unit obtains the event frame corresponding to each transmittance, it does not directly fuse the above event frames to obtain the environmental perception image of the event camera. Instead, it first performs Gaussian filtering on the event frames of each transmittance to denoise them, and then performs a weighted average on the denoised event frames to obtain the environmental perception image of the event camera.

[0083] For example, the event frames obtained by the perception computing unit include event frame A, event frame B, and event frame C. Event frame A corresponds to the event stream data captured by the photochromic glass at transmittance A, event frame B corresponds to the event stream data captured by the photochromic glass at transmittance B, and event frame C corresponds to the event stream data captured by the photochromic glass at transmittance C. Then, the perception computing unit can perform Gaussian filtering on event frames A, B, and C respectively to obtain denoised event frames A, B, and C. Finally, a weighted average is applied to these denoised event frames A, B, and C to obtain the environmental perception image from the event camera.

[0084] In this embodiment, the perception computing unit can first perform Gaussian filtering on the event frames corresponding to each transmittance to obtain denoised event frames, and then perform weighted averaging on the denoised event frames to obtain the final environmental perception image. The above method can improve the accuracy of the generated environmental perception image.

[0085] Furthermore, such as Figure 4 As shown, Gaussian filtering is applied to the event frames corresponding to each transmittance to obtain denoised event frames, which can further include:

[0086] Step S401: Obtain the current event frame and the transmittance corresponding to the current event frame; the current event frame is any one of the event frames corresponding to each transmittance.

[0087] The current event frame can refer to any one of the event frames corresponding to each transmittance, and the transmittance corresponding to the current event frame refers to the transmittance of the event stream data that yields the current event frame. In this embodiment, the perception computing unit can determine one of the event frames as the current event frame and determine the transmittance corresponding to the current event frame.

[0088] Step S402: Based on the current event frame and transmittance, obtain the Gaussian filter parameters corresponding to the current event frame;

[0089] Step S403: Perform Gaussian filtering on the current event frame based on the Gaussian filtering parameters to obtain the denoised event frame corresponding to the current event frame.

[0090] The Gaussian filter parameter refers to the filter parameter used to perform Gaussian filtering on the current event frame. In this embodiment, different filter parameters can be used for Gaussian filtering for different event frames, that is, adaptive filtering is performed on different event frames to obtain the denoised event frame corresponding to each current event frame.

[0091] For example, the input is the current event frame I and the transmittance p corresponding to the current event frame, and the adaptive filtering formula can be shown as follows:

[0092]

[0093]

[0094] Among them, I σ (x,y) represents the event frame after Gaussian filtering, ε σ (x,y) represents the square of the residual, 1 / p refers to the reciprocal of the transmittance, used to balance the effect of transmittance, I o (x,y) represents the final output after Gaussian filtering, i.e., the denoised event frame corresponding to the current event frame, σ best This refers to the Gaussian filter parameters corresponding to the current event frame. The overall goal is to balance the smoothing result and the result of detail features, that is, to obtain the minimum value of σ. Therefore, when optimizing the frame image, the value of σ will be selected automatically, achieving an adaptive effect.

[0095] In this embodiment, adaptive filtering can be performed on each current event frame with different transmittance to obtain the denoised event frames corresponding to each transmittance. The accuracy of Gaussian filtering denoising can be further improved by the above method.

[0096] In one embodiment, such as Figure 5 As shown, the following steps may be included before step S201:

[0097] Step S501: Obtain the current event stream; the current event stream is the event stream collected by the camera sensor during the current time period;

[0098] Step S502: Obtain the historical event stream corresponding to the current event stream; the historical event stream is the event stream collected by the camera sensor in the previous time period of the current time period.

[0099] The current event stream refers to the event stream collected by the camera sensor during the current time period, which can be the time period corresponding to the current moment. The historical event stream, on the other hand, is the event stream collected by the camera sensor during the previous time period. In this embodiment, the camera sensor can collect event streams according to a specific time period, and these collected event streams can be stored in the memory carried by the perception computing unit. Specifically, the perception computing unit can obtain the event stream collected by the camera sensor during the current time period and the event stream collected by the camera sensor during the previous time period, which are respectively used as the current event stream and the historical event stream.

[0100] Step S503: Compare the current event stream with the historical event stream, and determine the perception result of the relatively static object based on the comparison result.

[0101] After collecting the current event stream and the historical event stream, the perception computing unit can also compare the current event stream and the historical event stream, so as to determine the perception result of the relatively static object based on the degree of difference between the current event stream and the historical event stream.

[0102] In this embodiment, the perception computing unit can compare the current event stream collected by the camera sensor in the current time period with the historical event stream collected in the previous time period to determine the perception result of a relatively stationary object based on the comparison result. This method can improve the perception accuracy of relatively stationary objects.

[0103] Furthermore, the imaging region corresponding to the current event stream contains multiple first imaging region blocks; the imaging region corresponding to the historical event stream contains multiple second imaging region blocks, each corresponding to a first imaging region block; such as Figure 6 As shown, step S503 may further include:

[0104] Step S601: Obtain the first event contained in the current first imaging region block, and the second event contained in the current second imaging region block corresponding to the current first imaging region block; the current first imaging region block is any one of multiple first imaging region blocks.

[0105] In this context, the imaging region corresponding to the current event stream refers to the camera imaging region corresponding to the current event stream, while the imaging region corresponding to the historical event stream refers to the camera imaging region corresponding to the historical event stream. This imaging region can contain multiple image region blocks, i.e., imaging region blocks. The first imaging region block refers to the imaging region block used to form the imaging region corresponding to the current event stream, and the second imaging region block refers to the imaging region block used to form the imaging region corresponding to the historical event stream. Furthermore, each imaging region block can contain multiple event data points. The first event refers to the event data contained in the first imaging region block, and the second event refers to the event data contained in the second imaging region block.

[0106] Specifically, the perception computing unit can randomly select one of multiple first imaging region blocks corresponding to the current event stream as the current first imaging region block. It can also select the second imaging region block corresponding to the current first imaging region block—for example, a second imaging region block with the same position in the imaging region as the current first imaging region block—as the current second imaging region block. The perception computing unit can then collect the first event contained in the current first imaging region block and the second event contained in the current second imaging region block, respectively.

[0107] Step S602: Compare the first event with the second event. If the number of event differences between the first event and the second event is greater than a preset difference threshold, the current first imaging region block is taken as the target first imaging region block.

[0108] The preset difference threshold refers to a pre-defined threshold for the number of event differences between the first event and the second event. This threshold can be set to 100. Specifically, the perception calculation unit can subtract the first event contained in the current first imaging region block from the second event contained in the current second imaging region block to obtain the event difference between the first event and the second event, and count the number of such event differences. If the number of event differences is greater than the preset difference threshold, then the current first imaging region block is taken as the target first imaging region block, thereby identifying whether each first imaging region block is the target first imaging region block in the above manner.

[0109] Step S603: If there is at least one target first imaging region block in the imaging region corresponding to the current event stream, it is determined that there is a relatively stationary object in the current event stream.

[0110] Finally, if there is at least one target first imaging region block in the imaging region corresponding to the current event stream, the perception computing unit can determine that there is a relatively stationary object in the current event stream, thereby changing the light transmittance of the photochromic glass. If there is no target first imaging region block in the imaging region corresponding to the current event stream, the perception computing unit can determine that no relatively stationary object has been detected. In this case, the light transmittance of the photochromic glass will be kept in the initial state, for example, the photochromic glass can be kept in the state of maximum light transmittance.

[0111] In this embodiment, the perception computing unit can compare the first event contained in the first imaging region block with the second event contained in the second imaging region block corresponding to the first imaging region block, thereby determining whether the imaging region corresponding to the current event stream contains at least one target first imaging region block. If so, it is determined that there is a relatively stationary object in the current event stream. By comparing the events contained in the imaging region blocks, the recognition accuracy of relatively stationary objects can be further improved.

[0112] In addition, step S503 may also include: acquiring historical location data corresponding to the event camera; performing correction processing on the historical event stream based on the historical location data to obtain the corrected historical event stream; and comparing the current event stream with the corrected historical event stream.

[0113] Historical location data refers to historical data on the location of event cameras. This historical data may be stored in the odometer included in the sensing and computing unit. The corrected historical event stream refers to the historical time stream after processing using the historical location data. In this embodiment, before comparing the current event stream with the historical event stream, the sensing and computing unit can also obtain the historical camera location data from the odometer, i.e., the historical location data. This historical location data is then used to correct the historical event stream, resulting in the corrected historical event stream. Finally, the current event stream and the corrected historical event stream are compared to determine the relatively stationary object.

[0114] In this embodiment, the perception computing unit can use historical data of the event camera location to correct the historical event stream, and then compare the corrected historical event stream with the current event stream to determine relatively stationary objects. This method can further improve the recognition accuracy of relatively stationary objects.

[0115] In one embodiment, such as Figure 7 As shown, an imaging system for an event camera is provided. This system may include: a perception computing unit 701, a camera sensor 702, and a photochromic glass 703 disposed in front of the camera sensor 702; wherein,

[0116] The sensing and computing unit 701 is mainly used to change the light transmittance of the photochromic glass 703 when a relatively stationary object is detected within the sensing range of the event camera, that is, when the camera sensor and the object being photographed are both stationary within the sensing range of the event camera, or when the camera sensor and the object are moving in the same direction and at the same speed.

[0117] Subsequently, as the light transmittance of the photochromic glass 703 changes, the camera sensor 702 can capture event stream data corresponding to various light transmittances of the photochromic glass 703, and send the event stream data under different light transmittances to the perception computing unit 701, so that the perception computing unit 701 can obtain event stream data under different light transmittances.

[0118] Finally, the perception computing unit 701 can also capture event stream data under different transmittance based on the camera sensor 702, obtain corresponding event frames under various transmittance, and fuse multiple event frames under different transmittance to obtain an environmental perception image containing relatively stationary objects.

[0119] The aforementioned event camera imaging system includes: a perception computing unit 701, a camera sensor 702, and a photochromic glass 703 disposed in front of the camera sensor 702; wherein: the perception computing unit 701 is used to change the transmittance of the photochromic glass 703 when it is determined that there is a relatively stationary object within the perception range of the event camera; the camera sensor 702 is used to capture event stream data corresponding to various transmittances of the photochromic glass 703, and send the event stream data at various transmittances to the perception computing unit 701; the perception computing unit 701 is also used to obtain event frames of the camera sensor 702 at various transmittances based on the event stream data at different transmittances, and fuse multiple event frames at various transmittances to generate an environmental perception image of the event camera, wherein the environmental perception image contains images of relatively stationary objects. This application can change the transmittance of the photochromic glass 703 placed in front of the camera sensor 702 of the event camera when the perception computing unit 701 perceives a relatively stationary object. This allows the camera sensor 702 to capture event frames at different transmittances, and then fuse these event frames to form a perceived image containing the relatively stationary object. By changing the transmittance of the photochromic glass, this application can form an image containing the relatively stationary object, thereby improving the completeness of the image generated by the event camera.

[0120] In one embodiment, such as Figure 8 As shown, the imaging system of the event camera may further include: a change control unit 704 connected to the photochromic glass 703; a perception computing unit 701, which is further configured to acquire multiple preset input signals of different frequencies when it is determined that there is a relatively stationary object within the perception range of the event camera, and input the multiple input signals of different frequencies to the preset change control unit; and the change control unit 704, which is configured to generate control signals corresponding to the multiple input signals of different frequencies respectively, and use the control signals to change the light transmittance of the photochromic glass.

[0121] In this embodiment, the imaging system of the event camera, in addition to including the perception computing unit 701, the camera sensor 702, and the photochromic glass 703, may also include a change control unit 704 connected to the photochromic glass 703, and the light transmittance of the photochromic glass can be changed through the change control unit 704. Specifically, when the perception computing unit 701 determines that there is a relatively stationary object within the perception range of the event camera, it can generate multiple input signals corresponding to different preset frequencies and input them to the change control unit 704 for controlling the light transmittance of the photochromic glass 703. Then, the change control unit 704 outputs corresponding control signals according to the input signals of different frequencies, thereby changing the light transmittance of the photochromic glass through the control signals.

[0122] In this embodiment, the sensing and computing unit 701 can input input signals of different frequencies to the change control unit 704, and the change control unit 704 can control the light transmittance of the photochromic glass based on different control signals to achieve the change of the light transmittance of the photochromic glass. Compared with directly using the sensing and computing unit 701 to change the light transmittance of the photochromic glass, this embodiment can reduce the amount of computation of the sensing and computing unit 701 by using the change control unit.

[0123] In one embodiment, the perception computing unit 701 includes: a memory and a comparison unit; the memory is used to store the current event stream and the historical event stream; the current event stream is the event stream collected by the camera sensor 702 in the current time period; the historical event stream is the event stream collected by the camera sensor 702 in the previous time period of the current time period; the comparison unit is used to compare the current event stream and the historical event stream, and determine the perception result of the relatively stationary object based on the comparison result.

[0124] In this embodiment, the perception computing unit 701 may include a memory and a comparison unit. The memory can be used to store event streams, including the event stream collected by the camera sensor 702 in the current time period (i.e., the current event stream) and the event stream collected by the camera sensor 702 in the previous time period (i.e., the historical event stream). The comparison unit is a processing unit used to determine the perception result of a relatively stationary object. This unit can compare the current event stream and the historical event stream, and based on the comparison result, ultimately determine the perception result of the relatively stationary object. For example, it can be done by comparing the imaging area corresponding to the current event stream, and whether there is an event contained in a certain imaging area block, and whether the difference between the event contained in the corresponding imaging area block in the historical event stream and the event contained in the same imaging area block is greater than a certain preset difference threshold. If so, it is determined that a relatively stationary object exists.

[0125] In this embodiment, the perception calculation unit 701 may include a memory and a comparison unit. The memory stores the current event stream and the historical event stream, and the comparison unit compares the current event stream and the historical event stream to determine the perception result of the relatively stationary object based on the comparison result. This method can improve the perception accuracy of the relatively stationary object.

[0126] Furthermore, the perception computing unit 701 may also include: an odometer; the odometer is used to store historical location data corresponding to the event camera; a comparison unit is also used to perform correction processing on the historical event stream based on the historical location data to obtain the corrected historical event stream; and to compare the current event stream with the corrected historical event stream.

[0127] like Figure 9As shown, in this embodiment, the perception computing unit 701 may include, in addition to the memory 901 and the comparison unit 902, an odometer 903. The odometer 903 can be used to record historical data of the location of the event camera, that is, the historical location data corresponding to the event camera. The comparison unit 902 can first perform correction processing on the historical event stream based on the historical location data recorded by the odometer 903, and after obtaining the corrected historical event stream, compare the current event stream with the corrected historical event stream.

[0128] In this embodiment, the comparison unit 902 can use the historical data of the event camera location stored in the odometer 903 to correct the historical event stream, and then compare the corrected historical event stream with the current event stream to determine the relatively stationary object. This method can further improve the recognition accuracy of the relatively stationary object.

[0129] In one embodiment, such as Figure 10 As shown, the change control unit 704 may include: a trigger 1001 and a numerical control unit 1002; the trigger 1001 is used to receive multiple input signals of different frequencies and generate trigger signals corresponding to the multiple input signals of different frequencies, and input the multiple trigger signals to the numerical control unit 1002; the numerical control unit 1002 is used to receive multiple trigger signals and generate control signals corresponding to the multiple trigger signals, and use the control signals to change the light transmittance of the photochromic glass 703.

[0130] The trigger 1001 is a trigger unit used to trigger the numerical control unit 1002 to generate a control signal for changing the light transmittance of the photochromic glass 703. The numerical control unit 1002 is a control unit used to output the control signal, which can be used to control and change the light transmittance of the photochromic glass 703. Specifically, after the sensing and computing unit 701 inputs input signals of different frequencies to the change control unit 704, the change control unit 704 can first receive multiple input signals of different frequencies and generate trigger signals corresponding to the multiple input signals of different frequencies, and input the trigger signals to the numerical control unit 1002. Then, the numerical control unit 1002 can receive the multiple trigger signals and generate a corresponding control signal for each trigger signal, thereby using the control signals to change the light transmittance of the photochromic glass 703.

[0131] In this embodiment, the change control unit 704 can be composed of a trigger 1001 and a numerical control unit 1002. The trigger 1001 can receive input signals of different frequencies and output corresponding trigger signals, while the numerical control unit 1002 can output control signals for changing the light transmittance of the photochromic glass 703 based on the trigger signals. In this way, the efficiency of the change control unit 704 in controlling the light transmittance of the photochromic glass 703 can be improved.

[0132] In one embodiment, a method for imaging relatively stationary targets based on an event camera is also provided, the technical framework of which can be as follows: Figure 11 As shown, photochromic glass is placed in front of the lens, changing the intensity of ambient reflected light perceived by the event camera by adjusting its transmittance. The perception computing unit determines whether there are relatively stationary objects in the current scene. When a relatively stationary object is present, it sends a signal to the change control unit to activate the static perception mode. In static perception mode, the change control unit controls the transmittance of the photochromic glass at multiple frequencies to change the light intensity received by the event camera sensor, thereby perceiving and imaging the relatively stationary object. The reconstruction algorithm further fuses the multi-frequency perception results to form the final static perception.

[0133] The sensing and computing unit consists of a memory, a comparison unit, and an odometer, such as... Figure 12 As shown. The memory is used to store historical event stream data. The odometer outputs historical data of the camera position. The comparison unit uses the historical data of the camera position to correct the event stream data of the historical time period and compares it with the event stream data of the current time period to determine whether there are relatively stationary objects in the current environment and outputs a signal. When the perception computing unit is in static perception mode, the perception computing unit reads the multi-frequency event stream data stored in the memory, uses a reconstruction algorithm to fuse the multi-frequency event stream data, and forms the final static perception image.

[0134] The change control unit consists of triggers and numerical control units, such as... Figure 13 As shown. The trigger receives the output signal from the sensing and computing unit, generates a trigger signal, and transmits it to the numerical control unit. After being triggered, the numerical control unit outputs a control signal to the photochromic glass to adjust its light transmittance. In static sensing mode, multiple adjustments are required to ensure sufficient data in the memory for image reconstruction.

[0135] The specific process of the reconstruction algorithm is as follows: Figure 14 As shown, the algorithm input is multi-frequency event stream data. The event stream of a single frequency undergoes event transformation and keyframe selection to obtain the corresponding candidate frame. The transmittance corresponding to the frequency and the corresponding candidate frame are used as joint input. The candidate frame is denoised and the signal strength is adjusted by adaptive filtering. The denoised frame is weighted and averaged to output the final frame, which is the final static scene capture image.

[0136] Noise reduction and signal strength adjustment employ an adaptive Gaussian filter with transmittance adjustment. The inputs are transmittance p and the original event frame I. The adaptive filtering formula is as follows:

[0137]

[0138]

[0139] Among them, I σ (x,y) represents the original event frame after Gaussian filtering, ε σ (x,y) represents the square of the residual, and 1 / p refers to the reciprocal of the transmittance, used to balance the impact of transmittance. The overall goal is to balance the smoothing result and the result of detail features, that is, to achieve the minimum of the sum. Therefore, when optimizing a frame image, the size of σ is automatically selected to achieve an adaptive effect.

[0140] The specific process of this embodiment can be shown below:

[0141] Step 1: The photochromic glass is initially in its highest transmittance state. Ambient reflected light passes through the photochromic glass and enters the event camera sensor to obtain the event stream of objects in motion in the current scene;

[0142] Step 2: The event stream enters the perception and computing unit and is gradually saved to the memory. The memory transmits the event stream data saved in the previous time period and the event stream data in the current time period to the comparison unit. The comparison unit combines the odometer data to correct the event stream data of the historical time period and compares the corrected event stream with the event stream of the current time period to determine whether there are relatively stationary objects in the current scene and outputs a signal to the change control unit.

[0143] Step 3: If a relatively stationary object exists, the system enters static perception mode. The perception computing unit begins generating signals at different frequencies. The trigger in the change control unit receives the signal input and outputs a trigger signal, which is transmitted to the numerical control unit. The numerical control unit outputs a control signal to change the light transmittance of the color-changing glass. By using a pre-set multi-frequency signal stream, the perception information stream of the current scene under multiple light transmittance conditions can be controlled. Multiple information streams are combined with light transmittance information for corresponding scaling and reconstruction to form the final perception of the relatively stationary object.

[0144] Its core principle can be as follows Figure 15As shown, an event camera is a bio-like visual sensor that generates signals based on changes in light intensity. Because the transmittance of photochromic glass changes, ambient reflected light attenuates as it enters the glass, causing a change in light intensity signal. Therefore, the event camera sensor generates an event signal. Simultaneously, because the initial light intensity and wavelength differ at different locations, the transmittance of different wavelengths of light varies as they pass through the photochromic glass. Consequently, the amount of light intensity attenuation sensed by each pixel of the event camera sensor differs, resulting in different numbers of events being generated. By reconstructing the event streams generated by different transmittances and light intensities, details of the current static scene can be optimized and obtained.

[0145] This embodiment enables a camera in motion to capture other relatively stationary objects moving at high speeds within its observation range, thereby improving environmental perception.

[0146] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0147] Based on the same inventive concept, this application also provides an imaging apparatus for an event camera to implement the imaging method of the event camera described above. The solution provided by this apparatus is similar to the solution described in the above method. Therefore, the specific limitations of one or more imaging apparatus embodiments of the event camera provided below can be found in the limitations of the imaging method of the event camera above, and will not be repeated here.

[0148] In one embodiment, such as Figure 16 As shown, an imaging device for an event camera is provided, comprising: a transmittance changing module 1601, an event data acquisition module 1602, an event frame generation module 1603, and a perceived image generation module 1604, wherein:

[0149] The transmittance changing module 1601 is used to change the transmittance of a preset photochromic glass when it is determined that there is a relatively stationary object within the sensing range of the event camera; the photochromic glass is located in front of the camera sensor of the event camera.

[0150] The event data acquisition module 1602 is used to acquire event stream data captured by the camera sensor under various different transmittance conditions;

[0151] The event frame generation module 1603 is used to obtain event frames of the camera sensor under various transmittances based on event stream data.

[0152] The perception image generation module 1604 is used to fuse multiple event frames under different transmittances to generate an environmental perception image of the event camera, which contains images of relatively stationary objects.

[0153] In one embodiment, the transmittance changing module 1601 is further configured to acquire a plurality of pre-set input signals of different frequencies and input the plurality of input signals of different frequencies to a preset change control unit; wherein, the change control unit is connected to the photochromic glass, and the change control unit is configured to generate control signals corresponding to the plurality of input signals of different frequencies respectively, and use the control signals to change the transmittance of the photochromic glass.

[0154] In one embodiment, the transmittance changing module 1601 is further configured to initialize the transmittance of the photochromic glass to the maximum transmittance when the event camera is in the initial state.

[0155] In one embodiment, the perception image generation module 1604 is further configured to perform Gaussian filtering on the event frames corresponding to each transmittance to obtain denoised event frames; and to perform weighted averaging on the denoised event frames to obtain the environmental perception image of the event camera.

[0156] In one embodiment, the perceptual image generation module 1604 is further configured to acquire the current event frame and the transmittance corresponding to the current event frame; the current event frame is any one of the event frames corresponding to each transmittance; based on the current event frame and the transmittance, the Gaussian filter parameters corresponding to the current event frame are acquired; and the current event frame is subjected to Gaussian filtering based on the Gaussian filter parameters to obtain the denoised event frame corresponding to the current event frame.

[0157] In one embodiment, the imaging device of the event camera further includes: a relative stillness perception module, used to acquire the current event stream; the current event stream is the event stream collected by the camera sensor in the current time period; acquire the historical event stream corresponding to the current event stream; the historical event stream is the event stream collected by the camera sensor in the previous time period of the current time period; compare the current event stream with the historical event stream, and determine the perception result of the relatively still object based on the comparison result.

[0158] In one embodiment, the imaging region corresponding to the current event stream contains multiple first imaging region blocks; the imaging region corresponding to the historical event stream contains multiple second imaging region blocks corresponding to each of the first imaging region blocks; the relative stillness perception module is further configured to acquire a first event contained in the current first imaging region block, and a second event contained in the current second imaging region block corresponding to the current first imaging region block; the current first imaging region block is any one of the multiple first imaging region blocks; comparing the first event and the second event, when the number of event differences between the first event and the second event is greater than a preset difference threshold, the current first imaging region block is taken as the target first imaging region block; if there is at least one target first imaging region block in the imaging region corresponding to the current event stream, it is determined that there is a relatively still object in the current event stream.

[0159] In one embodiment, the relative stillness sensing module is further configured to acquire historical location data corresponding to the event camera; perform correction processing on the historical event stream based on the historical location data to obtain the corrected historical event stream; and compare the current event stream with the corrected historical event stream.

[0160] The various modules in the imaging device of the aforementioned event camera can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0161] In one embodiment, an event camera is provided, the internal structure of which can be shown in the following diagram: Figure 17 As shown. The event camera includes a camera sensor and a perception computing unit. With photochromic glass in front of the camera sensor, the perception computing unit can be used to implement an imaging method for the event camera.

[0162] Those skilled in the art will understand that Figure 17 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0163] In one embodiment, an event camera is also provided, including a camera sensor, a photochromic glass disposed in front of the camera sensor, and a perception computing unit, the perception computing unit being used to implement the steps in the above method embodiments.

[0164] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0165] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0166] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0167] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0168] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0169] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An imaging method for an event camera, characterized in that, The method includes: When it is determined that there is a relatively stationary object within the sensing range of the event camera, the transmittance of a preset photochromic glass is changed. The photochromic glass is placed in front of the camera sensor of the event camera. The relatively stationary object includes: a subject that is stationary at the same time as the camera sensor, and a subject that moves in the same direction and at the same speed as the camera sensor. Changing the transmittance of the photochromic glass is used to attenuate ambient reflected light when it enters the photochromic glass. In order to make the transmittance of light of different wavelengths different when the ambient reflected light is attenuated by the photochromic glass at different positions and initial light intensities and wavelengths, the attenuation of ambient reflected light by the photochromic glass will result in different transmittance of light of different wavelengths, thereby making the amount of light intensity attenuation felt by the camera sensor at each pixel position of the subject different. Acquire event stream data captured by the camera sensor under various different transmittance conditions; Based on the event stream data, the event frames of the camera sensor under various transmittance conditions are obtained; By fusing multiple event frames with different transmittances, an environment-aware image of the event camera is generated, the environment-aware image containing images of the relatively stationary objects. The process includes: acquiring a current event frame and the transmittance corresponding to the current event frame; the current event frame being any one of the event frames corresponding to each of the transmittances; acquiring Gaussian filter parameters corresponding to the current event frame based on the current event frame and the transmittance; performing Gaussian filtering on the current event frame based on the Gaussian filter parameters to obtain a denoised event frame corresponding to the current event frame; and performing a weighted average on the denoised event frame to obtain the environment-aware image of the event camera.

2. The method according to claim 1, characterized in that, The method of changing the light transmittance of the preset photochromic glass includes: Acquire multiple input signals of different frequencies that are preset, and input the multiple input signals of different frequencies to a preset change control unit; The change control unit is connected to the photochromic glass. The multiple input signals of different frequencies are used to trigger the change control unit to generate multiple corresponding control signals, and the control signals are used to change the light transmittance of the photochromic glass.

3. The method according to claim 2, characterized in that, The method further includes: When the event camera is in its initial state, the transmittance of the photochromic glass is initialized to the maximum transmittance within the adjustable range.

4. The method according to claim 1, characterized in that, The step of changing the transmittance of a preset photochromic glass when it is determined that there is a relatively stationary object within the sensing range of the event camera also includes: Obtain the current event stream; the current event stream is the event stream collected by the camera sensor during the current time period; Obtain the historical event stream corresponding to the current event stream; the historical event stream is the event stream collected by the camera sensor in the previous time period of the current time period; The current event stream is compared with the historical event stream, and the perception result of the relatively static object is determined based on the comparison result.

5. The method according to claim 4, characterized in that, The imaging region corresponding to the current event stream contains multiple first imaging region blocks; the imaging region corresponding to the historical event stream contains multiple second imaging region blocks that correspond to each of the first imaging region blocks respectively. The step of comparing the current event stream with the historical event stream and determining the perception result of the relatively static object based on the comparison result includes: Obtain a first event contained in the current first imaging region block, and a second event contained in the current second imaging region block corresponding to the current first imaging region block; the current first imaging region block is any one of the plurality of first imaging region blocks; By comparing the first event and the second event, if the number of event differences between the first event and the second event is greater than a preset difference threshold, the current first imaging region block is taken as the target first imaging region block; If at least one target first imaging region block exists in the imaging region corresponding to the current event stream, it is determined that there is a relatively stationary object in the current event stream.

6. The method according to claim 4, characterized in that, The comparison of the current event stream with the historical event stream includes: Obtain the historical location data corresponding to the event camera; The historical event stream is corrected based on the historical location data to obtain the corrected historical event stream. Compare the current event stream with the corrected historical event stream.

7. An imaging system for an event camera, characterized in that, include: The sensing and computing unit, the camera sensor, and the photochromic glass disposed in front of the camera sensor; wherein: The perception computing unit is used to change the transmittance of the photochromic glass when it is determined that there is a relatively stationary object within the perception range of the event camera. The relatively stationary object includes: a subject that is stationary at the same time as the camera sensor, and a subject that moves in the same direction and at the same speed as the camera sensor. Changing the transmittance of the photochromic glass is used to attenuate ambient reflected light when it enters the photochromic glass. In order to make the transmittance of light of different wavelengths different when the ambient reflected light is attenuated by the photochromic glass at different locations and initial light intensities and wavelengths, the light transmittance of different wavelengths will be different when the ambient reflected light is attenuated by the photochromic glass, so that the amount of light intensity attenuation felt by the camera sensor at each pixel position of the subject will be different. The camera sensor is used to capture event stream data corresponding to multiple different transmittances of the photochromic glass, and send the event stream data under multiple different transmittances to the sensing computing unit. The perception computing unit is further configured to obtain event frames of the camera sensor under various transmittances based on the event stream data under different transmittances, and fuse multiple event frames under various different transmittances to generate an environmental perception image of the event camera, wherein the environmental perception image includes the image of the relatively stationary object; further configured to obtain the current event frame and the transmittance corresponding to the current event frame; the current event frame is any one of the event frames corresponding to each of the transmittances; obtain the Gaussian filter parameters corresponding to the current event frame based on the current event frame and the transmittance; perform Gaussian filtering on the current event frame based on the Gaussian filter parameters to obtain the denoised event frame corresponding to the current event frame; and perform weighted averaging on the denoised event frame to obtain the environmental perception image of the event camera.

8. The system according to claim 7, characterized in that, Also includes: The change control unit is connected to the photochromic glass signal; The perception computing unit is also used to acquire a plurality of preset input signals of different frequencies when it is determined that there is a relatively stationary object within the perception range of the event camera, and to input the plurality of input signals of different frequencies to a preset change control unit. The change control unit is used to generate control signals corresponding to the multiple input signals of different frequencies, and to use the control signals to change the light transmittance of the photochromic glass.

9. The system according to claim 8, characterized in that, The perception computing unit includes: a memory and a comparison unit; The memory is used to store the current event stream and the historical event stream; the current event stream is the event stream collected by the camera sensor in the current time period; the historical event stream is the event stream collected by the camera sensor in the previous time period of the current time period. The comparison unit is used to compare the current event stream with the historical event stream, and determine the perception result of the relatively static object based on the comparison result.

10. The system according to claim 9, characterized in that, The sensing and computing unit further includes: an odometer; The odometer is used to store historical location data corresponding to the event camera; The comparison unit is further configured to perform correction processing on the historical event stream based on the historical location data to obtain a corrected historical event stream; and compare the current event stream with the corrected historical event stream.

11. The system according to claim 8, characterized in that, The change control unit includes: a trigger and a numerical control unit; The trigger is used to receive the multiple input signals of different frequencies, generate multiple trigger signals corresponding to the input signals of different frequencies, and input the multiple trigger signals to the numerical control unit. The numerical control unit is used to receive multiple trigger signals and generate control signals corresponding to the multiple trigger signals respectively, and use the control signals to change the light transmittance of the photochromic glass.

12. An imaging device for an event camera, characterized in that, The device includes: A transmittance alteration module is used to change the transmittance of a preset photochromic glass when a relatively stationary object is determined to exist within the sensing range of the event camera. The photochromic glass is positioned in front of the camera sensor of the event camera. The relatively stationary object includes: a subject that is stationary at the same time as the camera sensor, and a subject that moves in the same direction and at the same speed as the camera sensor. Changing the transmittance of the photochromic glass causes attenuation of ambient reflected light entering the photochromic glass. This ensures that, under conditions where the initial light intensity and wavelength of the ambient light are different at different locations, the transmittance of different wavelengths of light is different when the ambient reflected light is attenuated by the photochromic glass. Consequently, the amount of light intensity attenuation sensed by the camera sensor at each pixel position of the subject is different. The event data acquisition module is used to acquire event stream data captured by the camera sensor under various different transmittance conditions; An event frame generation module is used to obtain event frames of the camera sensor under various transmittances based on the event stream data. The perception image generation module is used to fuse multiple event frames under various transmittances to generate an environment-aware image of the event camera, wherein the environment-aware image includes images of the relatively stationary objects; it is further used to obtain the current event frame and the transmittance corresponding to the current event frame; the current event frame is any one of the event frames corresponding to each of the transmittances; based on the current event frame and the transmittance, it obtains Gaussian filtering parameters corresponding to the current event frame; it performs Gaussian filtering on the current event frame based on the Gaussian filtering parameters to obtain a denoised event frame corresponding to the current event frame; and it performs weighted averaging on the denoised event frame to obtain the environment-aware image of the event camera.

13. An event camera, characterized in that, The event camera includes a camera sensor and a perception computing unit. When a photochromic glass is provided in front of the camera sensor, the perception computing unit is used to implement the steps of the method according to any one of claims 1 to 6.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images