Event camera based dynamic face recognition in ultra-low light environments

Abstract

The present application relates to the technical field of image recognition, and particularly relates to a dynamic face recognition method in an ultra-low light environment based on an event camera, which comprises the following steps: in the process of dynamic face recognition in an ultra-low light environment, an event accumulation graph time sequence is acquired, a face region in each event accumulation graph is segmented, a face detail feature saliency factor and a face noise interference factor of each event accumulation graph are acquired, an adaptive gamma value of each event accumulation graph is acquired to perform gamma correction on each event accumulation graph, an enhanced event accumulation graph is acquired, face recognition is performed on the enhanced event accumulation graph, and a face recognition result is acquired. The present application improves the accuracy of face recognition by adaptively enhancing the event accumulation graph.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and more specifically to a dynamic face recognition method for ultra-low light environments based on event cameras. Background Technology

[0002] Event cameras, as a novel type of visual sensor, output brightness change events asynchronously, offering advantages such as high dynamic range, microsecond-level temporal resolution, low latency, and low power consumption. Even in extremely low-light conditions (e.g., nighttime without supplemental lighting, dimly lit indoor scenes), event cameras can still capture subtle brightness changes in a scene, thus providing a new technological path for dynamic face recognition. The event stream output by an event camera itself lacks the spatial structure of a traditional image; therefore, events are typically accumulated into a two-dimensional image (Event Accumulation Image) to facilitate feature extraction and face recognition using mature deep learning models.

[0003] Existing problems: The asynchronous nature of event cameras leads to naturally sparse cumulative images, meaning events only occur at points of brightness change, and these changes are even less pronounced in extremely low light. This makes key features such as texture and edges of facial regions in the cumulative image indistinct, directly impacting subsequent feature extraction. Furthermore, in insufficient lighting, the number of events decreases significantly, and effective signals are overwhelmed by noise, resulting in a "dark, blurry, and dirty" cumulative image, which is detrimental to facial landmark detection and feature matching. These issues limit the feature representation capabilities and make it difficult to guarantee recognition accuracy when directly using the raw event cumulative image for face recognition. Summary of the Invention

[0004] This invention provides a dynamic face recognition method for ultra-low light environments based on event cameras to solve existing problems.

[0005] The dynamic face recognition method for ultra-low light environments based on event cameras of the present invention adopts the following technical solution:

[0006] One embodiment of the present invention provides a dynamic face recognition method for ultra-low light environments based on an event camera, the method comprising the following steps:

[0007] In the process of dynamic face recognition under ultra-low light environment, the event accumulation map time sequence is obtained;

[0008] The facial region in each event cumulative image is segmented. Based on the difference in pixel values ​​between pixels in the facial region of each event cumulative image and adjacent pixels, as well as the distance between pixels with similar pixel values, the saliency factor of facial detail features in each event cumulative image is obtained.

[0009] Based on the differences and positional distribution of similar pixel values ​​within the facial region of each event cumulative image, and combined with the changes in pixel values ​​of pixels at the same coordinate position as the event cumulative images are accumulated sequentially, the spatial and temporal noise characteristics of each pixel within the facial region of each event cumulative image are obtained. Based on the spatial and temporal noise characteristics of all pixels within the facial region of each event cumulative image, the noise interference factor of the face in each event cumulative image is obtained.

[0010] Based on the magnitude of the facial detail feature saliency factor and the facial noise interference factor of each event cumulative image, an adaptive gamma value is obtained for each event cumulative image; based on the adaptive gamma value, gamma correction is performed on each event cumulative image to obtain an enhanced event cumulative image; face recognition is performed on the enhanced event cumulative image to obtain the face recognition result.

[0011] Furthermore, the specific steps for obtaining the facial detail feature saliency factor for each event cumulative image are as follows:

[0012] For any event accumulation image, based on the pixel value of each pixel, the Otsu algorithm is used to obtain the optimal segmentation threshold. The region formed by all pixels with pixel values ​​greater than the segmentation threshold is recorded as the face region.

[0013] Based on the difference in pixel value between each pixel in the face area and its neighboring pixels, the event count change value of each pixel is obtained to determine whether each pixel is a blurry pixel or a clear pixel, and then the direction of event count growth of clear pixels is obtained.

[0014] Based on the event count change values ​​of all pixels in the face area and the number of clear pixels in the same event count growth direction, obtain the clarity of the face movement trajectory in any event cumulative image;

[0015] Within the facial region, the absolute difference between the pixel values ​​of any two pixels is used as the clustering distance. A hierarchical clustering algorithm is used to cluster all pixels to obtain several clusters. Based on the pixel values ​​of the pixels in each cluster and the distance between the pixels, the facial edge structure clarity of any event accumulation image is obtained.

[0016] The average of the facial movement trajectory sharpness and the facial edge structure sharpness of any event cumulative image is denoted as the facial detail feature saliency factor of any event cumulative image.

[0017] Furthermore, the specific steps for obtaining the event count change value of each pixel based on the difference in pixel value between each pixel and its adjacent pixels in the face region, to determine whether each pixel is a blurred pixel or a clear pixel, and then obtaining the direction of event count growth for clear pixels, are as follows:

[0018] In any event accumulation map, any pixel in the face area is recorded as the target pixel, and all pixels adjacent to the target pixel are recorded as reference pixels. The maximum value of the absolute difference between the pixel values ​​of the target pixel and all reference pixels is recorded as the event count change value of the target pixel.

[0019] When the event count change value of the target pixel is 0, the target pixel is recorded as a blurred pixel.

[0020] When the event count change value of the target pixel is not 0, the target pixel is recorded as a clear pixel.

[0021] For clear pixels, the reference pixel corresponding to the change value of the event count is recorded as the marker reference pixel;

[0022] If the pixel value of the clear pixel is greater than that of the marker reference pixel, then the direction from the marker reference pixel to the clear pixel is recorded as the direction of the increase in the number of events of the clear pixel.

[0023] If the pixel value of the clear pixel is less than that of the marked reference pixel, then the direction from the clear pixel to the marked reference pixel is recorded as the direction of increase in the number of events for the clear pixel.

[0024] Furthermore, the specific steps for obtaining the facial movement trajectory clarity of any event accumulation image are as follows:

[0025] Within the facial region of any event accumulation image, the mean of the normalized values ​​of the event count changes of all pixels is obtained and recorded as the overall event count change amplitude. All clear pixels with the same event count growth direction are formed into a direction set. The number of clear pixels in each direction set is counted. The maximum value of the number of clear pixels in all direction sets is recorded as the target number. The ratio of the target number to the number of pixels in the facial region is recorded as the consistency of the event count growth direction. The mean of the overall event count change amplitude and the consistency of the event count growth direction is recorded as the facial movement trajectory clarity of any event accumulation image.

[0026] Furthermore, the specific steps for obtaining the facial edge structure clarity of any event accumulation image are as follows:

[0027] The mean value of all pixels in each cluster is recorded as the importance of each cluster.

[0028] The inversely proportional normalized value of the mean Euclidean distance between any two pixels in each cluster is denoted as the spatial centrality of each cluster.

[0029] The sum of the importance of all clusters is obtained and denoted as the total importance. The ratio of the importance of each cluster to the total importance is used as the weight of each cluster. The spatial concentration of all clusters is weighted and summed to obtain the facial edge structure clarity of any event accumulation map.

[0030] Furthermore, the specific steps for obtaining the noise spatial and temporal representations of each pixel within the facial region of each event accumulation image are as follows:

[0031] For any pixel in any cluster, if the pixel is a blurry pixel, then let the disorder of the pixel's motion direction be a preset constant; if the pixel is a sharp pixel, then the normalized value of the mean of the minimum angle between the pixel in the cluster and the direction of the increase of the number of events of all other sharp pixels is denoted as the disorder of the pixel's motion direction.

[0032] The connected component formed by adjacent pixels in each cluster is denoted as the target connected component.

[0033] Based on the disorder of the motion direction of each pixel in each cluster and the area of ​​each target connected region, the noise spatial representation of each pixel in the face region of any event accumulation map is obtained.

[0034] Preset duration In the event accumulation graph time sequence, with the first... Time to the The cumulative graph of all events between time points constitutes the first... Local sequence of events in the cumulative graph at time step;

[0035] Based on the pixel value changes of pixels at the same coordinate position when the event accumulation map is accumulated sequentially in the local sequence of the event accumulation map, the first... The temporal representation of noise in each pixel within the face region of the event accumulation map at any given time.

[0036] Furthermore, the specific steps for obtaining the noise spatial representation of each pixel within the facial region of any event accumulation image are as follows:

[0037] The inversely proportional normalized value of the area of ​​each target connected region is used as the spatial isolation of each pixel within each target connected region.

[0038] Within the face region of any event accumulation map, the mean of the disorder of motion direction and spatial isolation of each pixel is denoted as the noise spatial representation of each pixel.

[0039] Furthermore, the acquisition of the first The noise temporal representation of each pixel within the face region of the event accumulation map at any given time step includes the following specific steps:

[0040] In the local sequence of event accumulation graphs, the first event accumulation graph is denoted as the first updated event accumulation graph, the sum of the first and second event accumulation graphs is denoted as the second updated event accumulation graph, and so on, until the sum of all event accumulation graphs is denoted as the last updated event accumulation graph, thus forming an updated event accumulation graph sequence.

[0041] In the Within the face region of the event accumulation map at any given time, any pixel is designated as the master pixel, and the coordinates of the master pixel are designated as the master coordinates.

[0042] Get the The maximum value among all pixel values ​​in the cumulative image of the Zhang Gengxin event is recorded as the standard pixel value. The ratio of the pixel value of a pixel at the principal coordinate position to the standard pixel value in the cumulative graph of the Zhang Gengxin event is denoted as the first pixel. The cumulative event representation at the main coordinate position within Zhang Gengxin's event cumulative graph;

[0043] In the sequence of updated event accumulation graphs, the cumulative event performance at the main coordinate position within all updated event accumulation graphs is obtained sequentially to form a sequence of event accumulation performance.

[0044] Using the least squares method, a straight line is fitted to the cumulative event sequence, and the inversely proportional normalized value of the slope of the fitted line is obtained as the first... Noise temporal representation of the main pixels in the face region of the event accumulation map at any given time.

[0045] Furthermore, the specific steps for obtaining the face noise interference factor in each event cumulative image are as follows:

[0046] Within the facial region of any event accumulation image, the average of the spatial and temporal noise representations of each pixel is denoted as the overall noise representation of each pixel.

[0047] The average noise performance of all pixels is obtained and recorded as the total noise performance value. All pixels with a noise performance value greater than the preset noise threshold are recorded as suspected noise pixels. The ratio of the number of suspected noise pixels to the number of pixels in the face area is recorded as the noise quantity ratio. The average of the total noise performance value and the noise quantity ratio is recorded as the noise interference factor of the face in any event cumulative image.

[0048] Furthermore, the specific steps for obtaining the adaptive gamma value of each event accumulation map are as follows:

[0049] For any event cumulative image, the inverse proportional value of the noise interference factor on the face and the mean of the significant factor of facial detail features are denoted as the gamma adjustment coefficient.

[0050] The difference between the preset upper limit gamma value and the preset lower limit gamma value is recorded as the gamma adjustment value.

[0051] The sum of the product of the gamma adjustment coefficient and the gamma adjustment value of any event accumulation graph and the preset gamma lower limit value is denoted as the adaptive gamma value.

[0052] The beneficial effects of the technical solution of the present invention are:

[0053] In this embodiment of the invention, during dynamic face recognition in ultra-low light environments, a time sequence of event accumulation maps is acquired, the facial region in each event accumulation map is segmented, and the saliency factor of facial detail features and the noise interference factor of the face in each event accumulation map are obtained. This yields an adaptive gamma value for each event accumulation map. The adaptive gamma value can filter out noise interference while preserving facial detail features, resulting in a more natural visual effect for the enhanced event accumulation map. Based on the adaptive gamma value, gamma correction is performed on each event accumulation map to obtain an enhanced event accumulation map. Face recognition is then performed on the enhanced event accumulation map to obtain the face recognition result. Thus, using the adaptively enhanced high-quality event accumulation map for face recognition ensures the detectability of the facial region, the stability of key feature points, and the discriminative ability of feature representation. Therefore, this invention improves the accuracy of face recognition through adaptively enhanced event accumulation maps. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1This is a flowchart illustrating the steps of the dynamic face recognition method for ultra-low light environments based on an event camera according to the present invention.

[0056] Figure 2 This is a schematic diagram of a dynamic face recognition system for ultra-low light environments based on an event camera. Detailed Implementation

[0057] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of the event camera-based dynamic face recognition method for ultra-low light environments proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0059] In all division operations covered in this application, a smoothing mechanism can be employed to prevent computer program crashes or invalid values ​​from being generated due to a denominator of zero. Specifically, a positive correction factor can be added to the denominator term of the division operation. For example, a value of 0.01 can be used to ensure the robustness and feasibility of the algorithm under extreme conditions.

[0060] The following description, in conjunction with the accompanying drawings, details the specific scheme of the dynamic face recognition method for ultra-low light environments based on event cameras provided by this invention.

[0061] Please see Figure 1 The diagram illustrates a flowchart of a dynamic face recognition method for ultra-low light environments based on an event camera, according to an embodiment of the present invention. The method includes the following steps:

[0062] Step S001: In the process of dynamic face recognition under ultra-low light environment, obtain the event accumulation map time sequence.

[0063] Because the event camera outputs an asynchronous event stream, rather than traditional frame images, an event accumulation map needs to be acquired and constructed in order to convert the event stream into a structured image that can be used for face recognition.

[0064] In the process of dynamic face recognition under ultra-low light conditions, the continuous event stream collected by the event camera is divided into multiple event sequences according to a fixed time interval to obtain the event accumulation map time sequence.

[0065] Among them, using an event camera to generate an event accumulation map is a common-sense operation, which is a standard preprocessing method to convert asynchronous event streams into frame-based data to adapt to traditional vision algorithms.

[0066] It should be noted that the preset time interval is 20 milliseconds, that is, an event accumulation map is obtained every 20 milliseconds. Each event accumulation map corresponds to a timestamp. The pixel value of each pixel in the event accumulation map is the number of times the event occurs at the pixel's location within 20 milliseconds. The polarity of each event is not distinguished, that is, only the number of events is counted, and it does not matter whether the brightness increases or decreases.

[0067] Step S002: Segment the facial region in each event cumulative image. Based on the difference in pixel values ​​between pixels in the facial region of each event cumulative image and adjacent pixels, as well as the distance between pixels with similar pixel values, obtain the facial detail feature saliency factor of each event cumulative image.

[0068] It should be noted that under ultra-low light conditions, the impact of light changes is amplified, resulting in significant brightness fluctuations in the event accumulation map. Gamma correction, as a classic lighting compensation technique, can effectively reduce the impact of light changes on recognition performance and improve the algorithm's adaptability and recognition accuracy in extreme low-light environments. Therefore, in this embodiment, gamma correction is used to enhance the event accumulation map. Event accumulation maps often exhibit insufficient detail in dark areas under ultra-low light conditions, such as low event density and weak contrast in areas with gentle changes, like the cheeks and forehead. Gamma correction, through non-linear mapping of pixel intensity, can stretch the dynamic range of low-brightness areas, making the originally weak event structure clearer, thereby highlighting weak facial textures and edges. Conversely, areas with drastic facial expressions or high-contrast edges (eyes, mouth, nose) are prone to forming overly bright blocks or event accumulation in the event accumulation map. Gamma correction can compress the dynamic range of high-brightness areas, preventing these areas from dominating features and resulting in a more balanced overall brightness distribution. Furthermore, the output of the event camera is highly sensitive to motion, leading to significant differences in event density across different regions of the accumulated image. Gamma correction can adjust the global brightness distribution, making the features of different areas of the face more consistent, which helps to improve the stability and robustness of subsequent feature extraction.

[0069] It should be further noted that: when the gamma value in gamma correction... When the value is less than 1, it can stretch the dynamic range of low-brightness areas and compress the dynamic range of high-brightness areas. At gamma values... When the value is less than 1, the gamma value The smaller the value, the stronger the stretching of shadows in the enhanced event accumulation image, and the more obvious the shadow details. At the same time, the highlights are compressed, and the contrast of highlight noise is reduced. Therefore, highlight noise will be suppressed to a certain extent, but it is easy to cause overexposure of highlights and loss of details, and the gamma value will be affected. A higher gamma value results in a more natural enhanced event accumulation map, but the improvement in shadows is limited, and the suppression of noise in highlights is relatively poor due to reduced compression of highlights. Therefore, in event accumulation map enhancement, the gamma value... The choice needs to strike a balance between "detail enhancement" and "noise control".

[0070] Preferably, in one embodiment of the present invention, the method for obtaining the saliency factor of facial detail features in each event cumulative image includes:

[0071] In the event accumulation graph time sequence, for any event accumulation graph, the optimal segmentation threshold is obtained by using the Otsu algorithm based on the pixel value of each pixel. The region consisting of all pixels with pixel values ​​greater than the optimal segmentation threshold is denoted as the face region, and the region consisting of all pixels with pixel values ​​less than or equal to the optimal segmentation threshold is denoted as the background region.

[0072] The Otsu algorithm is a well-known technique, and its specific method will not be described here.

[0073] It should be noted that the event camera outputs an asynchronous event stream. In extremely low light conditions, the background usually has almost no brightness change, so the number of events in the background area is extremely small and the pixel value is close to zero. However, even in extremely low light conditions, the human face will have slight movements (breathing, facial expressions, slight head movements), which will continuously trigger events. Therefore, the pixel value in the face area is significantly higher than that in the background.

[0074] In any event accumulation map, any pixel in the face area is designated as the target pixel, and all pixels adjacent to the target pixel are designated as reference pixels. The absolute value of the difference between the pixel value of the target pixel and each reference pixel is obtained, and the maximum value among the absolute values ​​of the difference between the pixel value of the target pixel and all reference pixels is obtained and designated as the event count change value of the target pixel.

[0075] When the event count change value of the target pixel is 0, the target pixel is recorded as a blurred pixel.

[0076] When the event count change value of the target pixel is not 0, the target pixel is recorded as a clear pixel.

[0077] It should be noted that the event accumulation image is essentially an "event count image". Therefore, the event density varies greatly in areas with clear details, while the event density tends to be smooth in areas with blurred details.

[0078] For clear pixels, the reference pixel corresponding to the change value of the event count is recorded as the marker reference pixel.

[0079] If the pixel value of the clear pixel is greater than that of the marker reference pixel, then the direction from the marker reference pixel to the clear pixel is recorded as the direction of the increase in the number of events for the clear pixel.

[0080] If the pixel value of the clear pixel is less than that of the marked reference pixel, then the direction from the clear pixel to the marked reference pixel is recorded as the direction of increase in the number of events for the clear pixel.

[0081] Using the above method, it is possible to determine whether each pixel in the face region of the event accumulation map is a clear pixel, and the direction of the increase in the number of events for clear pixels.

[0082] In this embodiment, the min-max normalization method is used to normalize the event count variation values ​​of all pixels within the face region of the cumulative event map to a range of 0 to 1. The min-max normalization method is a well-known technique, and its specific method will not be described here.

[0083] Within the facial region of any event accumulation image, the mean of the normalized values ​​of the event count changes of all pixels is obtained and recorded as the overall event count change amplitude. All clear pixels with the same event count growth direction are formed into a direction set. The number of clear pixels in each direction set is counted. The maximum value of the number of clear pixels in all direction sets is recorded as the target number. The ratio of the target number to the number of pixels in the facial region is recorded as the consistency of the event count growth direction. The mean of the overall event count change amplitude and the consistency of the event count growth direction is recorded as the facial movement trajectory clarity of any event accumulation image.

[0084] It should be noted that in the event accumulation map, the event count growth direction of most pixels in the face region is dominated by the overall movement of the face, while the event count growth direction of a small number of pixels is generated by local movements such as the eyes and mouth, thus exhibiting the characteristics of superposition of overall and local movements. Therefore, if the change value of the event count of pixels in the face region is larger, and the proportion of pixels with the same event count growth direction is larger, it indicates that the overall movement of the face is more obvious in the event accumulation map, the facial movement trajectory is clearer, the event density change of the facial edge structure is more significant, and the interference of local movements (such as blinking and mouth movements) is relatively smaller.

[0085] It should be noted that in the event accumulation map, the dramatic changes in brightness at points where the facial contour and texture change trigger a large number of asynchronous events. These events accumulate spatially, forming high-density event areas, which are the locations of pixels where events occur frequently. Therefore, the sharpness of the facial contour and texture edges directly depends on the spatial distribution characteristics of these high-event-frequency pixels. The more concentrated the high-event-frequency pixels are in space, the sharper and clearer the edge structure; conversely, the more dispersed the high-event-frequency pixels are, the more blurred and smooth the edge structure.

[0086] Within the face region of any event accumulation map, the absolute value of the difference between the pixel values ​​of any two pixels is used as the clustering distance. A hierarchical clustering algorithm is then used to cluster all pixels to obtain several clusters.

[0087] In this cluster, the pixel values ​​of the pixels in each cluster are similar, meaning that the brightness change frequency of this region is similar and the degree of motion may be similar. The hierarchical clustering algorithm is a well-known technique, and the specific method will not be introduced here.

[0088] The mean of the pixel values ​​of all pixels in each cluster is recorded as the importance of each cluster.

[0089] The mean of the Euclidean distance between any two pixels in each cluster. The inverse proportional normalized value is denoted as the spatial centrality of each cluster.

[0090] Among them, with The ratio of the length of the event accumulation graph to the length of its diagonal is used as... The normalized value, minus 1 The difference between the normalized values ​​is used as The inverse proportional normalized value.

[0091] The sum of the importance of all clusters is obtained and denoted as the total importance. The ratio of the importance of each cluster to the total importance is used as the weight of each cluster. The spatial concentration of all clusters is weighted and summed to obtain the facial edge structure clarity of any event accumulation map.

[0092] It should be noted that: within the facial region of the event accumulation map, pixels with a high number of events may contain more motion information. Therefore, importance is used as the weight to perform a weighted summation of spatial concentration. The larger the calculated data value, the clearer the facial edge structure.

[0093] The mean values ​​of the facial movement trajectory clarity and facial edge structure clarity of any event cumulative image are obtained and denoted as the facial detail feature significance factor of any event cumulative image.

[0094] Step S003: Based on the differences and positional distribution of similar pixel values ​​in the face region of each event cumulative image, and combined with the pixel value changes of pixels at the same coordinate position when the event cumulative images are accumulated sequentially, obtain the noise spatial and temporal representation of each pixel in the face region of each event cumulative image; based on the noise spatial and temporal representation of all pixels in the face region of each event cumulative image, obtain the noise interference factor of the face in each event cumulative image.

[0095] It should be noted that in ultra-low light environments, the events output by the event camera are inherently sparse and sensitive to noise. The quality of the event accumulation map directly affects the detectability of face regions, the stability of key feature points, the discriminative power of feature representation, and robustness to pose and expression changes. Low-quality event accumulation maps can lead to face detection failures, unstable feature extraction, and decreased recognition accuracy. Specifically, event sparsity can cause facial edge structures to become blurred or discontinuous in the event accumulation map. Having analyzed the clarity of facial edge structures in the event accumulation map, it is further necessary to analyze the noise interference within the event accumulation map. Noise in the event accumulation map consists of random, isolated, sparse high-event points or small clusters, spatially disordered and temporally unstable.

[0096] Preferably, in one embodiment of the present invention, the method for obtaining the noise interference factor of the face in each event cumulative image includes:

[0097] The default constant is 1, and this will be used as an example for explanation.

[0098] For any pixel in any cluster within the facial region of any event accumulation image, if the pixel is blurry, then the disorder of the pixel's motion direction is set to a preset constant; if the pixel is sharp, then the average of the minimum angles between the pixel in the cluster and the minimum angles between the event count growth directions of all other sharp pixels is taken as the average of the minimum angles between the pixel in the cluster and the minimum angles between ... The normalized value is denoted as the disorder of the movement direction of the pixel.

[0099] It should be noted that if only one pixel in a cluster is a clear pixel, and the minimum angle value cannot be calculated, the disorder of the pixel's motion direction is also set to a preset constant. Since the maximum minimum angle value in different directions is 180 degrees, it is used as... The ratio to 180 degrees, as The normalized value. Since noise events are usually unrelated to real motion, and in event accumulation images, pixels within the same motion region (cluster) typically have similar event count growth directions. This is because event triggering by the event camera depends on brightness changes, and the edge directions of objects within the same motion region are continuous, and the event flow direction caused by motion is consistent, resulting in a significant consistency in the event count growth direction within that region. Therefore, when a pixel is a blurry pixel, i.e., it has no event count growth direction, or when the event count growth direction of a pixel in a cluster differs significantly from that of other pixels, it indicates that the greater the disorder in the pixel's motion direction, the more likely it is to be a noise pixel.

[0100] For any cluster within the face region of any event accumulation map, the connected component formed by adjacent pixels in the cluster is denoted as the target connected component, thereby dividing each cluster into several target connected components.

[0101] The inversely proportional normalized value of the area of ​​each target connected region is used as the spatial isolation of each pixel within each target connected region.

[0102] The min-max normalization method is used to normalize the area of ​​all target connected components in the cumulative graph of all events to a value between 0 and 1. The difference between 1 and the normalized area of ​​each target connected component is used as the inversely proportional normalized area of ​​each target connected component.

[0103] It should be noted that in the event accumulation image, noise pixels are spatially distributed as isolated bright spots, meaning that the number of events for one or a few pixels is significantly higher than the surrounding pixels, but they do not form lines or edges, and are discontinuous with the real edges. In other words, noise pixels are not distributed along the facial contour, but are "scattered everywhere." Therefore, the smaller the area of ​​the connected region within the same motion region (cluster) in the event accumulation image, the greater the spatial isolation of each pixel within the connected region, and the more likely it is to be a noise pixel.

[0104] Within the face region of any event accumulation map, the mean of the disorder of motion direction and spatial isolation of each pixel is denoted as the noise spatial representation of each pixel.

[0105] Preset duration Let's take 5 as an example.

[0106] In the event accumulation graph time sequence, with the first... Time to the The cumulative graph of all events between time points constitutes the first... Local sequence of events in the cumulative graph at time step.

[0107] Among them, the The event accumulation graph local sequence at time t includes the t_t. Time, Number Time, Number Time, Number Time, Number Time, Number A cumulative event graph at any given moment.

[0108] It should be noted that in this embodiment, the event accumulation graph at each time point is enhanced with a delay of 100 milliseconds. For example, for the event accumulation graph at the last time point in the event accumulation graph time sequence, it is necessary to continue using the event camera to collect an event stream for another 100 milliseconds, obtain 5 event accumulation graphs, and then perform enhancement processing to ensure that the local sequence of the event accumulation graph at each time point can be obtained.

[0109] In the In the local sequence of event accumulation graphs at time points, the first event accumulation graph is denoted as the first updated event accumulation graph. The sum of the first and second event accumulation graphs is denoted as the second updated event accumulation graph. The sum of the first, second, and third event accumulation graphs is denoted as the third updated event accumulation graph, and so on, until the sum of all event accumulation graphs is denoted as the last updated event accumulation graph (the sixth updated event accumulation graph). The updated event accumulation graph sequence is constructed by using all the updated event accumulation graphs in sequence.

[0110] Image addition is a known technique. It involves summing the pixel values ​​of all pixels at the same coordinate position in the cumulative image of multiple events that need to be added, and using this sum as the pixel value of the pixel at that coordinate position in the added image, thus obtaining the added image.

[0111] In the Within the face region of the event accumulation map at any given time, any pixel is designated as the master pixel, and the coordinates of the master pixel are designated as the master coordinates.

[0112] Since the event accumulation map and the updated event accumulation map are the same size, in this embodiment, a rectangular coordinate system is formed with the lower left corner of the event accumulation map or the updated event accumulation map as the origin, with the horizontal direction to the right as the positive direction of the horizontal axis and the vertical direction upward as the positive direction of the vertical axis. The coordinate position of each pixel in the event accumulation map and the updated event accumulation map is obtained on the rectangular coordinate system.

[0113] In the sequence of updating the event accumulation graph, obtain the first... The maximum value among all pixel values ​​in the cumulative image of the Zhang Gengxin event is recorded as the standard pixel value. The ratio of the pixel value of a pixel at the principal coordinate position to the standard pixel value in the cumulative graph of the Zhang Gengxin event is denoted as the first pixel. The cumulative event representation at the main coordinate position within Zhang Gengxin's event cumulative graph.

[0114] In the sequence of updated event accumulation graphs, the cumulative event performance at the main coordinate position within all updated event accumulation graphs is obtained sequentially to form a sequence of event accumulation performance.

[0115] It should be noted that the positions of noise points in the event accumulation map are unstable across different time periods, while the positions of real edges are stable or exhibit regular movement. Therefore, over a long period, real edges become increasingly continuous with a higher number of events, while noise events typically do not consistently appear in the same location, resulting in a slight increase in the number of events after accumulation. In other words, as the event accumulation map accumulates over time, the number of events at the coordinates of pixels on real edges increases normally, indicating stable event accumulation, while the number of events at the coordinates of noise pixels increases slightly or not at all, showing a significant decreasing trend in event accumulation.

[0116] Using the least squares method, a straight line is fitted to the cumulative event sequence, and the inversely proportional normalized value of the slope of the fitted line is obtained as the first... Noise temporal representation of the main pixels in the face region of the event accumulation map at any given time.

[0117] It should be noted that the least squares method is a well-known technique, and its specific method will not be described here. The horizontal axis of the fitted line represents the ordinal value in the event accumulation sequence, and the vertical axis represents the event accumulation performance in the event accumulation sequence. Since each pixel in the face region of the event accumulation map at each time step corresponds to the slope of a fitted line, in this embodiment, the minimum-maximum normalization method is used to normalize the slope of the fitted line corresponding to all pixels in the face region of the event accumulation map at all time steps to between 0 and 1. Then, the difference between 1 and the normalized value of the fitted line slope is used as the inverse proportional normalized value of the fitted line slope. The smaller the slope of the fitted line, the greater the decreasing trend of event accumulation performance, and the more obvious the noise performance over time.

[0118] Within the face region of any event accumulation map, the average of the spatial and temporal noise representations of each pixel is denoted as the overall noise representation of each pixel.

[0119] The preset noise threshold is 0.8, and this will be used as an example for explanation.

[0120] Within the face region of any event cumulative image, the average noise performance of all pixels is obtained and recorded as the total noise performance value. All pixels with a noise performance value greater than a preset noise threshold are recorded as suspected noise pixels. The ratio of the number of suspected noise pixels to the number of pixels in the face region is recorded as the noise quantity ratio. The average of the total noise performance value and the noise quantity ratio is recorded as the noise interference factor of the face in any event cumulative image.

[0121] Step S004: Based on the magnitude of the facial detail feature saliency factor and the facial noise interference factor of each event cumulative image, obtain the adaptive gamma value of each event cumulative image; perform gamma correction on each event cumulative image based on the adaptive gamma value to obtain the enhanced event cumulative image; perform face recognition on the enhanced event cumulative image to obtain the face recognition result.

[0122] Preferably, in one embodiment of the present invention, the method for obtaining face recognition results includes:

[0123] For any event accumulation map, the face is affected by noise interference factors. inverse proportional value The mean of the significant factors of facial detail features is denoted as the gamma adjustment coefficient.

[0124] It should be noted that in the event accumulation map, noise typically manifests as pixels with an abnormally high event count; that is, noise is usually located in bright areas. Regarding gamma values... When the value is less than 1, the more significant the facial details in the event accumulation image and the less noise interference, the larger the gamma value is needed. This means that only a slight increase in dark details and suppression of bright noise are needed, with the focus on ensuring that the enhanced event accumulation image looks more natural. On the other hand, the less significant the facial details in the event accumulation image and the greater the noise interference, the smaller the gamma value is needed, with the focus on ensuring that the dark details in the enhanced event accumulation image are more obvious while the bright noise is suppressed.

[0125] The preset lower gamma limit is 0.2 and the preset upper gamma limit is 0.8. This will be used as an example for explanation.

[0126] The difference between the preset upper limit gamma value and the preset lower limit gamma value is recorded as the gamma adjustment value.

[0127] For any event accumulation graph, the sum of the product of the gamma adjustment coefficient and the gamma adjustment value and the preset gamma lower limit value is denoted as the adaptive gamma value.

[0128] Based on the adaptive gamma value of each event cumulative graph, gamma correction is performed on each event cumulative graph to obtain the enhanced event cumulative graph.

[0129] Gamma correction is a well-known technique, and its specific methods will not be described here.

[0130] Based on each enhanced event accumulation map, the face recognition result is obtained through event-driven CNN face recognition technology.

[0131] It should be noted that Event-Driven CNN Face Recognition is a well-known technique. Specifically, it uses an Event-Driven CNN (Convolutional Neural Network) to extract facial features from an enhanced event accumulation map. The extracted facial features are then compared with templates in a pre-set facial feature database to calculate similarity (e.g., cosine similarity, Euclidean distance) to complete identity recognition.

[0132] In this embodiment, a schematic diagram of the structure of the dynamic face recognition system for ultra-low light environments based on an event camera is shown below. Figure 2 As shown, Figure 2 It includes: an event data acquisition module, an event data enhancement module, and a face recognition module.

[0133] The event data acquisition module is used to acquire the event accumulation map time sequence during dynamic face recognition in ultra-low light environment;

[0134] The event data enhancement module is used to segment the facial region in each event cumulative image. Based on the difference in pixel values ​​between pixels within the facial region of each event cumulative image and adjacent pixels, as well as the distance between pixels with similar pixel values, it obtains the facial detail feature saliency factor for each event cumulative image. Based on the difference in pixel values ​​and positional distribution of pixels with similar pixel values ​​within the facial region of each event cumulative image, combined with the pixel value change of pixels at the same coordinate position during sequential accumulation of the event cumulative image, it obtains the noise spatial and temporal representation of each pixel within the facial region of each event cumulative image. Based on the noise spatial and temporal representation of all pixels within the facial region of each event cumulative image, it obtains the facial noise interference factor for each event cumulative image. Based on the magnitude of the facial detail feature saliency factor and the facial noise interference factor, it obtains the adaptive gamma value for each event cumulative image. Based on the adaptive gamma value, it performs gamma correction on each event cumulative image to obtain the enhanced event cumulative image.

[0135] The face recognition module is used to perform face recognition on the enhanced event accumulation map and obtain the face recognition results.

[0136] This invention is now complete.

[0137] In summary, in this embodiment of the invention, during dynamic face recognition in ultra-low light environments, a time sequence of event accumulation maps is acquired, the facial region in each event accumulation map is segmented, and the saliency factor of facial detail features and the noise interference factor of the face in each event accumulation map are obtained. This yields an adaptive gamma value for each event accumulation map, which is then used to perform gamma correction on each event accumulation map to obtain an enhanced event accumulation map. Face recognition is then performed on the enhanced event accumulation map to obtain the face recognition result. This invention improves the accuracy of face recognition by adaptively enhancing the event accumulation map.

[0138] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A dynamic face recognition method for ultra-low light environments based on event cameras, characterized in that, The method includes the following steps: In the process of dynamic face recognition under ultra-low light environment, the event accumulation map time sequence is obtained; For any event accumulation image, based on the pixel value of each pixel, the Otsu algorithm is used to obtain the optimal segmentation threshold. The region formed by all pixels with pixel values ​​greater than the segmentation threshold is recorded as the face region. Based on the difference in pixel value between each pixel in the face area and its neighboring pixels, the event count change value of each pixel is obtained to determine whether each pixel is a blurry pixel or a clear pixel, and then the direction of event count growth of clear pixels is obtained. Based on the event count change values ​​of all pixels in the face area and the number of clear pixels in the same event count growth direction, obtain the clarity of the face movement trajectory in any event cumulative image; Within the facial region, the absolute difference between the pixel values ​​of any two pixels is used as the clustering distance. A hierarchical clustering algorithm is used to cluster all pixels to obtain several clusters. Based on the pixel values ​​of the pixels in each cluster and the distance between the pixels, the facial edge structure clarity of any event accumulation image is obtained. The average of the clarity of the facial movement trajectory and the clarity of the facial edge structure in any event cumulative image is denoted as the facial detail feature significance factor of any event cumulative image. Based on the differences and positional distribution of pixels with similar pixel values ​​in the face region of each event cumulative image, and combined with the changes in pixel values ​​of pixels at the same coordinate position when the event cumulative images are accumulated sequentially, the spatial and temporal noise characteristics of each pixel in the face region of each event cumulative image are obtained. Within the facial region of any event accumulation image, the average of the spatial and temporal noise representations of each pixel is denoted as the overall noise representation of each pixel. The average noise performance of all pixels is obtained and recorded as the total noise performance value. All pixels with a noise performance greater than the preset noise threshold are recorded as suspected noise pixels. The ratio of the number of suspected noise pixels to the number of pixels in the face area is recorded as the noise quantity ratio. The average of the total noise performance value and the noise quantity ratio is recorded as the noise interference factor of the face in any event cumulative image. For any event cumulative image, the inverse proportional value of the noise interference factor on the face and the mean of the significant factor of facial detail features are denoted as the gamma adjustment coefficient. The difference between the preset upper limit gamma value and the preset lower limit gamma value is recorded as the gamma adjustment value. The product of the gamma adjustment coefficient and the gamma adjustment value of any event accumulation chart and the sum of the preset gamma lower limit value is denoted as the adaptive gamma value. Based on the adaptive gamma value, gamma correction is performed on each event cumulative map to obtain an enhanced event cumulative map. Face recognition is then performed on the enhanced event cumulative map to obtain the face recognition result.

2. The method for dynamic face recognition in ultra-low light environments based on event cameras according to claim 1, characterized in that, The specific steps for obtaining the event count change value of each pixel based on the difference in pixel value between each pixel in the face region and its adjacent pixels, to determine whether each pixel is a blurry pixel or a clear pixel, and then obtaining the direction of event count growth for clear pixels, are as follows: In any event accumulation map, any pixel in the face area is recorded as the target pixel, and all pixels adjacent to the target pixel are recorded as reference pixels. The maximum value of the absolute difference between the pixel values ​​of the target pixel and all reference pixels is recorded as the event count change value of the target pixel. When the event count change value of the target pixel is 0, the target pixel is recorded as a blurred pixel. When the event count change value of the target pixel is not 0, the target pixel is recorded as a clear pixel. For clear pixels, the reference pixel corresponding to the change value of the event count is recorded as the marker reference pixel; If the pixel value of the clear pixel is greater than that of the marker reference pixel, then the direction from the marker reference pixel to the clear pixel is recorded as the direction of the increase in the number of events of the clear pixel. If the pixel value of the clear pixel is less than that of the marked reference pixel, then the direction from the clear pixel to the marked reference pixel is recorded as the direction of increase in the number of events for the clear pixel.

3. The method for dynamic face recognition in ultra-low light environments based on event cameras according to claim 1, characterized in that, The specific steps for obtaining the clarity of facial movement trajectory from any event cumulative image are as follows: Within the facial region of any event accumulation image, the mean of the normalized values ​​of the event count changes of all pixels is obtained and recorded as the overall event count change amplitude. All clear pixels with the same event count growth direction are formed into a direction set. The number of clear pixels in each direction set is counted. The maximum value of the number of clear pixels in all direction sets is recorded as the target number. The ratio of the target number to the number of pixels in the facial region is recorded as the consistency of the event count growth direction. The mean of the overall event count change amplitude and the consistency of the event count growth direction is recorded as the facial movement trajectory clarity of any event accumulation image.

4. The method for dynamic face recognition in ultra-low light environments based on event cameras according to claim 1, characterized in that, The specific steps for obtaining the facial edge structure clarity of any event accumulation image are as follows: The mean value of all pixels in each cluster is recorded as the importance of each cluster. The inversely proportional normalized value of the mean Euclidean distance between any two pixels in each cluster is denoted as the spatial centrality of each cluster. The sum of the importance of all clusters is obtained and denoted as the total importance. The ratio of the importance of each cluster to the total importance is used as the weight of each cluster. The spatial concentration of all clusters is weighted and summed to obtain the facial edge structure clarity of any event accumulation map.

5. The method for dynamic face recognition in ultra-low light environments based on event cameras according to claim 1, characterized in that, The specific steps for obtaining the spatial and temporal noise representations of each pixel within the facial region of each event cumulative image are as follows: For any pixel in any cluster, if the pixel is a blurry pixel, then let the disorder of the pixel's motion direction be a preset constant; if the pixel is a sharp pixel, then the normalized value of the mean of the minimum angle between the pixel in the cluster and the direction of the increase of the number of events of all other sharp pixels is denoted as the disorder of the pixel's motion direction. The connected component formed by adjacent pixels in each cluster is denoted as the target connected component. Based on the disorder of the motion direction of each pixel in each cluster and the area of ​​each target connected region, the noise spatial representation of each pixel in the face region of any event accumulation map is obtained. With a preset duration, in the event accumulation graph time sequence, the event accumulation graph local sequence at time 1 is constructed from all event accumulation graphs between time 1 and time 2. Based on the pixel value changes of pixels at the same coordinate position when the event accumulation map is accumulated sequentially in the local sequence of the event accumulation map, the noise temporal performance of each pixel in the face region of the event accumulation map at the first moment is obtained.

6. The method for dynamic face recognition in ultra-low light environments based on event cameras according to claim 5, characterized in that, The specific steps for obtaining the noise spatial representation of each pixel within the facial region of any event accumulation image are as follows: The inversely proportional normalized value of the area of ​​each target connected region is used as the spatial isolation of each pixel within each target connected region. Within the face region of any event accumulation map, the mean of the disorder of motion direction and spatial isolation of each pixel is denoted as the noise spatial representation of each pixel.

7. The method for dynamic face recognition in ultra-low light environments based on event cameras according to claim 5, characterized in that, The specific steps for obtaining the noise temporal representation of each pixel within the face region of the event accumulation map at time step 1 are as follows: In the local sequence of event accumulation graphs, the first event accumulation graph is denoted as the first updated event accumulation graph, the sum of the first and second event accumulation graphs is denoted as the second updated event accumulation graph, and so on, until the sum of all event accumulation graphs is denoted as the last updated event accumulation graph, thus forming an updated event accumulation graph sequence. In the face region of the event accumulation map at time step 1, any pixel is designated as the master pixel, and the coordinate position of the master pixel is designated as the master coordinate position. Get the maximum value among all pixels in the i-th update event cumulative image, and record it as the standard pixel value. Record the ratio of the pixel value of the pixel at the main coordinate position in the i-th update event cumulative image to the standard pixel value as the event cumulative performance at the main coordinate position in the i-th update event cumulative image. In the sequence of updated event accumulation graphs, the cumulative event performance at the main coordinate position within all updated event accumulation graphs is obtained sequentially to form a sequence of event accumulation performance. The least squares method is used to fit a straight line to the event accumulation sequence, and the slope of the fitted line is obtained as the inverse proportional normalized value, which is used as the noise temporal representation of the main pixels in the face region of the event accumulation map at time step i.

Images