Three-channel event determination method, device, equipment, medium and product based on motion dynamic characteristics

Abstract

This application discloses a method, apparatus, device, medium, and product for determining three-channel events based on motion dynamic characteristics, relating to the field of data processing. The method includes: acquiring an event sequence; the event sequence is determined by multiple event information; each event information includes pixel position, timestamp of the event occurrence, and brightness change polarity information; encoding the event sequence to obtain a three-channel event; the three-channel event includes a three-channel structured feature map; the three-channel event is used for visual motion recognition; wherein the three-channel structured feature map includes: a first channel for dynamic rhythm encoding, a second channel for motion stability encoding, and a third channel for brightness change direction encoding. This application can improve the recognition accuracy of motion dynamic characteristics and enhance its versatility and robustness.

Description

Technical Field

[0001] This application relates to the field of data processing, and in particular to a method, apparatus, device, medium, and product for determining three-channel events based on motion dynamic characteristics. Background Technology

[0002] Event cameras, as biomimetic asynchronous vision sensors, capture only brightness changes (i.e., events) rather than full-frame images. They offer significant advantages such as microsecond-level temporal resolution, wide dynamic range, and low power consumption, making them suitable for action-related visual tasks such as gait recognition and sign language recognition. The accuracy of downstream recognition models highly depends on the representation of event data. Effective event representation requires preserving key spatiotemporal and semantic information of the action, but the following solutions have significant shortcomings:

[0003] The EventImage class methods generate frame-like images by accumulating events within a fixed time window. While compatible with traditional CNNs, they are prone to temporal aliasing, resulting in blurring of fast movements and loss of fine-grained temporal order, making it impossible to capture the rhythmic changes of repetitive human movements.

[0004] The VoxelGrid method extends events into 3D spatiotemporal volumes, which can retain some temporal information, but it has high computational and memory costs and does not clearly distinguish event polarity (including subtle interaction clues between human movement and the environment).

[0005] TORE-type methods reduce memory consumption through compact spatiotemporal encoding, but focus on structural compression rather than extracting action-specific dynamics, ignoring the rhythmic structure of actions and polarity-driven brightness trends, making it difficult to distinguish subtle gestures or gait patterns.

[0006] Current methods generally lack a design centered on motion characteristics, failing to meet the needs of motion-related tasks for accurate and efficient event representation. Therefore, there is an urgent need for an event representation solution that balances motion capture and lightweight deployment. Summary of the Invention

[0007] The purpose of this application is to provide a three-channel event determination method, device, equipment, medium, and product based on motion dynamic characteristics, which can improve the recognition accuracy of motion dynamic characteristics and enhance versatility and robustness.

[0008] To achieve the above objectives, this application provides the following solution:

[0009] In a first aspect, this application provides a three-channel event determination method based on motion dynamic characteristics, wherein the three-channel event determination method based on motion dynamic characteristics is applied to an event camera, which is used as a biomimetic asynchronous vision sensor; the event camera is used to capture event information generated by changes in brightness.

[0010] The three-channel event determination method based on motion dynamic characteristics includes:

[0011] Obtain an event sequence; the event sequence is determined by multiple event information; each event information includes pixel position, timestamp of the event occurrence, and polarity information of brightness change;

[0012] The event sequence is encoded to obtain a three-channel event; the three-channel event includes a three-channel structured feature map; the three-channel event is used for visual action recognition; wherein, the three-channel structured feature map includes: a first channel for dynamic rhythm encoding, a second channel for motion stability encoding, and a third channel for brightness change direction encoding.

[0013] In one embodiment, the event sequence is encoded to obtain a three-channel event, specifically including:

[0014] The event sequence is subjected to dynamic rhythm encoding to determine a first channel; the first channel is an array matrix containing the first pixel value corresponding to the pixel position of each event.

[0015] The event sequence is subjected to motion stability encoding to determine the second channel; the second channel is an array matrix containing the second pixel values ​​corresponding to the pixel positions of each event.

[0016] The event sequence is encoded with brightness change direction to determine the third channel; the third channel is an array matrix containing the third pixel value corresponding to the pixel position of each event.

[0017] The first, second, and third channels are combined using a pixel value matrix to obtain a three-channel structured feature map.

[0018] The three-channel structured feature map is identified as a three-channel event.

[0019] In one embodiment, dynamic rhythm encoding is performed on the event sequence to determine a first channel, specifically including:

[0020] For each pixel position of the event sequence Extract pixel position All event information at the location is used to calculate the time interval between consecutive events and form a time interval set; the expression for the time interval is:

[0021] ;

[0022] The expression for the set of time intervals is:

[0023] ;

[0024] The first pixel value is calculated based on a set of time intervals, and the first channel is determined; the expression corresponding to the first pixel value is:

[0025] ;

[0026] in, For the first The time interval corresponding to each event; For the first The timestamp of the event; For the first The timestamp of the event; A set of time intervals; This is the time interval corresponding to the first event; This is the time interval corresponding to the second event; For the first The time interval corresponding to each event; The value of the first pixel; To take the median; To obtain the standard deviation.

[0027] In one embodiment, motion stability encoding is performed on the event sequence to determine the second channel, specifically including:

[0028] Determine the positive polarity event in the event sequence; the positive polarity event is determined based on the value of the brightness change polarity information; the value of the brightness change polarity information corresponding to the positive polarity event is 1;

[0029] Based on each pixel position of the event sequence For pixel position For positive polarity events, determine the variance of the time interval;

[0030] The second pixel value is calculated based on the variance of the time interval, and the second channel is determined; the expression corresponding to the second pixel value is:

[0031] ;

[0032] in, The second pixel value; pixel position The number of events corresponding to the event information; For the first The time interval corresponding to each event; The mean of the set of time intervals; The variance of the time interval; For serial numbers.

[0033] In one embodiment, the event sequence is encoded with brightness change direction to determine the third channel, specifically including:

[0034] The positive and negative polarity events in the event sequence are determined; both the positive and negative polarity events are determined based on the values ​​of brightness change polarity information; the value of the brightness change polarity information corresponding to the positive polarity event is 1; the value of the brightness change polarity information corresponding to the positive polarity event is -1.

[0035] Statistical analysis of each pixel position in the event sequence Number of positive polarity events Number of negative polarity events and total number of events And calculate the balance coefficient; the expression for the balance coefficient is:

[0036] ;

[0037] The third pixel value is calculated based on the balance coefficient, and the third channel is determined; the expression corresponding to the third pixel value is:

[0038] ;

[0039] in, This is the balance coefficient; This is the value of the third pixel.

[0040] In one embodiment, the three-channel events are input into the action recognition model, and the action recognition results are output.

[0041] Secondly, this application provides a three-channel event determination device based on motion dynamic characteristics, comprising:

[0042] An event sequence acquisition module is used to acquire an event sequence; the event sequence is determined by multiple event information; each event information includes pixel position, timestamp of the event occurrence, and brightness change polarity information;

[0043] The encoding processing module is used to encode the event sequence to obtain a three-channel event; the three-channel event includes a three-channel structured feature map; the three-channel event is used for visual action recognition; wherein, the three-channel structured feature map includes: a first channel for dynamic rhythm encoding, a second channel for motion stability encoding, and a third channel for brightness change direction encoding.

[0044] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described three-channel event determination method based on dynamic characteristics of actions.

[0045] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned three-channel event determination method based on dynamic action characteristics.

[0046] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the three-channel event determination method based on dynamic action characteristics described above.

[0047] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0048] This application provides a method, apparatus, device, medium, and product for determining three-channel events based on motion dynamic characteristics. By acquiring an event sequence and using it as the processing object, a three-channel structured feature map is generated through co-coding. Visual motion recognition is then performed based on the three-channel events. The event sequence in this application is determined by multiple event information items; each event information item includes pixel position, timestamp of the event occurrence, and brightness change polarity information. The three-channel structured feature map obtained by encoding the event sequence includes: a first channel for dynamic rhythm encoding, a second channel for motion stability encoding, and a third channel for brightness change direction encoding. The three-channel structured feature map enables a representation transformation from data-centric to motion-characteristic-centric, improving the recognition accuracy of motion dynamic characteristics. Furthermore, it is adaptable to various motion-related recognition tasks, thereby enhancing versatility and robustness. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 A flowchart of a three-channel event determination method based on motion dynamic characteristics;

[0051] Figure 2 A structural diagram of a three-channel event determination device based on motion dynamic characteristics;

[0052] Figure 3This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0053] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0054] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] In one exemplary embodiment, a three-channel event determination method based on motion dynamic characteristics is provided, which is applied to an event camera that serves as a biomimetic asynchronous vision sensor; the event camera is used to capture event information generated by changes in brightness.

[0056] like Figure 1 As shown, the three-channel event determination method based on motion dynamic characteristics includes:

[0057] Step 100: Obtain the event sequence. The event sequence is determined by multiple event information; each event information includes pixel position, timestamp of the event occurrence, and polarity information of brightness change.

[0058] Step 200: Encode the event sequence to obtain a three-channel event. The three-channel event includes a three-channel structured feature map; the three-channel event is used for visual action recognition; wherein, the three-channel structured feature map includes: a first channel for dynamic rhythm encoding, a second channel for motion stability encoding, and a third channel for brightness change direction encoding.

[0059] In one embodiment, the event sequence is encoded to obtain a three-channel event, specifically including:

[0060] Step 201: Perform dynamic rhythm encoding on the event sequence to determine the first channel; the first channel is an array matrix containing the first pixel value corresponding to the pixel position of each event.

[0061] As an optional implementation, dynamic rhythm encoding is performed on the event sequence to determine the first channel, specifically including:

[0062] For each pixel position in the event sequence Extract pixel position All event information at the location is used to calculate the time interval between consecutive events and form a time interval set; the expression for the time interval is:

[0063] .

[0064] The expression for the set of time intervals is:

[0065] .

[0066] The first pixel value is calculated based on a set of time intervals, and the first channel is determined; the expression corresponding to the first pixel value is:

[0067] .

[0068] in, For the first The time interval corresponding to each event; For the first The timestamp of the event; For the first The timestamp of the event; A set of time intervals; This is the time interval corresponding to the first event; This is the time interval corresponding to the second event; For the first The time interval corresponding to each event; The value of the first pixel; To take the median; To obtain the standard deviation.

[0069] Step 202: Perform motion stability encoding on the event sequence to determine the second channel; the second channel is an array matrix containing the second pixel values ​​corresponding to the pixel positions of each event.

[0070] As an optional implementation, motion stability encoding is performed on the event sequence to determine the second channel, specifically including:

[0071] Identify the positive polarity event in the event sequence; the positive polarity event is determined based on the value of the brightness change polarity information; the value of the brightness change polarity information corresponding to the positive polarity event is 1.

[0072] Based on each pixel position of the event sequence For pixel position For positive polarity events, determine the variance of the time interval.

[0073] The second pixel value is calculated based on the variance of the time interval, and the second channel is determined; the expression corresponding to the second pixel value is:

[0074] .

[0075] in, The second pixel value; pixel position The number of events corresponding to the event information; For the first The time interval corresponding to each event; The mean of the set of time intervals; The variance of the time interval; For serial numbers.

[0076] Step 203: Encode the brightness change direction of the event sequence to determine the third channel; the third channel is an array matrix containing the third pixel value corresponding to the pixel position of each event.

[0077] As an optional implementation, the event sequence is encoded with brightness change direction to determine the third channel, specifically including:

[0078] Determine the positive and negative polarity events in the event sequence; both positive and negative polarity events are determined based on the value of the brightness change polarity information; the value of the brightness change polarity information corresponding to a positive polarity event is 1; the value of the brightness change polarity information corresponding to a positive polarity event is -1.

[0079] Statistical event sequence for each pixel position Number of positive polarity events Number of negative polarity events and total number of events And calculate the balance coefficient; the expression for the balance coefficient is:

[0080] .

[0081] The third pixel value is calculated based on the balance coefficient, and the third channel is determined; the expression corresponding to the third pixel value is:

[0082] .

[0083] in, This is the balance coefficient; This is the value of the third pixel.

[0084] Step 204: Perform pixel value matrix combination processing on the first channel, the second channel, and the third channel to obtain a three-channel structured feature map.

[0085] Step 205: Determine the three-channel structured feature map as a three-channel event.

[0086] As an optional implementation, the three-channel event determination method based on motion dynamic characteristics further includes:

[0087] The three-channel event is input into the action recognition model, and the action recognition result is output. The action recognition model can be a CNN model, and the action recognition process can be implemented using existing solutions, which will not be elaborated here.

[0088] This application encodes the statistical characteristics (dynamic rhythm) of event time intervals, the temporal consistency of positive events (motion stability), and the ratio of positive and negative events (direction of brightness change) through three complementary channels, thereby realizing the representation conversion from "data form-centered" to "action characteristic-centered" and adapting to a variety of action-related recognition tasks.

[0089] Using the asynchronous event stream captured by the event camera as the processing object, a structured event representation is generated through three channels of collaborative encoding. The specific steps are as follows:

[0090] Event Data: Acquire the event sequence output by the event camera. Each event , For pixel position, The timestamp of the event. The polarity of brightness change (1 indicates an increase in brightness, and -1 indicates a decrease in brightness).

[0091] First channel (dynamic rhythm coding) calculation: for each pixel position Extract all events that occur at this location. Calculate the time interval of consecutive events Forming a set of time intervals ; Pixel value of this channel The median captures the central tendency of the time interval, while the standard deviation reflects the degree of dispersion; together they characterize the dynamic rhythm of the frequency of events occurring at a pixel.

[0092] Second channel (motion stability coding) calculation: for each pixel position For positive polarity events, calculate the variance of their time intervals, and the pixel value of that channel. The lower the variance, the more stable the motion; this channel is used to filter regions with stable motion patterns.

[0093] Third channel (brightness change direction encoding) calculation: Statistical analysis of each pixel position Number of positive polarity events Number of negative polarity events and total number of events Calculate the balance coefficient ; Pixel value of this channel A value close to 1 indicates that the brightness is continuously increasing, a value close to 0 indicates that the brightness is continuously decreasing, and a value close to 0.5 indicates that the brightness changes alternately.

[0094] Generate the final event representation: Combine the pixel value matrices of the three channels to form a three-channel structured feature map, which serves as the input to the downstream action recognition model.

[0095] The benefits of this application are:

[0096] High recognition accuracy: Thanks to the three-channel collaborative coding design, it accurately captures the dynamic rhythm of movements, motion stability, and direction of brightness changes. The accuracy rate reaches 97.81% in the ASL (American Sign Language) recognition task and 99.67% in the gait user recognition task, which is significantly better than existing methods such as EventImage (93.25% / 97.50%), VoxelGrid (94.43% / 96.67%), and TORE (95.21% / 94.50%).

[0097] High computational efficiency: It adopts lightweight statistical computing logic, which does not require complex spatiotemporal volume construction or deep learning encoding. The average processing time per sample is less than 0.1 seconds (72.09ms for ASL data and 74.78ms for gait data), meeting the needs of real-time applications.

[0098] High versatility: The coding design based on action characteristics does not require structural adjustments for specific tasks. It can be adapted to various action-related scenarios such as sign language recognition and gait recognition, and is robust to environmental interference such as changes in lighting and distance differences.

[0099] Excellent detail retention: By statistical analysis of time intervals and polarity ratios, fine-grained motion dynamics are effectively preserved, avoiding the time aliasing and detail loss problems of traditional methods.

[0100] The precision advantage stems from the complementary design of the three channels: the first channel captures changes in the rhythm of the action, the second channel filters motion noise, and the third channel reflects the changes in brightness due to the interaction between the environment and the action, comprehensively covering the core features required for action recognition.

[0101] The efficiency advantage comes from the concise statistical computing paradigm, which avoids the 3D tensor storage of VoxelGrid class methods and the redundant event accumulation of EventImage class methods, achieving a balance between coding efficiency and computational cost.

[0102] This application encodes an event stream, that is, a series of events (event sequences) arranged in chronological order (timestamps), to obtain a three-channel event, i.e., a three-channel structured feature map. It converts the event stream into a three-channel format, which can also be understood as converting the event stream into a three-channel (first channel, second channel, and third channel) format that can be processed by a CNN model, facilitating subsequent action recognition. In other words, the first, second, and third channels can be understood as the RGB channels currently used in digital image processing for storing and representing color information.

[0103] This application also provides an application scenario in which the above-described three-channel event determination method based on motion dynamic characteristics is applied. Specifically, the three-channel event determination method based on motion dynamic characteristics provided in this embodiment can be applied to action recognition scenarios. A three-channel structured feature map is determined according to the three-channel event determination method based on motion dynamic characteristics mentioned in this application. Then, the three-channel structured feature map is input into the action recognition model for visual action recognition processing to obtain the action recognition result.

[0104] Based on the same inventive concept, this application also provides a three-channel event determination device based on motion dynamic characteristics for implementing the above-mentioned three-channel event determination method based on motion dynamic characteristics. The solution provided by this device is similar to the solution described in the above-described method. Therefore, the specific limitations of one or more embodiments of the three-channel event determination device based on motion dynamic characteristics provided below can be found in the limitations of the three-channel event determination method based on motion dynamic characteristics described above, and will not be repeated here.

[0105] In one exemplary embodiment, such as Figure 2 As shown, a three-channel event determination device based on motion dynamic characteristics is provided, comprising:

[0106] The event sequence acquisition module is used to acquire event sequences; the event sequence is determined by multiple event information; each event information includes pixel position, timestamp of the event occurrence, and polarity information of brightness change.

[0107] The encoding processing module is used to encode the event sequence to obtain a three-channel event. The three-channel event includes a three-channel structured feature map. The three-channel event is used for visual action recognition. The three-channel structured feature map includes a first channel for dynamic rhythm encoding, a second channel for motion stability encoding, and a third channel for brightness change direction encoding.

[0108] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 3As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores three-channel event determination data based on dynamic action characteristics. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a three-channel event determination method based on dynamic action characteristics.

[0109] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0110] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0111] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0112] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0113] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0114] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0115] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logic devices, etc., and are not limited to these.

[0116] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0117] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A three-channel event determination method based on motion dynamic characteristics, characterized in that, The three-channel event determination method based on motion dynamic characteristics is applied to an event camera, which serves as a biomimetic asynchronous vision sensor. The event camera is used to capture event information generated by changes in brightness; The three-channel event determination method based on motion dynamic characteristics includes: Obtain an event sequence; the event sequence is determined by multiple event information; each event information includes pixel position, timestamp of the event occurrence, and polarity information of brightness change; The event sequence is encoded to obtain a three-channel event; the three-channel event includes a three-channel structured feature map; the three-channel event is used for visual action recognition; wherein, the three-channel structured feature map includes: a first channel for dynamic rhythm encoding, a second channel for motion stability encoding, and a third channel for brightness change direction encoding; For each pixel position of the event sequence Extract pixel position All event information at the location is used to calculate the time interval between consecutive events and form a time interval set; The first pixel value is calculated based on the time interval set, and the first channel is determined; Determine the positive polarity event in the event sequence; the positive polarity event is determined based on the value of the brightness change polarity information; the value of the brightness change polarity information corresponding to the positive polarity event is 1; Based on each pixel position of the event sequence For pixel position For positive polarity events, determine the variance of the time interval; The second pixel value is calculated based on the variance of the time interval, and the second channel is determined. The positive and negative polarity events in the event sequence are determined; both the positive and negative polarity events are determined based on the value of the brightness change polarity information; the value of the brightness change polarity information corresponding to the negative polarity event is -1; Statistical analysis of each pixel position in the event sequence Number of positive polarity events Number of negative polarity events and total number of events And calculate the balance coefficient; The third pixel value is calculated based on the balance coefficient, and the third channel is determined. The pixel value of the first channel represents the dynamic rhythm of the frequency of events occurring at the pixel; the second channel is used to filter areas with stable motion patterns; the pixel value of the third channel is close to 1 to indicate that the brightness is continuously increasing, close to 0 to indicate that the brightness is continuously decreasing, and close to 0.5 to indicate that the brightness changes alternately.

2. The three-channel event determination method based on motion dynamic characteristics according to claim 1, characterized in that, The event sequence is encoded to obtain a three-channel event, specifically including: The event sequence is subjected to dynamic rhythm encoding to determine a first channel; the first channel is an array matrix containing the first pixel value corresponding to the pixel position of each event. The event sequence is subjected to motion stability encoding to determine the second channel; the second channel is an array matrix containing the second pixel values ​​corresponding to the pixel positions of each event. The event sequence is encoded with brightness change direction to determine the third channel; the third channel is an array matrix containing the third pixel value corresponding to the pixel position of each event. The first, second, and third channels are combined using a pixel value matrix to obtain a three-channel structured feature map. The three-channel structured feature map is identified as a three-channel event.

3. The three-channel event determination method based on motion dynamic characteristics according to claim 2, characterized in that, The expression for the time interval is: ； The expression for the time interval set is: ； The expression corresponding to the first pixel value is: ； in, For the first The time interval corresponding to each event; For the first The timestamp of the event; For the first The timestamp of the event; A set of time intervals; This is the time interval corresponding to the first event; This is the time interval corresponding to the second event; For the first The time interval corresponding to each event; The value of the first pixel; To take the median; To obtain the standard deviation.

4. The three-channel event determination method based on motion dynamic characteristics according to claim 2, characterized in that, The expression corresponding to the second pixel value is: ； in, The second pixel value; pixel position The number of events corresponding to the event information; For the first The time interval corresponding to each event; The mean of the set of time intervals; The variance of the time interval; For serial numbers.

5. The three-channel event determination method based on motion dynamic characteristics according to claim 2, characterized in that, The expression for the balance coefficient is: ； The expression corresponding to the third pixel value is: ； in, This is the balance coefficient; This is the value of the third pixel.

6. The three-channel event determination method based on motion dynamic characteristics according to claim 1, characterized in that, Also includes: The three-channel events are input into the action recognition model, and the action recognition results are output.

7. A three-channel event determination device based on motion dynamic characteristics, characterized in that, include: The event sequence acquisition module is used to acquire event sequences. The event sequence is determined by multiple event information; each event information includes pixel position, timestamp of the event occurrence, and polarity information of brightness change; The encoding processing module is used to encode the event sequence to obtain three-channel events; The three-channel event includes: a three-channel structured feature map; the three-channel event is used for visual action recognition; wherein, the three-channel structured feature map includes: a first channel for dynamic rhythm encoding, a second channel for motion stability encoding, and a third channel for brightness change direction encoding; For each pixel position of the event sequence Extract pixel position All event information at the location is used to calculate the time interval between consecutive events and form a time interval set; The first pixel value is calculated based on the time interval set, and the first channel is determined; Determine the positive polarity event in the event sequence; the positive polarity event is determined based on the value of the brightness change polarity information; the value of the brightness change polarity information corresponding to the positive polarity event is 1; Based on each pixel position of the event sequence For pixel position For positive polarity events, determine the variance of the time interval; The second pixel value is calculated based on the variance of the time interval, and the second channel is determined. The positive and negative polarity events in the event sequence are determined; both the positive and negative polarity events are determined based on the value of the brightness change polarity information; the value of the brightness change polarity information corresponding to the negative polarity event is -1; Statistical analysis of each pixel position in the event sequence Number of positive polarity events Number of negative polarity events and total number of events And calculate the balance coefficient; The third pixel value is calculated based on the balance coefficient, and the third channel is determined. The pixel value of the first channel represents the dynamic rhythm of the frequency of events occurring at the pixel; the second channel is used to filter areas with stable motion patterns; the pixel value of the third channel is close to 1 to indicate that the brightness is continuously increasing, close to 0 to indicate that the brightness is continuously decreasing, and close to 0.5 to indicate that the brightness changes alternately.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the three-channel event determination method based on motion dynamic characteristics as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the three-channel event determination method based on the dynamic characteristics of motion as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the three-channel event determination method based on the dynamic characteristics of motion as described in any one of claims 1-6.

Images