High-speed transaction target identification method and device, storage medium and electronic equipment

Abstract

The invention discloses a high-speed transaction target identification method and device, a storage medium and electronic equipment, and relates to the technical field of information. The method comprises the following steps: acquiring original event stream data acquired by a dynamic vision sensor in a continuous time period; based on a preset space-time filtering kernel, performing noise reduction processing on the original event stream data to obtain event stream data after noise reduction processing; according to the polarity corresponding to each de-noised event in the de-noised event stream data, mapping each de-noised event to a corresponding image channel to obtain polarity channel image frame data; respectively inputting the polarity channel image frame data into a preset target detection model and a preset target segmentation model for processing to obtain a target detection result and a target segmentation result; and determining a transaction target identification result according to the target detection result and the target segmentation result. The problems of high power consumption, large data volume and complex detection system structure can be solved.

Description

Technical Field

[0001] The present application relates to the field of information technology, and in particular to a high-speed moving target recognition method, device, storage medium and electronic equipment. Background Art

[0002] Real-time recognition of high-speed object movements has important application value in scenarios such as traffic management, autonomous driving, aircraft navigation and obstacle avoidance, production line quality inspection, and sports event analysis.

[0003] At present, high-speed cameras are usually used for image capture. High-speed cameras can shoot at extremely high frame rates and capture high-speed motion details that cannot be seen by the naked eye, thereby providing clear image sequences for subsequent analysis. However, due to the large amount of instantaneous data from high-speed cameras, usually above 10MB / s, and high power consumption, typically 30-50W or even higher, they can only work for a short time due to this feature, and the requirements for back-end data transmission bandwidth and data processing capabilities are also relatively high. They can only be processed on servers or desktop-level hosts, resulting in the overall complexity of the detection system. Summary of the invention

[0004] In view of this, the present application provides a high-speed moving target recognition method, device, storage medium and electronic device, the main purpose of which is to solve the problems of high power consumption, large data volume and complex detection system structure.

[0005] According to the first aspect of the present application, a high-speed moving target recognition method is provided, which is applied to an edge computing platform, and the method includes: Obtaining raw event stream data collected by dynamic vision sensors in continuous time periods; Based on a preset spatiotemporal filter kernel, detecting whether there is a target event within the spatiotemporal neighborhood of any event in the original event stream data, and performing noise reduction processing on the original event stream data according to the detection result to obtain event stream data after noise reduction processing; Determine the polarity corresponding to each denoised event in the denoised event stream data, and map each denoised event to a corresponding image channel according to the polarity to obtain polarity channel image frame data; Inputting the polarity channel image frame data into a preset target detection model and a preset target segmentation model for processing respectively, to obtain a target detection result and a target segmentation result; A moving target recognition result is determined according to the target detection result and the target segmentation result.

[0006] According to a second aspect of the present application, a high-speed moving target recognition device is provided, the device comprising: An acquisition unit, used for acquiring raw event stream data collected by a dynamic vision sensor in a continuous period of time; A denoising unit, configured to detect whether there is a target event within the spatiotemporal neighborhood of any event in the original event stream data based on a preset spatiotemporal filter kernel, and to perform denoising processing on the original event stream data according to the detection result to obtain the denoised event stream data; A mapping unit, used to determine the polarity corresponding to each denoised event in the denoised event stream data, and map each denoised event to a corresponding image channel according to the polarity to obtain polarity channel image frame data; A detection and segmentation unit, used for inputting the polarity channel image frame data into a preset target detection model and a preset target segmentation model for processing, to obtain a target detection result and a target segmentation result; A determination unit is used to determine a moving target recognition result based on the target detection result and the target segmentation result.

[0007] According to a third aspect of the present application, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned high-speed moving target recognition method is implemented.

[0008] According to the fourth aspect of the present application, an electronic device is provided, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the above-mentioned high-speed moving target recognition method when executing the program.

[0009] By means of the above technical scheme, the present application provides a method, device, storage medium and electronic device for high-speed moving target recognition. Compared with the prior art, the present application can obtain the original event stream data collected by the dynamic visual sensor in a continuous period, and then perform noise reduction on the original event stream data, and convert the noise-reduced event stream data into polarity channel image frame data, and then perform target recognition and target segmentation on the polarity channel image frame data respectively, and finally determine the moving target recognition result according to the target detection result and the target segmentation result. It can be seen that the present application uses a low-power dynamic visual sensor to capture high-speed object motion and picture change information, which can realize motion data collection with extremely low power consumption and extremely low data rate, and can be input into the embedded edge computing platform in the form of event stream. The embedded edge computing platform of the present application has AI computing power and good portability, so that a high-speed moving target detection system with low power consumption, miniaturization and lightweight deployment at the edge can be built. In addition, the present application can continuously track high-speed moving targets, and can realize real-time, video visualization and continuous detection and segmentation of high-speed moving targets.

[0010] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A schematic diagram of a high-speed moving target recognition method provided in an embodiment of the present application is shown; Figure 2 A schematic diagram of the structure of a high-speed moving target detection system provided in an embodiment of the present application is shown; Figure 3 A schematic diagram of a noise reduction process provided in an embodiment of the present application is shown; Figure 4 The overall process diagram of high-speed moving target recognition provided by the embodiment of the present application is shown; Figure 5 A schematic structural diagram of a high-speed moving target recognition device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0012] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0013] The existing solutions have high power consumption, large data volume, and the resulting problems of weak data processing capabilities and complex systems.

[0014] In order to solve the above problems, an embodiment of the present invention provides a high-speed moving target recognition method, which is applied to an edge computing platform, such as Figure 1 As shown, the method includes: Step 10: Obtain the original event stream data collected by the dynamic vision sensor in a continuous period of time.

[0015] The dynamic visual sensor includes an event camera, and the raw event stream data includes the spatial coordinates of the data points, label information, timestamp, and the number of data points.

[0016] In order to solve the problems of high power consumption, large data volume and complex detection system structure in the prior art, the embodiment of the present invention provides a high-speed moving target detection system, such as Figure 2As shown, it includes a low-power dynamic vision sensor, an embedded GPU edge computing platform and a visualization platform, wherein the dynamic vision sensor is a low-power event camera. The main function of the event camera is to capture high-speed object motion and picture change information. It has good sensitivity to dynamic changes, and can achieve motion data collection with extremely low power consumption and extremely low data rate, and input the collected data in the form of event stream to the embedded GPU edge computing platform for processing; the embedded GPU edge computing platform has AI computing power and good portability, and its main function is to perform real-time target detection and target segmentation based on dynamic visual event stream data; the visualization platform is a set of software algorithms that can display detection results and segmentation results in the form of continuous frame images.

[0017] When the dynamic vision sensor based on the above system performs data collection, the spatial coordinates, timestamp and label information of each event collected by the dynamic vision sensor in a continuous time period are obtained, and the original event stream data is determined based on the spatial coordinates, timestamp and label information of each event.

[0018] Among them, each event corresponds to a data point, and the label information is used to indicate the direction of the change in light intensity. For example, the label information is represented by positive and negative. When the label information is positive, it indicates that the light intensity at the corresponding spatial position is increased; when the label information is negative, it indicates that the light intensity at the corresponding spatial position is decreased.

[0019] Specifically, based on the preset integration time, the collected data for a period of time is accumulated to form the original event stream data (x, y, t, l) N0, where x and y represent the spatial coordinates of the data point, l represents the label information of the data point, t represents the timestamp of the data point, and N0 represents the number of data points, where each data point in the original event stream data represents an event.

[0020] Step 20: Based on a preset spatiotemporal filter kernel, detect whether there is a target event within the spatiotemporal neighborhood of any event in the original event stream data, and perform noise reduction processing on the original event stream data according to the detection result to obtain the event stream data after noise reduction processing.

[0021] The target event is an event in the original event stream data except any event itself.

[0022] For the embodiments of the present invention, since the original event stream data collected by the dynamic vision sensor also contains sensor thermal noise and environmental noise, in order to improve the quality of the collected data and ensure the accuracy of the subsequent target detection results and target segmentation results, it is necessary to perform noise reduction processing on the events in the collected original event stream data according to the spatiotemporal neighborhood information. Figure 3As shown, including: Step 21: Based on the size corresponding to the preset spatiotemporal filter kernel, determine whether there is a target event within the spatiotemporal neighborhood of any one of the events.

[0023] The size corresponding to the preset spatiotemporal filter kernel includes a spatial size and a temporal size, and the spatial size and the temporal size can be set according to actual business requirements, which is not specifically limited in the embodiment of the present invention.

[0024] For the embodiment of the present invention, when determining whether there is a target event within the space-time neighborhood of any event, the spatial position distance and time difference between the any event and the other events are calculated based on the spatial coordinates and timestamp of the any event, and the spatial coordinates and timestamps of other events in the original event stream data except the any event; if the spatial position distance is less than or equal to the spatial size, and the time difference is less than or equal to the time size, it is determined that there is a target event within the space-time neighborhood of the any event, and the other event is the target event.

[0025] Specifically, assume that the original event stream data is E={e1,e2,…,ei,…,eN}, where ei represents any event in the original event stream data, and any event ei=(xi, yi, ti) has spatial coordinates (xi, yi) and timestamp ti. For any event ei, detect its neighborhood within the space and time range, that is, under the influence of the preset spatiotemporal filter kernel K(x,y,t), whether there is a target event ej=(xj, yj, tj) such that: |xi-xj|≤Δx, |yi-yj|≤Δy, |ti-tj|≤Δt Among them, Δx and Δy represent the spatial dimensions of the preset spatiotemporal filter kernel, and Δt represents the temporal dimension of the preset spatiotemporal filter kernel. The spatiotemporal neighborhood range of any event can be defined by the spatial dimensions (Δx, Δy) and temporal dimensions (Δt).

[0026] If there is a target event ej that satisfies the above relationship, it is determined that any event ei is not noise and does not need to be removed from the original event stream data.

[0027] Step 22: If there is no target event within the spatiotemporal neighborhood, determine that any one of the events is noise, remove the any one of the events from the original event stream data, and obtain the event stream data after noise reduction processing.

[0028] For the embodiment of the present invention, if there is no target event ej that satisfies the above relationship, then any event ei is determined to be noise and removed from the original event stream data. The embodiment of the present invention performs the above detection process for each event in the original event stream data, and can determine whether it is a noise point. If it is determined to be a noise point, it is removed from the original event stream data, thereby obtaining the event stream data (x, y, t, l) after noise processing. N1, where N1 represents the number of data points after noise reduction.

[0029] Step 30: determine the polarity corresponding to each denoised event in the denoised event stream data, and map each denoised event to a corresponding image channel according to the polarity to obtain polarity channel image frame data.

[0030] The polarity corresponding to each noise-reduced event includes positive polarity and negative polarity.

[0031] For the embodiment of the present invention, after the original event stream data is subjected to denoising, the denoised event stream data needs to be converted into polarity channel image frame data. For the conversion process, the method includes: determining the first several events corresponding to each denoised event according to the timestamp corresponding to each denoised event; and determining the polarity corresponding to each denoised event according to the label information of the first several events and the label information of each denoised event.

[0032] When determining the polarity corresponding to each denoised event based on the label information of the previous events and the label information of each denoised event, determine the number of consistent label information of any two adjacent events in each denoised event and its corresponding previous events based on the label information of the previous events and the label information of each denoised event; if the number of consistent label information is less than a preset number, determine the polarity corresponding to each denoised event based on the label information of the previous event corresponding to each denoised event and the label information of each denoised event; if the number of consistent label information is greater than or equal to the preset number, determine the number of events corresponding to different label information based on the label information of the previous events, and determine the target label information corresponding to the maximum number of events based on the number of events corresponding to different label information; determine the polarity corresponding to each denoised event based on the target label information and the label information of each denoised event.

[0033] The preset number may be set according to actual business requirements, and the embodiment of the present invention does not specifically limit this.

[0034] Specifically, each denoised event in the event stream data after denoising has a timestamp and label information. For the timestamp of any denoised event, several events before the event can be determined, and then the polarity corresponding to any denoised event can be determined based on the label information of the previous several events and the label information of any denoised event. For example, for any denoised event ei, the number of consistent labels of any two adjacent events is determined based on the label information of the 10 denoised events before it and the label information of event ei. Assuming that the labels of 2 groups of events in 10 groups of any two adjacent events are consistent, the number of consistent label information can be determined to be 2. Since the number of consistent label information 2 is less than the preset number 5, the label information of the denoised event ei and its previous event ej are compared, wherein the previous event ej is adjacent to the event ei. If the label information of event ei is consistent with that of event ej, the polarity of the denoised event ei is determined to be positive; on the contrary, if the label information of event ei is opposite to that of event ej, the polarity of the denoised event ei is determined to be negative. Assuming that there are 7 groups of events with consistent labels among any two adjacent events in 10 groups, it can be determined that the number of consistent label information is 7. Since the number of consistent label information 7 is greater than the preset number 5, the number of events with positive and negative label information is determined respectively, and the label information with the largest number of events is determined, that is, the target label information. Then, the target label information is compared with the label information of the denoised event ei. If the two are consistent, the polarity of the denoised event ei is determined to be positive; on the contrary, if the two are inconsistent, the polarity of the denoised event ei is determined to be negative.

[0035] Therefore, according to the above method, the polarity corresponding to each noise-reduced event in the noise-reduced event stream data can be accurately determined, thereby ensuring the conversion effect of subsequent image frames.

[0036] Further, after determining the polarity corresponding to each denoised event, each denoised event is mapped to a corresponding image channel according to the polarity, thereby obtaining polarity channel image frame data. For this process, the method includes: creating a blank image frame with a size consistent with the field of view of the dynamic vision sensor for each image channel, wherein different image channels correspond to different polarities; mapping each denoised event to a blank image frame of the corresponding image channel according to the polarity and spatial coordinates corresponding to each denoised event, thereby obtaining the polarity channel image frame data; Specifically, the size of the polarity channel image is (H, W, C), where W and H represent the width and height of the image, and C represents the number of channels of the image. The number of channels of the image is 2, corresponding to positive and negative polarities, respectively. Then, a blank image frame is created for each image channel.

[0037] For each denoised event ei=(xi,yi,ti,pi), where (xi,yi) represents the spatial coordinates, ti represents the timestamp, and pi represents the polarity, each denoised event is mapped to the blank image frame corresponding to the corresponding image channel according to its corresponding spatial coordinates. For example, if the polarity corresponding to the denoised event ei is positive, the pixel value at the corresponding position (xi,yi) is updated in channel 1; if the polarity corresponding to the denoised event ei is negative, the pixel value at the corresponding position (xi,yi) is updated in channel 2, and each update is cumulative.

[0038] Therefore, each noise-reduced event can be accurately mapped to the corresponding image channel in the above manner, thereby completing the conversion of polarity channel image frame data.

[0039] Step 40: Input the polarity channel image frame data into a preset target detection model and a preset target segmentation model for processing, respectively, to obtain a target detection result and a target segmentation result.

[0040] Among them, the targets to be detected and segmented specifically include vehicles, animals, etc., and the targets in the embodiments of the present invention are not limited to the above examples. The preset target detection models include R-CNN model, single shot multi-frame detector (SSD), YOLO model, RetinaNet model, etc., and the preset target segmentation models include FCN model, U-Net model, Mask R-CNN model, PSPNet model, etc. It should be noted that the preset target detection model and the preset target segmentation model in the embodiments of the present invention are not limited to the above examples, and can also be other models.

[0041] In the embodiment of the present invention, after obtaining the polarity channel image frame data, the polarity channel image frame data is input into a preset target detection model for target detection to identify the target in the polarity channel image. At the same time, the polarity channel image frame data is input into a preset target segmentation model for target segmentation to obtain a target mask image.

[0042] Step 50: Determine a moving target recognition result based on the target detection result and the target segmentation result.

[0043] For the embodiment of the present invention, after determining the target detection result and the target segmentation result, the target detection result and the target segmentation result are rendered to obtain the moving target recognition result. For this process, the method includes: determining the center point coordinates, size and number of the detection frame according to the target detection result; superimposing the detection frame on the polarity channel image frame data according to the center point coordinates, size and number of the detection frame to obtain the superimposed image frame data; rendering and displaying the superimposed image frame data; determining the mask image according to the target segmentation result; separately displaying the mask image in the form of a color block; and determining the moving target recognition result based on the displayed superimposed image frame data and the mask image in the form of a color block. The overall detection process of the embodiment of the present invention is as follows: Figure 4 As shown, the process receives real-time dynamic visual sensor data, can perform efficient and continuous processing by setting the integration time, and can output it in the form of video to obtain video tracking results.

[0044] The embodiment of the present invention adopts dynamic vision sensors and embedded GPU edge computing platforms to form a detection system suitable for the edge end, which solves the problems of high data rate, high power consumption and complex system structure in the prior art. Compared with the prior art, the real-time detection system of the embodiment of the present invention has a lower data rate and system power consumption, and simplifies the system structure, has portability, and can support assembly line and outdoor field work. In addition, the embodiment of the present invention adopts the process of event stream data denoising, preprocessing, image conversion, detection segmentation, and output, and can obtain the processing results in the form of continuous frame video. Compared with the prior art, the data processing method in the real-time detection system of the embodiment of the present invention is more real-time, and can achieve continuous detection, tracking, and video visualization of high-speed targets, with better visibility.

[0045] Further, as Figure 1 and Figure 3 The specific implementation of the method shown in the embodiment provides a high-speed moving target recognition device, such as Figure 5 As shown, the device includes: an acquisition unit 101, a noise reduction unit 102, a mapping unit 103, a detection and segmentation unit 104 and a determination unit 105.

[0046] The acquisition unit 101 may be used to acquire raw event stream data collected by a dynamic vision sensor in a continuous period of time.

[0047] The noise reduction unit 102 can be used to detect whether there is a target event within the spatiotemporal neighborhood of any event in the original event stream data based on a preset spatiotemporal filter kernel, and perform noise reduction processing on the original event stream data according to the detection result to obtain the event stream data after noise reduction processing.

[0048] The mapping unit 103 can be used to determine the polarity corresponding to each denoised event in the denoised event stream data, and map each denoised event to a corresponding image channel according to the polarity to obtain polarity channel image frame data.

[0049] The detection and segmentation unit 104 may be used to input the polarity channel image frame data into a preset target detection model and a preset target segmentation model for processing, respectively, to obtain a target detection result and a target segmentation result.

[0050] The determination unit 105 may be configured to determine a moving target recognition result based on the target detection result and the target segmentation result.

[0051] In some embodiments, the acquisition unit 101 can be specifically used to acquire the spatial coordinates, timestamp and label information of each event collected by the dynamic vision sensor in a continuous time period; and determine the original event stream data based on the spatial coordinates, timestamp and label information of each event.

[0052] In some embodiments, the noise reduction unit 102 includes: a determination module and a removal module.

[0053] The determination module may be used to determine whether there is a target event within the spatiotemporal neighborhood of any one of the events based on a size corresponding to a preset spatiotemporal filter kernel.

[0054] The removal module can be used to determine that any one of the events is noise if there is no target event within the spatiotemporal neighborhood, and remove the any one of the events from the original event stream data to obtain the event stream data after noise reduction processing.

[0055] In some embodiments, the determination module can be specifically used to calculate the spatial position distance and time difference between any one event and the other events based on the spatial coordinates and timestamp of the any one event and the spatial coordinates and timestamps of other events in the original event stream data except the any one event; if the spatial position distance is less than or equal to the spatial dimension, and the time difference is less than or equal to the time dimension, it is determined that there is a target event within the space-time neighborhood of the any one event, and the other event is the target event.

[0056] In some embodiments, the mapping unit 103 includes: a first determination module and a second determination module.

[0057] The first determination module can be used to determine the previous events corresponding to each denoised event according to the timestamp corresponding to each denoised event. The second determination module can be used to determine the polarity corresponding to each denoised event according to the label information of the previous events and the label information of each denoised event.

[0058] In some embodiments, the second determination module can be specifically used to determine the number of consistent label information of any two adjacent events in each denoised event and its corresponding previous events based on the label information of the previous events and the label information of each denoised event; if the number of consistent label information is less than a preset number, the polarity corresponding to each denoised event is determined based on the label information of the previous event corresponding to each denoised event and the label information of each denoised event; if the number of consistent label information is greater than or equal to the preset number, the number of events corresponding to different label information is determined based on the label information of the previous events, and the target label information corresponding to the maximum number of events is determined based on the number of events corresponding to different label information; the polarity corresponding to each denoised event is determined based on the target label information and the label information of each denoised event.

[0059] In some embodiments, the mapping unit 103 can be specifically used to create a blank image frame for each image channel whose size is consistent with the field of view of the dynamic vision sensor, wherein different image channels correspond to different polarities; and map each denoised event to the blank image frame of the corresponding image channel according to the polarity and spatial coordinates corresponding to each denoised event to obtain the polarity channel image frame data.

[0060] In some embodiments, the determination unit 105 can be specifically used to determine the center point coordinates, size and number of the detection frame according to the target detection result; superimpose the detection frame on the polarity channel image frame data according to the center point coordinates, size and number of the detection frame to obtain the superimposed image frame data; render and display the superimposed image frame data; determine the mask image according to the target segmentation result; display the mask image separately in the form of a color block; and determine the moving target recognition result based on the displayed superimposed image frame data and the mask image in the form of a color block.

[0061] It should be noted that for other corresponding descriptions of the functional units involved in the high-speed moving target recognition device provided in this embodiment, reference can be made to Figure 1 and Figure 3 The corresponding description in will not be repeated here.

[0062] Based on the above Figure 1 and Figure 3 The method shown in the embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the above-mentioned Figure 1 and Figure 3 The high-speed moving target recognition method shown.

[0063] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.), including a number of instructions for enabling an electronic device (which can be a personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of the present application.

[0064] Based on the above Figure 1 and Figure 3 The method shown, and Figure 5 In order to achieve the above-mentioned purpose, the embodiment of the present application also provides an electronic device, which can be a personal computer, a tablet computer, a server, or other network equipment, etc. The device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1 and Figure 3 The high-speed moving target recognition method shown.

[0065] Optionally, the above-mentioned physical device may also include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), etc.

[0066] Those skilled in the art will appreciate that the above-mentioned physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different arrangements of components.

[0067] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the above-mentioned physical device, and supports the operation of the information processing program and other software and / or programs. The network communication module is used to realize the communication between the components inside the storage medium, and the communication with other hardware and software in the information processing physical device.

[0068] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform, or by hardware.

[0069] The embodiment of the present invention adopts dynamic vision sensors and embedded GPU edge computing platforms to form a detection system suitable for the edge end, which solves the problems of high data rate, high power consumption and complex system structure in the prior art. Compared with the prior art, the real-time detection system of the embodiment of the present invention has a lower data rate and system power consumption, and simplifies the system structure, has portability, and can support assembly line and outdoor field work. In addition, the embodiment of the present invention adopts the process of event stream data denoising, preprocessing, image conversion, detection segmentation, and output, and can obtain the processing results in the form of continuous frame video. Compared with the prior art, the data processing method in the real-time detection system of the embodiment of the present invention is more real-time, and can achieve continuous detection, tracking, and video visualization of high-speed targets, with better visibility.

[0070] Those skilled in the art will appreciate that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily necessary for implementing the present application. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the description of the implementation scenario, or can be changed accordingly and located in one or more devices different from the present implementation scenario. The modules of the above-mentioned implementation scenario can be combined into one module, or can be further split into multiple submodules.

[0071] The above serial numbers of this application are only for description and do not represent the advantages and disadvantages of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of this application, but this application is not limited to them, and any changes that can be thought of by technicians in this field should fall within the scope of protection of this application.

Claims

1. A high-speed moving target recognition method, characterized in that: Applied to edge computing platforms, including: Obtaining raw event stream data collected by dynamic vision sensors in continuous time periods; Based on a preset spatiotemporal filter kernel, detecting whether there is a target event within the spatiotemporal neighborhood of any event in the original event stream data, and performing noise reduction processing on the original event stream data according to the detection result to obtain event stream data after noise reduction processing; Determine the polarity corresponding to each denoised event in the denoised event stream data, and map each denoised event to a corresponding image channel according to the polarity to obtain polarity channel image frame data; Inputting the polarity channel image frame data into a preset target detection model and a preset target segmentation model for processing respectively, to obtain a target detection result and a target segmentation result; A moving target recognition result is determined according to the target detection result and the target segmentation result.

2. The method according to claim 1, characterized in that The obtaining of raw event stream data collected by the dynamic vision sensor in a continuous period of time includes: Acquire spatial coordinates, timestamps and label information of each event collected by the dynamic vision sensor in a continuous period of time; The original event stream data is determined according to the spatial coordinates, timestamp and tag information of each event.

3. The method according to claim 1, characterized in that The method of detecting whether there is a target event within the spatiotemporal neighborhood of any event in the original event stream data based on a preset spatiotemporal filter kernel, and performing noise reduction processing on the original event stream data according to the detection result to obtain the event stream data after noise reduction processing includes: Based on the size corresponding to the preset spatiotemporal filter kernel, determining whether there is a target event within the spatiotemporal neighborhood of any one of the events; If there is no target event within the space-time neighborhood, it is determined that any one of the events is noise, and the any one of the events is removed from the original event stream data to obtain the event stream data after noise reduction processing.

4. The method according to claim 3, characterized in that: The size corresponding to the preset spatiotemporal filter kernel includes a spatial size and a temporal size, and the determining whether there is a target event within the spatiotemporal neighborhood of any one of the events based on the size corresponding to the preset spatiotemporal filter kernel includes: Calculate the spatial position distance and time difference between the any one event and the other events in the original event stream data according to the spatial coordinates and timestamp of the any one event and the spatial coordinates and timestamps of other events except the any one event; If the spatial position distance is less than or equal to the spatial size, and the time difference is less than or equal to the time size, it is determined that there is a target event within the spatiotemporal neighborhood of any one of the events, and the other event is the target event.

5. The method according to claim 1, characterized in that The determining the polarity corresponding to each denoised event in the denoised event stream data includes: Determine, according to the timestamp corresponding to each denoised event, a number of previous events corresponding to each denoised event; According to the label information of the first several events and the label information of each noise-reduced event, the polarity corresponding to each noise-reduced event is determined.

6. The method according to claim 5, characterized in that The determining, according to the label information of the first several events and the label information of each noise-reduced event, the polarity corresponding to each noise-reduced event includes: Determine, according to the label information of the first several events and the label information of each denoised event, the number of identical label information of any two adjacent events in each denoised event and its corresponding first several events; If the number of the consistent label information is less than a preset number, determining the polarity corresponding to each denoised event according to the label information of the previous event corresponding to each denoised event and the label information of each denoised event; If the number of the consistent label information is greater than or equal to the preset number, the number of events corresponding to the different label information is determined according to the label information of the previous events, and the target label information corresponding to the maximum number of events is determined according to the number of events corresponding to the different label information; According to the target label information and the label information of each denoised event, a polarity corresponding to each denoised event is determined.

7. The method according to claim 1, characterized in that Mapping each noise-reduced event to a corresponding image channel according to the polarity to obtain polarity channel image frame data includes: Creating a blank image frame for each image channel whose size is consistent with the field of view of the dynamic vision sensor, wherein different image channels correspond to different polarities; According to the polarity and spatial coordinates corresponding to each denoised event, each denoised event is mapped to a blank image frame of a corresponding image channel to obtain the polarity channel image frame data; and / or Determining a moving target recognition result according to the target detection result and the target segmentation result includes: Determine the center point coordinates, size and number of the detection frame according to the target detection result; According to the center point coordinates, size and number of the detection frame, superimposing the detection frame on the polarity channel image frame data to obtain superimposed image frame data; Rendering and displaying the superimposed image frame data; Determining a mask image according to the target segmentation result; The mask image is displayed separately in the form of color blocks; The moving target recognition result is determined based on the displayed superimposed image frame data and the mask image in the form of color blocks.

8. A high-speed moving target recognition device, characterized in that: include: An acquisition unit, used for acquiring raw event stream data collected by a dynamic vision sensor in a continuous period of time; A denoising unit, configured to detect whether there is a target event within the spatiotemporal neighborhood of any event in the original event stream data based on a preset spatiotemporal filter kernel, and to perform denoising processing on the original event stream data according to the detection result to obtain the denoised event stream data; A mapping unit, used to determine the polarity corresponding to each denoised event in the denoised event stream data, and map each denoised event to a corresponding image channel according to the polarity to obtain polarity channel image frame data; A detection and segmentation unit, used for inputting the polarity channel image frame data into a preset target detection model and a preset target segmentation model for processing, to obtain a target detection result and a target segmentation result; A determination unit is used to determine a moving target recognition result based on the target detection result and the target segmentation result.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images