A target detection method, device, equipment and readable storage medium

Abstract

The application provides a target detection method, device and equipment and readable storage medium. The method comprises: aggregating event data of a target frame number in event stream information cached by an event camera to obtain target event data; generating an overall event image based on the target event data; performing block processing on the overall event image to obtain a plurality of sub-event images; inputting valid sub-event images in the plurality of sub-event images into a target detection neural network for classification and recognition to obtain a target detection result. Through implementation of the application, the event image is subjected to block processing, and then valid event images are extracted for target detection, so that the sparseness of the to-be-detected data is reduced, and the utilization rate of computing power and the efficiency of target detection are improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a target detection method, apparatus, device, and readable storage medium. Background Technology

[0002] With the continuous development of science and technology, computer vision technology has become increasingly mature. The emergence of event cameras has attracted more and more attention in the field of machine vision. Event cameras simulate the human retina, responding to pixel pulses caused by changes in brightness due to motion. Therefore, they can capture changes in scene brightness at extremely high frame rates, recording events at specific times and locations in the image, forming an event stream rather than a frame stream. This solves the problems of information redundancy, large amounts of data storage, and real-time processing associated with traditional cameras.

[0003] In practical applications, there is often a need to perform target recognition on event images output by event cameras. In related technologies, the general approach is to perform target detection on global event images. However, event images may contain a large amount of redundant data, so performing target detection on global event images with a certain degree of sparsity will result in wasted computing power and reduced detection efficiency. Summary of the Invention

[0004] This application provides a target detection method, apparatus, device, and readable storage medium, which can at least solve the problems of large computational waste and low detection efficiency of target detection schemes provided in related technologies.

[0005] The first aspect of this application provides a target detection method, including:

[0006] The target event data is obtained by aggregating the event data of the target frame number in the event stream information cached by the event camera;

[0007] Generate an overall event image based on the target event data;

[0008] The overall event image is divided into blocks to obtain multiple sub-event images;

[0009] The valid sub-event images from the multiple sub-event images are input into the target detection neural network for classification and recognition to obtain the target detection result.

[0010] A second aspect of this application provides a target detection device, comprising:

[0011] The aggregation module is used to aggregate the event data of the target frame number in the event stream information cached by the event camera to obtain the target event data;

[0012] The generation module is used to generate an overall event image based on the target event data;

[0013] The segmentation module is used to segment the overall event image into multiple sub-event images.

[0014] The detection module is used to input the valid sub-event images from the multiple sub-event images into the target detection neural network for classification and recognition, so as to obtain the target detection result.

[0015] A third aspect of this application provides an electronic device, including a memory and a processor, wherein the processor is used to execute a computer program stored in the memory, and when the processor executes the computer program, it implements the steps of the target detection method provided in the first aspect of this application.

[0016] The fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it implements the steps of the target detection method provided in the first aspect of this application.

[0017] As can be seen from the above, according to the target detection method, apparatus, device, and readable storage medium provided in this application, the event data of the target frame number in the event stream information cached by the event camera are aggregated to obtain target event data; an overall event image is generated based on the target event data; the overall event image is divided into blocks to obtain multiple sub-event images; and the effective sub-event images from the multiple sub-event images are input into the target detection neural network for classification and recognition to obtain the target detection result. Through the implementation of this application, the effective event images are extracted after block processing of the event image for target detection, reducing the sparsity of the data to be detected and improving the utilization rate of computing power and the efficiency of target detection. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the basic process of a target detection method provided in the first embodiment of this application;

[0019] Figure 2 This is a schematic diagram of four consecutive frames of event data provided in the first embodiment of this application;

[0020] Figure 3 This is a schematic diagram of target event data provided in the first embodiment of this application;

[0021] Figure 4 A schematic diagram of another target event data provided in the first embodiment of this application;

[0022] Figure 5 A schematic diagram illustrating an event data resizing process provided in the first embodiment of this application;

[0023] Figure 6 A schematic diagram of a linear transformation provided for the first embodiment of this application;

[0024] Figure 7 This is a schematic diagram of the structure of a classification and recognition network provided in the first embodiment of this application;

[0025] Figure 8 A detailed flowchart illustrating a target detection method provided in the second embodiment of this application;

[0026] Figure 9 This is a schematic diagram of the program modules of the target detection device provided in the third embodiment of this application;

[0027] Figure 10 This is a schematic diagram of the structure of an electronic device provided in the fourth embodiment of this application. Detailed Implementation

[0028] To make the inventive objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0029] In the description of the embodiments of this application, it should be understood that the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings. They are only for the convenience of describing the embodiments of this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting the present invention.

[0030] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0031] In the embodiments of this application, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0032] The above description is merely a preferred embodiment of this application and is not intended to limit the invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

[0033] To address the issues of high computational waste and low detection efficiency in existing target detection schemes, the first embodiment of this application provides a target detection method, such as... Figure 1 This is a basic flowchart illustrating the target detection method provided in this embodiment. The target detection method includes the following steps:

[0034] Step 101: Aggregate the event data of the target frame number in the event stream information cached by the event camera to obtain the target event data.

[0035] Specifically, in this embodiment, the event camera, also known as the event-based vision sensor (EVS), is a novel type of image sensor. This image sensor includes a pixel array composed of multiple pixels, each of which operates independently. An event is only output when the brightness change of a pixel reaches a certain threshold. It should be noted that each pixel sensor in the event camera's pixel array is an integrated circuit. In this integrated circuit, a photodiode can be integrated with a capacitor that accumulates charge. The photodiode generates a photocurrent in response to the intensity of incident light, and a real-time voltage is generated accordingly based on the photocurrent.

[0036] Then, after subtracting each real-time voltage from a preset reference voltage, each voltage difference is compared with a preset voltage threshold range. Based on the comparison results, a binary value is generated for each pixel. This binary value is used to characterize whether the light signal has changed. It should be noted that when the voltage difference exceeds the voltage threshold range, the generated binary value is 1, indicating that an event has occurred at the pixel; when the voltage difference does not exceed the voltage threshold range, the generated binary value is 0, indicating that no event has occurred at the pixel. Finally, pixel coordinate information and timestamp information are embedded into each binary value to generate event data.

[0037] like Figure 2 The diagram shown is a schematic diagram of four consecutive frames of event data provided in this embodiment. Event data 201, 202, 203, and 204 are sequentially continuous. Figure 3 The diagram shown illustrates a target event data provided in this embodiment, where event data 30 represents the target event data. In practical applications, outputting event images typically requires multiple frames of event data with continuous temporal sequence. This embodiment aggregates multiple frames of event data with continuous temporal sequence into an event matrix. Preferably, this embodiment can perform a logical OR operation on the binary values ​​of the same pixel positions in multiple frames of event data to obtain the aggregated target event data.

[0038] Step 102: Generate an overall event image based on the target event data.

[0039] Specifically, in this embodiment, the event data is represented as e = {x} i ,y i ,t i ,p i}, i∈0,1,…,n-1. In practical applications, event data can be divided into multiple voxels according to time, and then the event of one voxel can be converted into a two-dimensional event image.

[0040] In one embodiment of this invention, the step of generating an overall event image based on target event data includes: dividing the target event data into multiple voxels according to the event; wherein, the event data is represented as e = {x} i ,y i ,t i ,p i}, i∈0,1,…,n-1,(x i ,y i ) represents the pixel position, t i p represents a timestamp. i To characterize the polarity of an event, the value of the event polarity is a binary value;

[0041] Each voxel is converted into a global event image based on a preset conversion formula; the conversion formula is expressed as:

[0042]

[0043]

[0044] Where V(x,y,t) represents a voxel, and B represents the number of voxels.

[0045] In another embodiment of this example, the step of generating an overall event image based on the target event data includes: denoising the target event data to obtain denoised event data; and generating an overall event image based on the denoised event data.

[0046] Specifically, in practical applications, the data collected by event cameras usually carries a large amount of discrete noise. Noise filtering of the collected data is an important means to ensure the quality of event images. Based on this, this embodiment first performs denoising processing on the event data, and then generates event images based on the denoised event data to improve the quality of the generated event images and facilitate subsequent image target detection.

[0047] Furthermore, in one embodiment of this example, the step of denoising the target event data to obtain denoised event data includes: for each event pixel position in the target event data, counting the total number of event pixels in the pixel array region including the event pixel position; wherein, the binary value of the event pixel position is a first value, the first value indicating that an event has occurred at the sensor pixel; comparing the total number of event pixels with a preset number threshold; when the total number of event pixels is less than the number threshold, setting the binary value of the event pixel position in the target event data to a second value to obtain denoised event data; wherein, the second value indicates that no event has occurred at the sensor pixel.

[0048] Specifically, in this embodiment, for the event pixel position in the target event data with a binary value of the first value, the total number of all first values ​​in the pixel array region composed of the pixel position and its neighboring pixel positions is counted, and this total number is compared with a specific threshold to determine whether each event pixel position is noise. In this embodiment, if the total number of first values ​​in the pixel array region where the event pixel position is located is less than the threshold, it indicates that the event pixel position is noise, and the event it generates is a false event. It should be filtered out, that is, the original first value is set to the second value, that is, the first value 1 is reset to the second value 0.

[0049] like Figure 4 The diagram illustrates another type of target event data provided in this embodiment. Event data 40 represents the target event data. The overall pixel array of this event data includes 9*9 single pixel positions (e.g., A in the diagram). Then, for a 3*3 pixel array (e.g., 401 in the diagram) containing the event pixel position A to be denoised, the total number of event pixels is counted. The total number of event pixels in pixel array 401 is 6. Assuming a preset threshold of 5, since the total number of event pixels is greater than the threshold, it indicates that the event pixel position A to be denoised is not noise, and this should be a valid event. It should be noted that in practical applications, the pixel array region divided for the event pixel to be denoised can be of regular or irregular shape, and the size of the pixel array region can also be flexibly set according to requirements. This embodiment does not impose a unique limitation on this.

[0050] Furthermore, in another embodiment of this example, the step of generating an overall event image based on denoised event data includes: adjusting the original data size of the denoised event data to a target data size to obtain dense event data; wherein the target data size is smaller than the original data size; and generating an overall event image based on the dense event data.

[0051] Specifically, in the field of data processing, in order to improve the density of data while preserving the feature information of the original data, the original data is usually resized. Common solutions include nearest neighbor interpolation and bilinear interpolation. When using nearest neighbor interpolation to process event data, the pixel value of the nearest point is used to generate the pixel value at the corresponding position after resizing. This can easily lose the image's detailed texture information, especially in sparse event data, causing the already sparse data to become even sparser, resulting in a loss of detail. When using traditional bilinear interpolation, the pixel values ​​of the four pixels surrounding the corresponding pixel position in the original event data are used to generate the pixel value at the corresponding position in the resizing event data. However, this method is computationally intensive and time-consuming, unsuitable for binarized event data, and also easily leads to more sparse data.

[0052] Based on this, in a preferred embodiment of this example, the step of adjusting the original data size of the denoised event data to the target data size to obtain dense event data includes: extracting unit event data from the denoised event data whose pixel positions satisfy the first position condition, the second position condition, the third position condition, and the fourth position condition respectively; combining all unit event data that satisfy each type of position condition to obtain four sub-event data; and performing a logical OR operation on the binary values ​​of the same pixel positions of the four sub-event data to obtain dense event data.

[0053] Specifically, the first positional condition is that both the pixel row position and column position are odd values; the second positional condition is that the pixel row position is odd and the column position is even; the third positional condition is that the pixel row position is even and the column position is odd; and the fourth positional condition is that both the pixel row position and column position are even values. For example... Figure 5The diagram illustrates an event data resizing process provided in this embodiment. In the diagram, 50 represents denoised event data, 501, 502, 503, and 504 represent sub-event data extracted according to the first, second, third, and fourth position conditions, respectively, and 505 represents dense event data obtained through resizing. For denoised event data with an array size of 8*8, the following sub-event data are fixed: 501 (row and column positions 1, 3, 5, 7); 502 (row and column positions 2, 4, 6, 8); 503 (row and column positions 2, 4, 6, 8); and 504 (row and column positions 2, 4, 6, 8). Finally, a logical OR operation is performed on the four sub-event data to obtain the desired result. Figure 5 The dense event data shown in Figure 505 can improve the density of the data and retain the feature information of the original data as much as possible.

[0054] Step 103: Divide the overall event image into blocks to obtain multiple sub-event images.

[0055] Specifically, in this embodiment, the overall event image is divided into multiple sub-event images. For example, the overall event image with a size of 256*256 is divided into 4*4 sub-event images of the same size, and each sub-event image has a size of 64*64. It should be noted that the event occurrence rate is different in the multiple sub-event images obtained by the block division in this embodiment.

[0056] Step 104: Input the valid sub-event images from multiple sub-event images into the target detection neural network for classification and recognition to obtain the target detection result.

[0057] Specifically, the target detection result in this embodiment includes the target object type and its corresponding location. In this embodiment, target detection of event images is achieved based on a deep learning algorithm. The neural network used can include any one of deep neural networks (DNN), convolutional neural networks (CNN), and recurrent neural networks (RNN). It should be noted that the effective sub-event images in this embodiment can be images with an event generation rate higher than a preset threshold. When the threshold is 0, the effective sub-event images are all sub-event images with events occurring, while sub-event images without events occurring are discarded as redundant data and do not participate in classification and recognition calculations. This effectively reduces the computational load of the model, improves the model's operating efficiency, and ensures the response speed of target detection in event images.

[0058] In one embodiment of this invention, the target detection neural network includes a linear transformation network and a classification and recognition network. The classification and recognition network includes a first normalization network, an attention network, a second normalization network, and a multilayer perceptron network. Accordingly, the step of inputting the effective sub-event images from multiple sub-event images into the target detection neural network for classification and recognition to obtain the target detection result includes: inputting the effective sub-event images from multiple sub-event images into the linear transformation network to obtain the input features of the classification and recognition network; inputting the input features into the first normalization network for normalization processing to obtain the first normalized features; inputting the first normalized features into the attention network to obtain the mask corresponding to the input features; fusing the input features and the mask to obtain the fused features; inputting the fused features into the second normalization network for normalization processing to obtain the second normalized features; and inputting the second normalized features into the multilayer perceptron network for classification and recognition to obtain the target detection result.

[0059] Specifically, the object detection network model in this embodiment first performs a linear transformation (also known as a linear mapping) on ​​the input sub-event images. That is, after the sub-event images obtained by segmentation are unfolded into one-dimensional vectors, they are mapped to a low-dimensional space to extract the main features of the image, and then further mapped to a high-dimensional space. For example Figure 6 The diagram shown illustrates a linear transformation provided in this embodiment. Assuming the scale of the sub-event image obtained by segmentation is 16*1024, it is linearly mapped to a lower dimension, such as 16*512, 16*256, 16*128, etc., to extract the main features of the image. Then, the 16*128 is mapped to a higher-dimensional space and recombined into a two-dimensional vector, such as 16*256, 16*512, 16*1024, etc., to identify objects in the captured image. Thus, the input features of the subsequent classification and recognition network are obtained.

[0060] like Figure 7 The diagram shown is a structural schematic of a classification and recognition network provided in this embodiment. Further, in this embodiment, the feature map obtained through the linear transformation network is input into the first normalization network of the classification and recognition network 70 for mean and variance calculation to normalize the feature map. According to the normalization principle, the weight of each pixel in the feature map ensures that the original distribution of the data is centrally symmetric. Next, the normalized features are input into the attention network for weight optimization to obtain a mask corresponding to the input features, extracting more effective features. Then, the mask is fused with the input features to retain lower-level features. Further, the fused features are normalized and input into a multilayer perceptron for classification and recognition, outputting the target detection result of the event image.

[0061] Based on the technical solution of the above embodiments of this application, the event data of the target frame number in the event stream information cached by the event camera is aggregated to obtain target event data; an overall event image is generated based on the target event data; the overall event image is divided into blocks to obtain multiple sub-event images; the effective sub-event images among the multiple sub-event images are input into a target detection neural network for classification and recognition to obtain target detection results. By implementing the solution of this application, the effective event images are extracted after the event image is divided into blocks for target detection, which reduces the sparsity of the data to be detected and improves the utilization rate of computing power and the efficiency of target detection.

[0062] Figure 8 The method described in the second embodiment of this application is a refined target detection method, which includes:

[0063] Step 801: Aggregate the event data of the target frame number in the event stream information cached by the event camera to obtain the target event data.

[0064] Specifically, in this embodiment, a logical OR operation can be performed on the binary values ​​of the same pixel position in multiple consecutive frames of event data to obtain aggregated target event data.

[0065] Step 802: For each event pixel position in the target event data, count the total number of event pixels in the pixel array region including the event pixel position.

[0066] Specifically, in this embodiment, the binary value of the event pixel position is 1, indicating that an event has occurred.

[0067] Step 803: When the total number of event pixels is less than the number threshold, modify the binary value of the event pixel position in the target event data from the first value to the second value to obtain the denoised event data.

[0068] Specifically, in this embodiment, when the total number of first values ​​in the pixel array region where the event pixel is located is less than the number threshold, it indicates that the event pixel is noise and the event it generates is a false event. The event should be filtered out, that is, the original first value is set to the second value, that is, the first value 1 is modified to the second value 0, which represents that no event is generated.

[0069] Step 804: Extract the unit event data from the denoised event data where the pixel position satisfies the first position condition, the second position condition, the third position condition, and the fourth position condition.

[0070] Specifically, in this embodiment, the first position condition is that both the pixel row position and the column position are odd values, the second position condition is that the pixel row position is odd values ​​and the column position is even values, the third position condition is that the pixel row position is even values ​​and the column position is odd values, and the fourth position condition is that both the pixel row position and the column position are even values.

[0071] Step 805: Combine all unit event data that meet various location conditions, and perform a logical OR operation on the binary values ​​of the same pixel position of the four sub-event data obtained by combination to obtain dense event data.

[0072] Specifically, this embodiment resizes the original event data to obtain dense event data, which can improve the density of the data and retain the feature information of the original data as much as possible.

[0073] Step 806: Generate an overall event image based on dense event data, and divide the overall event image into blocks to obtain multiple sub-event images.

[0074] Step 807: Input the valid sub-event images with an event generation rate higher than a preset threshold from multiple sub-event images into the target detection neural network for classification and recognition, and obtain the target detection result.

[0075] Specifically, in this embodiment, the event generation rates of the multiple sub-event images obtained by segmentation are different. Subsequently, only the sub-event images with rich event information are input into the neural network for target detection. Redundant data does not participate in classification and recognition calculations. As a result, the computational load of the model can be effectively reduced and the model running efficiency can be improved.

[0076] It should be understood that the sequence number of each step in this embodiment does not imply the order in which the steps are executed. The execution order of each step should be determined by its function and internal logic, and should not constitute a unique limitation on the implementation process of this application embodiment.

[0077] Figure 9 This application provides a target detection device according to a third embodiment. This target detection device can be applied to the aforementioned target detection method. For example... Figure 9 As shown, the target detection device mainly includes:

[0078] The aggregation module 901 is used to aggregate the event data of the target frame number in the event stream information cached by the event camera to obtain the target event data;

[0079] Generation module 902 is used to generate an overall event image based on the target event data;

[0080] The segmentation module 903 is used to segment the overall event image into multiple sub-event images;

[0081] The detection module 904 is used to input the valid sub-event images from multiple sub-event images into the target detection neural network for classification and recognition, and to obtain the target detection result.

[0082] In some implementations of this embodiment, the generation module is specifically used to: denoise the target event data to obtain denoised event data; and generate an overall event image based on the denoised event data.

[0083] Furthermore, in some embodiments of this example, when the generation module performs the above-described function of denoising the target event data to obtain denoised event data, it is specifically used to: count the total number of event pixels in the pixel array region including the event pixel position for each event pixel position in the target event data; wherein, the binary value of the event pixel position is a first value, the first value indicating that an event has occurred at the sensor pixel; compare the total number of event pixels with a preset number threshold; when the total number of event pixels is less than the number threshold, set the binary value of the event pixel position in the target event data to a second value to obtain denoised event data; wherein, the second value indicates that no event has occurred at the sensor pixel.

[0084] Furthermore, in some other embodiments of this example, when the generation module performs the function of generating an overall event image based on the denoised event data, it is specifically used to: adjust the original data size of the denoised event data to the target data size to obtain dense event data; wherein, the target data size is smaller than the original data size; and generate an overall event image based on the dense event data.

[0085] Furthermore, in some implementations of this embodiment, when the generation module performs the function of adjusting the original data size of the denoised event data to the target data size to obtain dense event data, it is specifically used to: extract unit event data from the denoised event data whose pixel positions satisfy the first position condition, the second position condition, the third position condition, and the fourth position condition; wherein, the first position condition is that both the pixel row position and the column position are odd values, the second position condition is that the pixel row position is odd value and the column position is even value, the third position condition is that the pixel row position is even value and the column position is odd value, and the fourth position condition is that both the pixel row position and the column position are even values; combine all unit event data that satisfy each type of position condition to obtain four sub-event data; perform a logical OR operation on the binary values ​​of the same pixel position of the four sub-event data to obtain dense event data.

[0086] In some embodiments of this example, the target detection neural network includes a linear transformation network and a classification and recognition network. The classification and recognition network includes a first normalization network, an attention network, a second normalization network, and a multilayer perceptron network. Accordingly, the detection module is specifically used for: inputting valid sub-event images from multiple sub-event images into the linear transformation network to obtain input features for the classification and recognition network; inputting the input features into the first normalization network for normalization processing to obtain first normalized features; inputting the first normalized features into the attention network to obtain a mask corresponding to the input features; fusing the input features and the mask to obtain fused features; inputting the fused features into the second normalization network for normalization processing to obtain second normalized features; and inputting the second normalized features into the multilayer perceptron network for classification and recognition to obtain the target detection result.

[0087] It should be noted that the target detection methods in the first and second embodiments can be implemented based on the target detection device provided in this embodiment. Those skilled in the art can clearly understand that, for the sake of convenience and brevity, the specific working process of the target detection device described in this embodiment can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0088] According to the target detection device provided in this embodiment, the event data of the target frame number in the event stream information cached by the event camera is aggregated to obtain target event data; an overall event image is generated based on the target event data; the overall event image is divided into blocks to obtain multiple sub-event images; the effective sub-event images among the multiple sub-event images are input into the target detection neural network for classification and recognition to obtain the target detection result. By implementing the solution of this application, the effective event images are extracted after the event image is divided into blocks for target detection, which reduces the sparsity of the data to be detected and improves the utilization rate of computing power and the efficiency of target detection.

[0089] Figure 10 An electronic device is provided in the fourth embodiment of this application. This electronic device can be used to implement the target detection method in the foregoing embodiments, and mainly includes:

[0090] The system includes a memory 1001, a processor 1002, and a computer program 1003 stored on the memory 1001 and executable on the processor 1002. The memory 1001 and the processor 1002 are connected via communication. When the processor 1002 executes the computer program 1003, it implements the method described in Embodiment 1 or 2 above. The number of processors can be one or more.

[0091] The memory 1001 can be a high-speed random access memory (RAM) or a non-volatile memory, such as a disk storage device. The memory 1001 is used to store executable program code, and the processor 1002 is coupled to the memory 1001.

[0092] Furthermore, embodiments of this application also provide a computer-readable storage medium, which may be disposed in the aforementioned electronic device, and the computer-readable storage medium may be as described above. Figure 10 The memory in the illustrated embodiment.

[0093] The computer-readable storage medium stores a computer program that, when executed by a processor, implements the target detection method described in the foregoing embodiments. Furthermore, the computer-readable storage medium can also be a USB flash drive, a portable hard drive, a read-only memory (ROM), RAM, a magnetic disk, or an optical disk, or any other medium capable of storing program code.

[0094] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0095] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0096] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0097] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0098] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0099] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0100] The above is a description of the target detection method, apparatus, device, and readable storage medium provided in this application. For those skilled in the art, based on the ideas of the embodiments of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A target detection method, characterized in that, include: The target event data is obtained by aggregating the event data of the target frame number in the event stream information cached by the event camera; The target event data is denoised to obtain denoised event data; Unit event data in the denoised event data that satisfy the first position condition, the second position condition, the third position condition, and the fourth position condition are extracted respectively; wherein, the first position condition is that both the pixel row position and the column position are odd values, the second position condition is that the pixel row position is odd value and the column position is even value, the third position condition is that the pixel row position is even value and the column position is odd value, and the fourth position condition is that both the pixel row position and the column position are even values; The unit event data that satisfy various location conditions are combined to obtain four sub-event data; Perform a logical OR operation on the binary values ​​of the same pixel positions of the four sub-event data to obtain dense event data; Generate an overall event image based on the dense event data; The overall event image is divided into blocks to obtain multiple sub-event images; The valid sub-event images from the multiple sub-event images are input into the target detection neural network for classification and recognition to obtain the target detection result.

2. The target detection method according to claim 1, characterized in that, The step of denoising the target event data to obtain denoised event data includes: For each event pixel position in the target event data, the total number of event pixels in the pixel array region including the event pixel position is counted; wherein, the binary value of the event pixel position is a first value, the first value indicating that an event has occurred at the sensor pixel; Compare the total number of event pixels with a preset number threshold; When the total number of event pixels is less than the number threshold, the binary value of the event pixel position in the target event data is set to a second value to obtain denoised event data; wherein, the second value indicates that no event is generated at the sensor pixel.

3. The target detection method according to claim 1 or 2, characterized in that, The step of generating an overall event image based on the target event data includes: The target event data is divided into multiple voxels based on the event; where the event data is represented as e= , ( ) represents the pixel position. Represents a timestamp. To characterize the polarity of an event, the value of the event polarity is a binary value; Each voxel is converted into a whole event image based on a preset conversion formula; the conversion formula is expressed as: in, This refers to the voxel. This indicates the number of voxels.

4. The target detection method according to claim 1 or 2, characterized in that, The target detection neural network includes a linear transformation network and a classification and recognition network. The classification and recognition network includes a first normalization network, an attention network, a second normalization network, and a multilayer perceptron network. The step of inputting the valid sub-event images from the multiple sub-event images into the target detection neural network for classification and recognition to obtain the target detection result includes: The effective sub-event images from the multiple sub-event images are input into the linear transformation network to obtain the input features of the classification and recognition network; The input features are input into the first normalization network for normalization processing to obtain the first normalized features; The first normalized feature is input into the attention network to obtain the mask corresponding to the input feature; The input features are fused with the mask to obtain fused features; The fused features are input into the second normalization network for normalization processing to obtain the second normalized features; The second normalized feature is input into the multilayer perceptron for classification and recognition to obtain the target detection result.

5. A target detection device, characterized in that, include: The aggregation module is used to aggregate the event data of the target frame number in the event stream information cached by the event camera to obtain the target event data; A generation module is used to perform noise reduction processing on the target event data to obtain denoised event data; Unit event data in the denoised event data that satisfy the first position condition, the second position condition, the third position condition, and the fourth position condition are extracted respectively; wherein, the first position condition is that both the pixel row position and the column position are odd values, the second position condition is that the pixel row position is odd value and the column position is even value, the third position condition is that the pixel row position is even value and the column position is odd value, and the fourth position condition is that both the pixel row position and the column position are even values; The unit event data that satisfy various location conditions are combined to obtain four sub-event data; Perform a logical OR operation on the binary values ​​of the same pixel positions of the four sub-event data to obtain dense event data; Generate an overall event image based on the dense event data; The segmentation module is used to segment the overall event image into multiple sub-event images. The detection module is used to input the valid sub-event images from the multiple sub-event images into the target detection neural network for classification and recognition, so as to obtain the target detection result.

6. An electronic device, characterized in that, Includes memory and processor, of which: The processor is used to execute computer programs stored in the memory; When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images