Pulse image target detection method based on fast attention

Abstract

The invention provides a pulse image target detection method based on fast attention. The method comprises the following steps: S1, preparing a pulse camera image data set; s2, constructing a fast attention mechanism definition layer; s3, designing a trunk neural network structure; s4, adaptively calling an attention mechanism; and S5, training verification by using the pulse camera image data set. According to the pulse image target detection method based on fast attention, the problem that the model training speed is low due to the fact that the pulse image is poor in recognition efficiency in an extreme illumination environment in an existing method is effectively solved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a pulse image target detection method based on rapid attention. Background Art

[0002] In the future development of image and video technology, pulse cameras will be a new development direction. Ultra-high-speed pulse cameras simulate the coding principle of primate retina, combining the speed of the machine with the mechanism of biological vision. Currently, they can convert light signals into pulse bit streams at a frequency of 40KHZ, achieving high-speed target detection and tracking 1,000 times faster than human vision. The visual spatiotemporal changes recorded by common digital cameras are more complete, and are expected to fundamentally change the concepts of images and videos and related industries. In this context, the recognition and processing of pulse images is also in urgent need of research.

[0003] At the same time, most of the target detection models based on the convolutional neural network mechanism (CNN) do not use the attention mechanism within the model, or only use a lightweight traditional attention mechanism. The model's expression and generalization capabilities are weak, and there is still a lot of room for improvement in training speed and convergence speed. It is still difficult to process long sequences and large images. In open source target detection models such as "You Only Look Once v8" (hereinafter referred to as YOLOv8), it only has lightweight traditional attention mechanisms such as "SE" and "CBAM". The target detection model using the traditional attention mechanism has weak recognition and processing capabilities for high-speed pulse images in extreme lighting environments, and it is difficult to detect the correct target object. In addition, the training recognition speed and resource utilization are low, making it difficult to complete the image detection function in future pulse camera applications.

[0004] The main problems existing in the prior art are: in the prior art, general convolutional neural network-based target detection models lack references to the attention mechanism or only have traditional attention mechanisms, which will lead to weak expression and generalization capabilities of the model, difficulty in recognizing long sequences or large images, and low robustness to noise, ultimately leading to target loss or misrecognition in object recognition of pulse images.

[0005] To solve the problems and defects of the above existing technologies, it is necessary to completely redesign the model backbone neural network. The difficulty of the redesign lies in the need to build a fast attention mechanism layer, replace or add it in the model backbone neural network, and add a fast attention mechanism with adaptive reference and multi-scale feature extraction to the model to improve the target detection model. As a new technology, the fast attention mechanism has only appeared recently, and research on its application is still limited. The present invention has successfully tested it in the target detection model, which not only realizes the practical application of the fast attention mechanism and improves the detection effect of the model, but also fills the gap in the field of pulse image object recognition. Summary of the invention

[0006] The present invention provides a pulse image target detection method based on fast attention, which effectively solves the problems of poor recognition efficiency of pulse images in extreme lighting environments and slow model training speed in the old methods.

[0007] The technical solution of the present invention is as follows: The fast attention-based pulse image target detection method of the present invention comprises the following steps: S1. preparing a pulse camera image dataset; S2. constructing a fast attention mechanism definition layer; S3. designing a backbone neural network structure; S4. adaptively calling the fast attention mechanism; and S5. training and verification using the pulse camera image dataset.

[0008] Optionally, in the above-mentioned pulse image target detection method based on rapid attention, in step S1, the input images are uniformly adjusted to images of the same resolution, and then the objects to be detected in the images are annotated. The annotation format must meet the model requirements. One image corresponds to one annotation file, and the image and image annotation labels are obtained respectively. The data are classified according to the training set, test set and evaluation set and stored according to the model format requirements.

[0009] Optionally, in the above-mentioned fast attention-based pulse image target detection method, in step S2, a fast attention mechanism definition layer is constructed so that it can be adaptively referenced in the backbone neural network, and the model code is modified to make the fast attention mechanism definition layer compatible with the model.

[0010] Optionally, in the above-mentioned fast attention-based pulse image target detection method, in step S3, the backbone neural network structure of the model is designed, and the fast attention mechanism definition layer constructed by reference can be inserted before different convolutional layers (C2F layers), and the convolutional layer is added when the fast attention mechanism layer is inserted again before the fast spatial pyramid pooling layer to achieve the effect of multi-scale feature extraction.

[0011] Optionally, in the above-mentioned fast attention-based pulse image target detection method, in step S4, when called in the backbone neural network, the adaptive program in the constructed fast attention mechanism definition layer will determine whether the current layer belongs to a non-important layer that can improve efficiency, and realize adaptive calling of the attention mechanism to further improve the accuracy and efficiency of the new model while maintaining the effect of the original model.

[0012] Optionally, in the above-mentioned pulse image target detection method based on fast attention, in step S5, a pulse camera image dataset is referenced in the training file, and the debugging parameters are modified before and after training to achieve the best training effect. Through the training of its model, the optimal weights and process records of the final training are obtained, and it is checked whether the results meet the conditions for completing the experiment.

[0013] According to the technical solution of the present invention, the beneficial effects produced are: The present invention realizes a pulse image target detection method based on "Flash Attention". Aiming at the lack of research on pulse images in the current computer vision field and the lack of more advanced self-attention mechanism in the traditional convolutional neural network-based target detection model, a pulse image recognition model based on "Flash Attention" is constructed, which effectively solves the problem that the old method has poor recognition efficiency and slow model training speed due to pulse images in extreme lighting environments. The method of the present invention, according to the input block and parallel computing characteristics of the "Flash Attention" mechanism, is applied to multi-head attention to block large images for calculation, and through its unique optimization and memory access to the graphics processing unit (GPU), it can greatly reduce the computing cost and memory overhead while improving the accuracy, and design adaptive references in the backbone network to realize multi-scale feature extraction while ensuring the original efficiency of the model, thereby improving the robustness of the model in complex real-world scenes, thereby avoiding the problem of poor recognition rate and slow calculation process of pulse images in the old method, and finally realizing efficient recognition of pulse image targets.

[0014] In order to better understand and illustrate the concept, working principle and effect of the present invention, the present invention is described in detail below through specific embodiments in conjunction with the accompanying drawings: BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the specific implementation of the present invention or the technical solution in the prior art, the drawings required for use in the specific implementation or the description of the prior art are briefly introduced below.

[0016] Figure 1 It is a flow chart of the pulse image target detection method based on rapid attention of the present invention; Figure 2is a schematic diagram of preparing a pulse image data set involved in the method of the present invention; Figure 3 is a schematic diagram of the design backbone neural network of the present invention; Figure 4 It is a flow chart of the implementation method of the present invention; Figure 5 It is a schematic diagram of the reasoning results in the implementation method of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical method and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and specific examples. These examples are only illustrative and not limiting of the present invention.

[0018] The pulse image target detection method based on fast attention of the present invention particularly relates to a pulse image target detection algorithm based on deep learning and neural network. The algorithm can be redesigned on the existing target detection model, adding the latest "Flash Attention" mechanism, and can adaptively use fast attention for the model backbone network. By extracting multi-scale features, a more efficient algorithm for pulse image target detection is realized. The principle of the present invention is to utilize the improved self-attention mechanism of the fast attention mechanism to process the input data in blocks for parallel calculation and precise calculation of Softmax, and adaptively extract multi-scale features, so as to improve the training speed, memory optimization and recognition accuracy of the model, thereby realizing the pulse image object recognition function based on the target detection model.

[0019] The present invention utilizes the powerful function of the "Flash Attention" mechanism and implements adaptive addition and introduction in the existing convolutional neural network image recognition model to realize multi-scale feature extraction of the model, thereby realizing an algorithm model that can identify and detect high-speed pulse flow image targets with more accuracy and higher efficiency, thereby solving the problems of poor recognition accuracy and low efficiency of pulse images under extreme lighting conditions in the old method.

[0020] The method of the present invention provides a method for improving the detection of pulse image objects. The pulse image containing the object to be identified and the pulse image object annotation label are used as input data of the object recognition algorithm model at the same time. The object recognition algorithm of the backbone neural network designed with a fast attention mechanism is used to construct a generation network for identifying pulse image objects, thereby realizing a function that can identify pulse image objects under extreme lighting or low resolution conditions.

[0021] like Figure 1As shown, the pulse image target detection method based on fast attention of the present invention is a method for improving the recognition of pulse image targets by a neural network, and includes the following steps from the beginning to the end: S1. Prepare the pulse camera image dataset: adjust the input images to the same resolution, and then annotate the objects to be detected (recognized) in the images. The annotation format must meet the model requirements. One image corresponds to one annotation file. The images and image annotation labels are obtained respectively. The datasets are classified into training sets, test sets, and evaluation sets and stored according to the model format (e.g., pulse picture-1.png, object annotation-1.txt...). The schematic diagram of preparing the pulse image dataset is as follows: Figure 2 shown.

[0022] S2. Build a fast attention mechanism definition layer: Build a fast attention mechanism definition layer in the model, and design it in an adaptive mechanism. Specifically, build a definition layer that implements the "fast attention" mechanism so that it can be adaptively referenced in the backbone neural network, and modify the model code to make the fast attention mechanism definition layer compatible with the model.

[0023] S3. Design the backbone neural network structure: Figure 3 As shown in the figure, the backbone neural network structure of the designed model can insert the referenced fast attention mechanism definition layer (step 1) before different convolutional layers (C2F layers), and when the referenced fast attention mechanism layer is inserted again before the fast spatial pyramid pooling layer (SPPF layer), it is also necessary to add a convolutional layer (C2F layer)-step 2 to achieve the effect of multi-scale feature extraction.

[0024] S4. Adaptive call of the fast attention mechanism. When called in the backbone neural network, the adaptive program in the constructed fast attention mechanism definition layer will determine whether the current layer belongs to a non-important layer that can improve efficiency, and implement adaptive call of the attention mechanism to further improve the accuracy and efficiency of the new model while maintaining the effect of the original model.

[0025] S5. Training and verification using pulse camera image dataset: Use the prepared annotated pulse camera image dataset for training, and improve the model training speed and pulse image recognition ability through the training process of its model improvement. Specifically, reference the pulse camera image dataset in the training file, and modify the debugging parameters before and after training to achieve the best training effect. Through the training of its model, obtain the optimal weight and process record of the final training completion, and check whether the results meet the conditions for completing the experiment.

[0026] The method of the present invention utilizes a fast attention mechanism to construct a brand-new pulse image target recognition algorithm model, so that the model supports pulse image recognition in low-resolution or extreme lighting environments. The neural network inside the algorithm model can adaptively use the fast attention mechanism to automatically extract the basic features of multi-scale images, and utilize the self-attention mechanism to optimize the learning efficiency of the model, accurately detect pulse images with poor recognition conditions, thereby solving the problems of lack of pulse image target detection and low recognition efficiency, slow speed and poor quality in existing methods.

[0027] like Figure 4 FIG. 1 is a flowchart of the method of the present invention during implementation. From the beginning to the end, the specific steps are as follows: Step 1: Prepare the pulse image dataset, such as Figure 2 FIG. 1 is a schematic diagram of preparing a pulse image training data set according to the present invention. Step 1 includes:

[0028] 1.1 Adjust the image The prepared images are resized to a uniform size, such as 1920*1080, and numbered to obtain the resized images. They are allocated and stored as training sets, test sets, and validation sets as required, so that they can be retrieved and referenced in future training.

[0029] 1.2 Object Annotation The objects that need to be identified and detected in the image are labeled. In this experiment, the objects can be labeled according to the YOLOv8 detection model labeling format requirements and the data needs to be normalized. A labeling file is generated for each image and storage is allocated according to demand.

[0030] Step 2: Use pulse image training, including: 2.1 Setting up the training model Create a pulse image training data set import file, set the type and number of objects to be identified, and introduce them into the model. Set parameters including training equipment, number of iterations, and whether to verify in the training file, and try to adjust the training parameters before and after training to achieve the best training effect.

[0031] 2.2 Training Run the training file to get the training process and result demonstration diagram. After checking that the training results are valid, save the best weights for subsequent model image inference.

[0032] Step 3: Use optimal weight inference, such as Figure 5 , which is a schematic diagram of the reasoning results in the implementation method of the present invention, is as follows: 3.1 Setting up the inference model Create an import file for the pulse image inference dataset to be detected and introduce it into the model. Reference the best model weight obtained after training in the inference file.

[0033] 3.2 Put into use In the foreseeable future, the pulse image target detection model will be put into practical application. By utilizing the characteristics of high-speed photography of pulse cameras, it can be used in high-speed pulse camera photography scenarios that require high frame rate and high refresh rate, such as sports event photography, high-speed movie photography, and scientific experiment recording photography, and can perform target detection on objects that need to be identified in photographed pictures or videos.

[0034] Table 1 Comparison of detection results between the fast detection model of the present invention and the YOLOv8 model

[0035] As shown in Table 1 above, the detection results of the fast detection model of the present invention are compared with those of the YOLOv8 model. From the results in Table 1, it can be seen that the present invention studies a method for pulse source image target detection based on "Flash Attention", and successfully verifies it on pulse visual target detection. This method improves the traditional attention mechanism by designing the backbone network structure of the target detection model based on the convolutional neural network mechanism, designs a "fast attention" mechanism layer in the model, and introduces it in the backbone network structure, thereby realizing the adaptive use of "fast attention" and multi-scale feature extraction. The model has a self-attention mechanism layer that performs better in complex tasks, is more powerful, and has higher efficiency and utilization, thereby enhancing the model's ability to process features of different sizes, shapes, and details, and making it more robust in complex real-world scenes, solving the problems of insufficient recognition ability of high-speed pulse stream images in extreme lighting environments in existing models, and low model training speed and resource utilization, and finally realizing the application of target detection in high-speed pulse stream images in the future.

[0036] The above description is the best embodiment based on the concept and working principle of the invention. The above embodiment should not be understood as limiting the protection scope of the present claims, and other implementations and combinations of implementations according to the concept of the present invention belong to the protection scope of the present invention.

Claims

1. A pulse image target detection method based on fast attention, characterized in that: The following steps are involved: S1. Prepare pulse camera image dataset; S2. Construct a fast attention mechanism definition layer; S3. Design the backbone neural network structure; S4. Adaptive invocation of fast attention mechanism; and S5. Use the pulse camera image dataset for training and verification.

2. The pulse image target detection method based on rapid attention according to claim 1 is characterized in that: In step S1, the input images are uniformly adjusted to images of the same resolution, and then the objects to be detected in the images are annotated. The annotation format must meet the model requirements. One image corresponds to one annotation file, and the image and image annotation labels are obtained respectively. The data are classified according to the training set, test set and evaluation set and stored according to the model format requirements.

3. The pulse image target detection method based on rapid attention according to claim 1 is characterized in that: In step S2, the fast attention mechanism definition layer is constructed so that it can be adaptively referenced in the backbone neural network, and the model code is modified to make the fast attention mechanism definition layer compatible with the model.

4. The pulse image target detection method based on rapid attention according to claim 1 is characterized in that: In step S3, the backbone neural network structure of the model is designed, and the fast attention mechanism definition layer constructed by reference can be inserted before different convolutional layers (C2F layers), and the convolutional layer is added when the fast attention mechanism layer is inserted again before the fast spatial pyramid pooling layer to achieve the effect of multi-scale feature extraction.

5. The pulse image target detection method based on rapid attention according to claim 1 is characterized in that: In step S4, when called in the backbone neural network, the adaptive program in the constructed fast attention mechanism definition layer will determine whether the current layer belongs to a non-important layer where efficiency can be improved, and implement adaptive calling of the attention mechanism to further improve the accuracy and efficiency of the new model while maintaining the effect of the original model.

6. The pulse image target detection method based on rapid attention according to claim 1 is characterized in that: In step S5, the pulse camera image dataset is referenced in the training file, and the debugging parameters are modified before and after training to achieve the best training effect. Through the training of its model, the best weight and process record of the final training are obtained, and it is checked whether the result meets the conditions for completing the experiment.

Images