Image detection method and device based on pulse image reconstruction, equipment and medium

Abstract

The application relates to the technical field of image detection and processing, and more specifically, the application relates to an image detection method and device based on pulse image reconstruction, equipment and a medium. The method comprises the following steps: acquiring a target pulse image, and determining a plurality of key frame images in the target pulse image; reconstructing the plurality of key frame images through a TFP reconstruction module in an image FPGA reconstruction image detection system to obtain a reconstructed image; and detecting the reconstructed image through a CNN detection module in the image FPGA reconstruction image detection system to obtain a recognition result, wherein the CNN detection module is pre-trained. The application solves the problem that reconstruction algorithms need to interact with CNNs in CPU implementation and waste a large amount of bandwidth, reduces the delay, improves the detection speed after reconstruction, and therefore selects to simultaneously implement detection and reconstruction on an FPGA. The detection adopts the form of a convolutional neural network accelerator, greatly improves the detection speed and detection accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image detection and processing, and more particularly to an image detection method and device based on pulse image reconstruction, an image detection equipment and a medium. BACKGROUND

[0002] Real-time high-speed multi-target tracking has become an urgent demand for autonomous driving and intelligent video analysis. Traditional cameras cannot keep up with the speed of object motion, and inevitably introduce motion blur due to insufficient sampling rate. The pixels of a pulse camera output pulses asynchronously at a high temporal resolution according to the change of scene light intensity, which can well capture high-speed motion in the scene, and the output pulses are naturally suitable as inputs for a spiking neural network (SNN), but some existing SNN algorithms are far inferior to the performance of a convolutional neural network (CNN).

[0003] However, using a CNN for detection requires a large amount of computing power, which wastes the high-frequency characteristics of the pulses, so the CNN network is selected to be accelerated on an FPGA, and the CNN accelerator design is relatively easy to complete on the FPGA, but if the TFP algorithm is reconstructed on a CPU and then input to the neural network accelerator through a bus, the transmission delay in the middle is relatively large. SUMMARY

[0004] Based on the above technical problems, the present application aims to realize accurate and rapid detection by combining pulse image reconstruction and CNN network acceleration on an FPGA.

[0005] The first aspect of the present application provides an image detection method based on pulse image reconstruction, the method comprising:

[0006] acquiring a target pulse image and determining a plurality of key frame images in the target pulse image;

[0007] reconstructing the plurality of key frame images through a TFP reconstruction module in an image FPGA reconstruction image detection system to obtain a reconstructed image;

[0008] detecting the reconstructed image through a CNN detection module in the image FPGA reconstruction image detection system to obtain a recognition result, wherein the CNN detection module is pre-trained.

[0009] In some embodiments of the present application, the image FPGA reconstruction image detection system is designed based on an FPGA chip, and the TFP reconstruction module is deployed on the FPGA chip; the TFP reconstruction module includes an accumulator, a shifter and a ping-pong buffer to realize a sliding window function.

[0010] In some embodiments of the present application, the ping-pong buffer comprises a first ping-pong buffer and a second ping-pong buffer; the multi-frame key frame image is reconstructed by a TFP reconstruction module in an image FPGA reconstruction image detection system to obtain a reconstructed image, comprising:

[0011] The buffer step: for the multi-frame key frame image, determine the current to-be-reconstructed pixel, and store the corresponding value of the current to-be-reconstructed pixel in the preset number of frames of key frame images in the first ping-pong buffer;

[0012] The accumulation step: the number of 1s in the first ping-pong buffer is accumulated by the accumulator to obtain an accumulated sum, and the accumulated sum is stored in the second ping-pong buffer;

[0013] The shift step: the accumulated sum in the second ping-pong buffer is normalized by the shifter to obtain a first reconstructed pixel;

[0014] Repeat the buffer step, the accumulation step and the shift step until all pixels in the multi-frame key frame image are reconstructed to obtain a reconstructed image.

[0015] In some embodiments of the present application, the buffer step, the accumulation step and the shift step are repeated until all pixels in the multi-frame key frame image are reconstructed to obtain a reconstructed image, comprising:

[0016] For the remaining to-be-reconstructed pixels in the multi-frame key frame image except the current to-be-reconstructed pixel, the remaining to-be-reconstructed pixels are executed according to the buffer step, the accumulation step and the shift step to obtain reconstructed pixels corresponding to the number of remaining to-be-reconstructed pixels;

[0017] The first reconstructed pixel and the reconstructed pixels corresponding to the number of remaining to-be-reconstructed pixels are combined to form a reconstructed image.

[0018] In some embodiments of the present application, the CNN detection module comprises a plurality of convolutional layers, and a preset number of convolutional layers in front are selected to fuse the pooling function, and a convolution buffer is further arranged behind each convolutional layer with fused pooling function, wherein the preset number of convolutional layers in front are selected to fuse the pooling function, which comprises fusing a preset number of convolution kernels and a preset number of pooling kernels to realize the acceleration of the CNN detection module.

[0019] In some embodiments of the present application, the convolutional layer comprises a multiply-accumulate unit, a comparator and a selector.

[0020] In some embodiments of the present application, the reconstructed image is detected by a CNN detection module in the image FPGA reconstructed image detection system to obtain an identification result, including:

[0021] An input step: inputting the reconstructed image into the CNN detection module and obtaining weight values and biases required by the CNN detection module;

[0022] A convolution step: combining the reconstructed image with the weight values and the biases to perform first-round convolution and pooling fusion calculation through a first convolution layer to obtain a first calculation result, and storing the first calculation result in a convolution buffer;

[0023] The first calculation result is subjected to second-round convolution and pooling fusion calculation through a second convolution layer, and the convolution step is repeatedly executed until the reconstructed image is identified.

[0024] The second aspect of the present application provides an image FPGA reconstructed image detection system, which comprises:

[0025] An acquisition module configured to acquire a target pulse image and determine a plurality of key frame images in the target pulse image;

[0026] A TFP reconstruction module configured to reconstruct the plurality of key frame images to obtain a reconstructed image;

[0027] A CNN detection module configured to detect the reconstructed image to obtain an identification result, wherein the CNN detection module is pre-trained.

[0028] The third aspect of the present application provides a computer device comprising a memory and a processor, the memory stores computer readable instructions, and the computer readable instructions are executed by the processor to make the processor execute the method in each embodiment of the present application.

[0029] The fourth aspect of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method in each embodiment of the present application.

[0030] The technical scheme provided in the embodiments of the present application has at least the following technical effects or advantages:

[0031] The application obtains a target pulse image, determines a plurality of key frame images in the target pulse image, reconstructs the plurality of key frame images through a TFP reconstruction module in an image FPGA reconstruction image detection system, obtains a reconstructed image, detects the reconstructed image through a CNN detection module in the image FPGA reconstruction image detection system, and obtains a recognition result. The CNN detection module is pre-trained, solves the problem of waste of a large bandwidth caused by interaction between a reconstruction algorithm and a CPU, reduces delay, and improves the speed of detection after reconstruction. Therefore, detection and reconstruction are simultaneously implemented on an FPGA, detection is in the form of a convolutional neural network accelerator, and the detection speed and accuracy are greatly improved.

[0032] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the application. BRIEF DESCRIPTION OF DRAWINGS

[0033] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments and are not intended to limit the scope of the application. Moreover, the same reference numerals are used throughout the various drawings to designate identical parts. In the drawings:

[0034] Figure 1 A pulse image reconstruction-based image detection method step schematic diagram in an exemplary embodiment of the application is shown;

[0035] Figure 2 A TFP reconstruction module schematic diagram in an exemplary embodiment of the application is shown;

[0036] Figure 3 A sliding window schematic diagram in an exemplary embodiment of the application is shown;

[0037] Figure 4 A CNN detection module schematic diagram in an exemplary embodiment of the application is shown;

[0038] Figure 5 A convolution kernel and pooling kernel fusion working schematic diagram in an exemplary embodiment of the application is shown;

[0039] Figure 6 A convolution kernel and pooling kernel fusion convolution layer schematic diagram in an exemplary embodiment of the application is shown;

[0040] Figure 7 A pulse image reconstruction-based image detection device structure schematic diagram in an exemplary embodiment of the application is shown;

[0041] Figure 8A structural diagram of a computer device according to an example embodiment of the present application is shown. DETAILED DESCRIPTION

[0042] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. It is to be understood, however, that these descriptions are merely exemplary of the application and are intended to provide a more detailed description of the application as claimed. Further, in the following description, well-known functions or constructions are not described in detail because they would obscure the application in unnecessary detail. Obviously, the application can be practiced without one or more of these specific details. In other instances, well-known structures and technologies are not shown or described in order not to unnecessarily obscure the application.

[0043] It is to be understood that the terms used herein are merely exemplary based on the example embodiments of the present application, and are not intended to limit the scope of the example embodiments according to the present application. As used herein, the singular forms are intended to include the plural forms unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "has," when used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or any combinations thereof.

[0044] The example embodiments according to the present application will now be described in more detail with reference to the accompanying drawings. The example embodiments may, however, be implemented in many different ways and should not be construed as being limited to the embodiments set forth herein. The drawings are not drawn to scale and some details may be exaggerated for the sake of clarity. The shapes and relative sizes of the various regions, layers, and elements illustrated in the drawings are exemplary only and may vary depending on manufacturing processes or technical levels. The shapes and relative sizes of the regions, layers, and elements illustrated in the drawings are not intended to limit the scope of the present application.

[0045] The example embodiments according to the present application will now be described in more detail with reference to the accompanying drawings. The example embodiments may, however, be implemented in many different ways and should not be construed as being limited to the embodiments set forth herein. The drawings are not drawn to scale and some details may be exaggerated for the sake of clarity. The shapes and relative sizes of the various regions, layers, and elements illustrated in the drawings are exemplary only and may vary depending on manufacturing processes or technical levels. The shapes and relative sizes of the regions, layers, and elements illustrated in the drawings are not intended to limit the scope of the present application. Figure 1 - FIGS. Figure 8 Several embodiments will be given to describe the example embodiments according to the present application. It is noted that the following application scenarios are merely shown for the sake of understanding the spirit and principles of the present application, and the embodiments of the present application are not limited in this respect. On the contrary, the embodiments of the present application can be applied to any applicable scenario.

[0046] In some example embodiments of the present application, an image detection method based on pulse image reconstruction is provided, as shown in Figure 1 The method includes:

[0047] S1. Acquire the target pulse image and determine multiple key frame images in the target pulse image;

[0048] S2. The multi-frame keyframe images are reconstructed by the TFP reconstruction module in the image FPGA reconstruction image detection system to obtain the reconstructed image;

[0049] S3. The reconstructed image is detected by the CNN detection module in the image FPGA reconstruction image detection system to obtain the recognition result, wherein the CNN detection module is pre-trained.

[0050] In one specific implementation, the image FPGA reconstruction and image detection system is designed based on an FPGA chip, and the TFP reconstruction module is deployed on the FPGA chip; such as Figure 2 As shown, the TFP reconstruction module includes an accumulator and a shifter. Figure 2 (Not shown in the image) A ping-pong buffer is used to implement the sliding window function. A schematic diagram of the sliding window function is shown below. Figure 3 As shown. The ping-pong buffer includes a first ping-pong buffer (i.e. Figure 2 BRAM1) and the second ping-pong buffer (i.e. Figure 2 The process involves reconstructing the multi-frame keyframe images using the TFP reconstruction module (where TFP stands for Texture from Playback) in the image FPGA reconstruction image detection system to obtain a reconstructed image. This includes: a buffering step: determining the current pixel to be reconstructed for the multi-frame keyframe images and storing the corresponding value of the current pixel in a preset number of keyframe images in the first ping-pong buffer; an accumulation step: accumulating the number of 1s in the first ping-pong buffer using the accumulator to obtain a sum, and storing the sum in the second ping-pong buffer; a shifting step: normalizing the sum in the second ping-pong buffer using the shifter to obtain a first reconstructed pixel; repeating the buffering step, the accumulation step, and the shifting step until all pixels in the multi-frame keyframe images are reconstructed, thus obtaining a reconstructed image. It should be noted that FPGA (Field Programmable Gate Array) is a further development based on programmable devices such as PAL (Programmable Array Logic) and GAL (General Purpose Array Logic). It emerged as a semi-custom circuit in the field of Application-Specific Integrated Circuits (ASICs), addressing the shortcomings of custom circuits while overcoming the limitation of the limited gate count of traditional programmable devices. BRAM, on the other hand, is a dedicated block RAM resource added to FPGAs by FPGA manufacturers in addition to their logic resources.

[0051] Specifically, such as Figure 2As shown, assuming that 16 frames of pulse data are stored in BRAM1, the number of 1s in the sequence is calculated by the accumulator, Figure 2 The ellipsis in the sequence in the above formula indicates other pixel points. Figure 2 For example, the sequence in the above formula is 4 after accumulation, and is stored in BRAM2. Since the window is 16, the normal gray image is 0-255, and therefore normalization needs to be performed on 4. The normalization method is to multiply 4 by 256 by the window size, and the specific operation needs to be completed by shifting the shifter left by two bits. Figure 3 The sum of the number of 0s and 1s in the box in the above formula is 16, which indicates that the sliding window displays 16, and the value corresponding to the current pixel to be reconstructed in the preset number of frames of key frame images. The preset number of frames here is 16 frames, that is, when a pixel point is selected for reconstruction, the features of the pixel point in 16 frames need to be found at the same time. Of course, reference can also be made to the 17th frame. When the 17th frame arrives, the pulse data of the 17th frame and the first frame need to be referred to at the same time. There are four cases. If the 17th frame pulse is 1 and the first frame pulse is 0, the accumulated value is 4+1. If the 17th frame pulse is 0 and the first frame pulse is 1, the accumulated value is 4-1. If the 17th frame pulse is 0 and the first frame pulse is 0, the accumulated value is 4. If the 17th frame pulse is 1 and the first frame pulse is 1, the accumulated value is 4. The advantage of the sliding window is that the calculation amount is greatly reduced, and only two judgments and one addition and subtraction operation can replace the previous 16 addition operations.

[0052] In a preferred embodiment, the buffer step, the accumulation step and the shift step are repeated until all pixels in the multiple frames of key frame images are reconstructed to obtain a reconstructed image, including: for the remaining to-be-reconstructed pixels in the multiple frames of key frame images except the current to-be-reconstructed pixel, the remaining to-be-reconstructed pixels are executed according to the buffer step, the accumulation step and the shift step to obtain reconstructed pixels corresponding to the number of the remaining to-be-reconstructed pixels; and the first reconstructed pixel and the reconstructed pixels corresponding to the number of the remaining to-be-reconstructed pixels are combined into a reconstructed image.

[0053] In some embodiments of the present application, the CNN detection module includes a plurality of convolution layers, and a preset number of convolution layers at the front are selected to fuse the pooling function. A convolution buffer is further arranged behind each convolution layer with the fused pooling function. In the present application, the convolution layers with the fused pooling function include a plurality of convolution kernels and a plurality of pooling kernels, which are fused to achieve the acceleration of the CNN detection module. In some embodiments of the present application, the convolution layer includes a multiplication and accumulation unit, a comparator and a selector, which are the convolution layers with the fused pooling function. The reconstructed image is detected by the CNN detection module in the image FPGA reconstructed image detection system to obtain a recognition result, which is referred to in the above formula. Figure 6The process includes: Input steps: inputting the reconstructed image into the CNN detection module, and obtaining the weight values ​​and biases required by the CNN detection module. Figure 6 (The memory storing weight values ​​and biases is not shown in the diagram); Convolution step: The reconstructed image, combined with the weight values ​​and biases, is passed through a first convolutional layer for a first round of convolution and pooling fusion calculation to obtain a first calculation result, which is stored in a convolution buffer; The first calculation result is passed through a second convolutional layer for a second round of convolution and pooling fusion calculation, and the convolution step is executed cyclically until the reconstructed image is identified. Figure 4 As shown, the first four convolutional layers each fuse the convolutional kernel and the pooling kernel (or the first five layers). Figure 4 The convolutional layer integrates four layers, with the fifth layer being a separate convolutional kernel. A convolutional buffer is deployed after each convolutional layer. Figure 5 As shown, the combined effect of convolution and pooling in the earlier convolutional layers differs from that of using convolutional layers alone in the later layers. For example, pooling extracts the maximum value within a 2x2 pixel block and replaces the value of that 2x2 pixel block with that maximum value. Therefore, as... Figure 5 As shown, the original convolution sequence is a row-by-row scan. After incorporating pooling operations into the convolutional layer, the original feature map is divided into 2x2 blocks. Convolution is performed within these 2x2 blocks, and the result is compared with the previous pixel's result during convolution. When the convolution of the 2x2 small pixel blocks is complete, the largest result is naturally obtained. Then, other small blocks are convolved. Subsequent convolutional layers do not require pooling. When all convolutional operations are complete, the calculation result of the last convolutional layer is input into an off-chip buffer, and the reconstructed image is identified based on this result. Figure 6 In this process, N multiply-accumulate units perform simultaneous calculations to achieve acceleration. The features corresponding to a 3x3 pixel block are multiplied by their weights and then accumulated. A bias is added to the accumulated value, and the result is input to a comparator to be compared with the value of the previous pixel for pooling operations. The pooled value is then used for activation. Implementing the activation function in hardware is relatively easy. Figure 6 The comparator in the code represents the activation function. The activation function only needs one comparator to implement. Taking the ReLU function as an example, if the activation value is greater than 0, it emits itself; if the activation value is less than 0, it emits 0.

[0054] The training of the CNN detection module is performed according to a training manner commonly used in the prior art. The BN layer is fused into the convolution layer for training in a manner of BN layer fusion. The weight is quantized to 4 bits, and the feature is quantized to 8 bits, so as to be stored on the FPGA. In addition, in view of the problem that the feature map of each layer of convolution is too large to be accommodated on the FPGA, a manner of calculating only one row of convolution result each time is adopted to realize row-level pipelining. Moreover, due to the receptive field problem caused by the pooling kernel, 32 rows of pipelining are adopted for the first layer. After each layer of pooling, the level of pipelining is reduced. After 16, 8, 4 and 2 poolings, single-row pipelining is performed. For 3*3 convolution operation, 3 rows of input features are required. In order to realize row-level pipelining operation, a buffer with 4 rows of storage is required between layers (i.e. a buffer), one row of which stores the output of the upper layer, and the other three rows of which input the lower layer. A total of 4 states of the buffer can be completed.

[0055] The application obtains a target pulse image, determines a plurality of key frame images in the target pulse image, reconstructs the plurality of key frame images through a TFP reconstruction module in an image FPGA reconstruction image detection system to obtain a reconstructed image, detects the reconstructed image through a CNN detection module in the image FPGA reconstruction image detection system to obtain a recognition result, wherein the CNN detection module is pre-trained, solves the problem of waste of a large bandwidth caused by interaction between a reconstruction algorithm and a CPU in CPU implementation, reduces the delay, and improves the detection speed after reconstruction, so that detection and reconstruction are simultaneously implemented on the FPGA, the detection adopts the form of a convolutional neural network accelerator, and the detection speed and detection accuracy are greatly improved.

[0056] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the application.

[0057] In some exemplary embodiments of the application, an image detection device based on pulse image reconstruction is also provided, which executes the image detection method based on pulse image reconstruction described in the embodiments of the application. Figure 7 As shown in the figure, the device comprises:

[0058] The acquisition module 701 is configured to acquire a target pulse image, and determine a plurality of key frame images in the target pulse image.

[0059] The TFP reconstruction module 702 reconstructs the plurality of key frame images to obtain a reconstructed image.

[0060] The CNN detection module 703 detects the reconstructed image to obtain a recognition result, wherein the CNN detection module is pre-trained.

[0061] It should be understood that the general description above and the detailed description that follows are merely exemplary and explanatory and are not restrictive of the application.

[0062] It should also be emphasized that the system provided in the embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) is the use of digital computers or computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, robot technology, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0063] Reference is made below to Figure 8 which shows a schematic diagram of a computer device provided by some embodiments of the present application. As shown in Figure 8 the computer device 2 comprises a processor 200, a memory 201, a bus 202 and a communication interface 203, the processor 200, the communication interface 203 and the memory 201 are connected through the bus 202; the memory 201 stores a computer program executable on the processor 200, and the processor 200 executes the computer program to perform the image detection method based on pulse image reconstruction provided by any of the preceding embodiments of the present application.

[0064] The memory 201 can include a high-speed random access memory (RAM) and can also include a non-volatile memory such as at least one disk memory. The communication between the system network element and at least one other network element is realized through at least one communication interface 203 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. can be used.

[0065] The bus 202 can be an ISA bus, a PCI bus, an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 201 is used to store programs, and the processor 200 executes the programs after receiving execution instructions. The image detection method based on pulse image reconstruction disclosed in any of the preceding embodiments of the present application can be applied to the processor 200 or implemented by the processor 200.

[0066] The processor 200 can be an integrated circuit chip with signal processing capability. In implementation, each step of the above method can be completed by integrated logic circuits or instructions in the form of software in the processor 200. The processor 200 described above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a ready programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. Each method, step and logic block diagram disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium in the art. The storage medium is located in the memory 201, and the processor 200 reads the information in the memory 201, and combines the hardware to complete the steps of the above method.

[0067] The embodiments of the present application also provide a computer readable storage medium corresponding to the image detection method based on pulse image reconstruction provided by the foregoing embodiments, and a computer program is stored on the computer readable storage medium. When the computer program is run by a processor, the image detection method based on pulse image reconstruction provided by any of the foregoing embodiments is executed.

[0068] In addition, examples of the computer readable storage medium can also include, but are not limited to, a phase change memory (PRAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), other types of random access memory (RAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory or other optical, magnetic storage medium, which will not be described one by one here.

[0069] The embodiment of the present application further provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the image detection method based on pulse image reconstruction provided by any of the foregoing embodiments, and the method comprises: acquiring a target pulse image, and determining a plurality of key frame images in the target pulse image; reconstructing the plurality of key frame images through a TFP reconstruction module in an image FPGA reconstruction image detection system to obtain a reconstructed image; and detecting the reconstructed image through a CNN detection module in the image FPGA reconstruction image detection system to obtain a recognition result, wherein the CNN detection module is pre-trained.

[0070] It should be noted that the algorithms and displays provided herein are not inherently related to any particular computer, virtual apparatus, or other apparatus. Various general purpose devices can be used with these teachings based on the description herein. Those skilled in the art will recognize that structures required to construct such devices are readily apparent in this description. Moreover, the present application is not necessarily limited to any particular programming language. It will be appreciated that a variety of programming languages can be used to implement the teachings of the present application as described herein, and any references below to specific languages are provided for disclosure of the best mode of the present application. Numerous specific details are described herein in order to provide a thorough understanding of the application. However, it will be recognized by those of ordinary skill that the application can be practiced without some or all of these specific details. In some instances, well known methods, structures and techniques have not been described in detail in order to not obscure the understanding of this description.

[0071] Those skilled in the art can understand that various component embodiments of the present application can be implemented in hardware, or implemented in software modules running on one or more processors, or implemented in a combination thereof. Those skilled in the art should understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the apparatus for creating a virtual machine according to the embodiments of the present application.

[0072] The above description is only the preferred embodiment of the present application, but the protection scope of the present application is not limited to this, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An image detection method based on pulse image reconstruction, characterized by, The method comprises: acquiring a target pulse image and determining a plurality of key frame images in the target pulse image; reconstructing the plurality of key frame images through a TFP reconstruction module in an image FPGA reconstruction image detection system to obtain a reconstructed image; detecting the reconstructed image through a CNN detection module in the image FPGA reconstruction image detection system to obtain an identification result, wherein the CNN detection module is pre-trained; The image FPGA reconstruction image detection system is based on an FPGA chip design, the TFP reconstruction module is deployed on the FPGA chip, the CNN detection module is deployed on the FPGA chip, and the TFP reconstruction module comprises an accumulator, a shifter and a ping-pong buffer to realize a sliding window function.

2. The image detection method based on pulsed image reconstruction according to claim 1, characterized in that, The ping-pong buffer comprises a first ping-pong buffer and a second ping-pong buffer. The method comprises: buffering step: for the plurality of key frame images, determining a current pixel to be reconstructed, and storing a value corresponding to the current pixel to be reconstructed in a preset number of key frame images in the first ping-pong buffer; accumulation step: accumulating the number of 1s in the first ping-pong buffer through the accumulator to obtain an accumulation sum, and storing the accumulation sum in the second ping-pong buffer; shifting step: normalizing the accumulation sum in the second ping-pong buffer through the shifter to obtain a first reconstructed pixel; repeating the buffering step, the accumulation step and the shifting step until all pixels in the plurality of key frame images are reconstructed to obtain a reconstructed image.

3. The image detection method based on pulsed image reconstruction according to claim 2, characterized in that, The method comprises: for the remaining pixels to be reconstructed except the current pixel to be reconstructed in the plurality of key frame images, executing the remaining pixels to be reconstructed according to the buffering step, the accumulation step and the shifting step to obtain reconstructed pixels corresponding to the number of the remaining pixels to be reconstructed; combining the first reconstructed pixel and the reconstructed pixels corresponding to the number of the remaining pixels to be reconstructed into a reconstructed image.

4. The image detection method based on pulsed image reconstruction according to claim 1, characterized in that, The CNN detection module comprises a plurality of convolution layers, and a preset number of convolution layers at the front of the plurality of convolution layers are selected to fuse a pooling function, and a convolution buffer is further arranged behind each convolution layer with the fused pooling function, wherein the preset number of convolution layers at the front of the plurality of convolution layers are selected to fuse the pooling function, which comprises fusing a preset number of convolution kernels and a preset number of pooling kernels to realize the acceleration of the CNN detection module.

5. The image detection method based on pulsed image reconstruction according to claim 4, characterized in that, The convolution layer comprises a multiply-accumulate unit, a comparator and a selector.

6. The image detection method based on pulsed image reconstruction according to claim 5, characterized in that, The method comprises: input step: inputting the reconstructed image into the CNN detection module, and acquiring a weight value and a bias required by the CNN detection module; The convolution step: combining the reconstructed image, weight values and bias through a first layer of convolutional layers to perform a first round of convolution and pooling fusion calculation to obtain a first calculation result, and storing the first calculation result in a convolution buffer; The first calculation result is subjected to a second round of convolution and pooling fusion calculation through a second layer of convolutional layers, and the convolution step is repeatedly executed until the reconstructed image is identified.

7. An image FPGA reconfiguration image detection system, characterized in that, The system comprises: An acquisition module configured to acquire a target pulse image and determine a plurality of key frame images in the target pulse image; A TFP reconstruction module configured to reconstruct the plurality of key frame images to obtain a reconstructed image; A CNN detection module configured to detect the reconstructed image to obtain an identification result, wherein the CNN detection module is pre-trained. The image FPGA reconstructed image detection system is based on FPGA chip design, the TFP reconstruction module is deployed on the FPGA chip, the CNN detection module is deployed on the FPGA chip, and the TFP reconstruction module comprises an accumulator, a shifter and a ping-pong buffer to realize a sliding window function. 8.A computer device, comprising a memory and a processor, and characterized in that, The computer readable instructions are stored in the memory and executed by the processor, so that the processor executes the method of any one of claims 1-6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the method of any one of claims 1-6.

Images