High-speed target detection method, system and device based on pulse and event fusion data
Through the fusion data processing of event cameras and pulse cameras, real-time detection of high-speed moving targets and reconstruction of scene details are achieved, solving the problem of real-time tracking and background information capture of traditional cameras in high-speed target detection, and improving detection acuity and information acquisition efficiency.
Patent Information
- Application Number
- CN202411295499.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-18
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-09-18
AI Technical Summary
Traditional CCD and CMOS cameras are difficult to achieve real-time detection and tracking of high-speed targets, event cameras are only sensitive to motion and difficult to capture background information, and pulse cameras are slow to detect under low light intensity conditions.
Using fusion data based on pulse and event cameras, the pixel-level image fusion of event cameras and pulse cameras is realized through time synchronization and spatial alignment calibration, and the motion capture capability of event cameras and the full-field light intensity information of pulse cameras are used to achieve high-speed targets. Real-time detection tracking and background reconstruction of motion scenes.
Real-time detection of high-speed motion targets and reconstruction of scene details are achieved, and the detection sensitivity and acquisition of scene information are improved. Compared with the event or pulse camera alone, it has lower data bandwidth and richer scene details information.
Smart Images

Figure CN118828149B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computer vision and relates to a high-speed target detection method, system and device based on pulse and event fusion data. Background Art
[0002] High-speed target detection technology is a hot topic in scientific research and industrial applications. Traditional CCD and CMOS cameras are limited by their fixed exposure time and are difficult to detect and track high-speed targets in real time. Even if high-speed cameras with ultra-high frame rates are used for imaging, high data transmission delays will occur due to the large transmission bandwidth of the high-speed camera.
[0003] In view of the problem that traditional imaging methods are difficult to achieve real-time detection of high-speed targets, new bionic visual imaging technologies are gradually applied to high-speed target detection. Among them, event cameras and pulse cameras are two representative bionic visual imaging cameras. The pixels in the event camera will only generate output when the brightness change exceeds the threshold. The integration time of each pixel to count the brightness change is usually within 10 microseconds. Therefore, the event camera can achieve asynchronous data output with ultra-high temporal resolution. At the same time, the output values of the event camera are only 1 and 0, corresponding to the increase and decrease of brightness, respectively, so the data output bandwidth of the event camera is also very low. The pulse camera imitates the information transmission structure of the retinal synaptic cells of primates. Each pixel independently senses the accumulation of photons. When the accumulation reaches the set threshold, a pulse output will be performed. The pulse output frequency of each pixel is proportional to the light intensity at the pixel. This structure can obtain the visual information of the scene with a lower data output bandwidth, and then use the corresponding pulse decoding algorithm to reconstruct the motion scene with high frame rate and high texture details.
[0004] However, there are still some problems in using event cameras or pulse cameras alone to detect and track high-speed targets. First, event cameras are only sensitive to motion and have difficulty capturing background information of moving scenes. They may also lose detailed information about areas where the brightness of the moving target surface remains unchanged. Second, the pulse output frequency of the pulse camera is proportional to the light intensity. When the light intensity on the surface of the moving target is very low, the pulse camera's detection of the moving target will become slow. Summary of the invention
[0005] The purpose of the present invention is to solve the above technical problems and provide a high-speed target detection method, system and device based on pulse and event fusion data.
[0006] The present invention provides a high-speed target detection device based on pulse and event fusion data, comprising a mounting seat and an outer cover, wherein an event camera and a pulse camera for capturing feature point images are arranged in the outer cover, a mainboard box is arranged in the middle of the event camera and the pulse camera, the mainboard box is fixedly connected in the outer cover, and control components for determining the relative positions of the event camera and the pulse camera are arranged on both sides of the mainboard box;
[0007] The control component includes a control pull plate, a side sliding hole is opened on the side wall of the outer cover, and the side wall of the outer cover is placed on one side of the side sliding hole and may have a scale line. The control pull plate slides through the side sliding hole, and one end of the side sliding hole placed outside the outer cover is fixedly connected with a scale pointer, and the tip of the scale pointer points to the scale line.
[0008] Specifically, side brackets are fixedly connected on both sides of the mainboard box, and sliding seats are slidably provided on the side walls of the side brackets, and the event camera and the pulse camera are fixedly connected to the corresponding side walls of the sliding seats; a guide support plate is fixedly connected to the upper surface of the side bracket close to the event camera, a sliding track is opened in the middle of the guide support plate, and a sliding member is slidably provided in the middle of the sliding track, and a first connecting rod and a second connecting rod are respectively fixedly connected on both sides of the sliding member, the first connecting rod is connected to the sliding seat close to the event camera, and the second connecting rod is connected to the sliding seat close to the pulse camera.
[0009] Specifically, a plug-in groove is provided on the upper surface of the sliding seat, the tops of the first connecting rod and the second connecting rod are both plugged into the corresponding plug-in grooves, and a side tightening groove is provided on the side wall of the sliding seat; one end of the control pull plate placed inside the outer cover is plugged into the side tightening groove, and one end of the control pull plate placed in the side tightening groove is fixedly connected to a friction strip plate, and the end of the friction strip plate close to the control pull plate is fixedly connected to a tightening spring, and the end of the tightening spring away from the friction strip plate is fixedly connected to the inner wall of the side tightening groove.
[0010] Specifically, the outer wall of the outer cover is connected to an adjusting motor, and the adjusting motor is fixedly connected to the outer cover through a mounting bracket. The output end of the adjusting motor is fixedly connected to a threaded rod, and the threaded rod extends to the end of the side bracket close to the event camera, and the threaded rod is threadedly connected to the sliding seat.
[0011] The present invention also provides a high-speed target detection method based on pulse and event fusion data. The method utilizes the motion capture capability of the event camera to achieve real-time detection and tracking of the target, and then utilizes the full-field light intensity information of the pulse camera to complete the background reconstruction of the motion scene and the detailed texture reconstruction of the motion target. The method specifically includes the following steps:
[0012] Step 1: Create a high-speed target detection system. Two synchronous trigger signals of the high-speed target detection system are input to the event camera and the pulse camera as external triggers to achieve time synchronization between the event camera and the pulse camera.
[0013] Step 2: Reconstruct the synchronization frame based on the synchronization trigger signal of the event camera and the pulse camera to complete the spatial alignment calibration of the two;
[0014] Step 3: Calibrate the image of the event camera according to the result of the spatial alignment calibration, and then perform pixel-level image fusion on the event data and pulse data after the temporal and spatial alignment;
[0015] Step 4: Use fused data and target detection algorithms to achieve real-time detection and tracking of high-speed moving targets in motion scenes.
[0016] The present invention also provides a high-speed target detection system, comprising an event camera, a pulse camera and an FPGA;
[0017] Event cameras are used for event imaging of moving targets;
[0018] Pulse cameras are used to reconstruct global texture details of moving scenes;
[0019] The FPGA is responsible for achieving time synchronization between the two cameras.
[0020] Specifically, the specific process of step one is:
[0021] A periodic pulse signal is generated by using FPGA, and the pulse signal is input into the event camera and the pulse camera as an external trigger signal;
[0022] The event camera and the pulse camera reconstruct an event image and a pulse image respectively from the first rising edge to the second rising edge of the pulse signal.
[0023] Specifically, the event image reconstructed by the event camera and the pulse image reconstructed by the pulse camera are synchronized in time.
[0024] Specifically, the event image is an event frame image obtained by integrating the light intensity change event using the adjacent rising edge timestamps of the trigger event output by the event camera as the start timestamp and the end timestamp;
[0025] The pulse image is a pulse frame image obtained by using the adjacent rising edge timestamps of the pulse signal as the start timestamp and the end timestamp and performing texture decoding in combination with a pulse delay texture decoding algorithm.
[0026] Specifically, the specific process of step 2 is:
[0027] Firstly, a time-synchronized event camera and a pulse camera are used to collect feature point images and detect the pixel coordinates of the feature points;
[0028] Then, the affine transformation matrix is calculated based on the pixel coordinates of the feature points and saved for spatial alignment calibration.
[0029] Specifically, the specific process of step three is:
[0030] Performing spatial calibration on the event frame image according to the affine transformation matrix saved in step 2;
[0031] Then the calibrated event frame image and the pulse frame image are fused at the pixel level to obtain a fused image.
[0032] Specifically, the spatial calibration is to perform affine transformation on the event frame image based on the size of the pulse frame image as the reference size. After the transformation, the pixel values whose pixel coordinate values exceed the reference size range are directly discarded, and the blank areas within the reference size range are filled with zeros.
[0033] The beneficial effects of the present invention are as follows: the present invention can adjust the positions of an event camera and a pulse camera according to specific usage scenarios, and the installation position relationship between the event camera and the pulse camera. Because the two cameras differ in design, function and application scenarios, their installation positions will also be different. The positions of the event camera and the pulse camera are adjusted according to actual needs, and the position relationship between the event camera and the pulse camera can be intuitively seen through the coordination of scale pointers and scale lines. The present invention is used to solve the problem of real-time detection of high-speed moving targets. The two cameras use a synchronous trigger signal to achieve time synchronization and use affine transformation calibration to achieve spatial alignment. The present invention realizes reconstruction of high-speed moving scenes and real-time detection and tracking of targets with a lower data bandwidth. Compared with the target detection method using an event camera alone, the present invention has more scene detail information, and also has a more acute moving target detection capability than using a pulse camera alone. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 It is a schematic diagram of the three-dimensional structure of the high-speed target detection device of the present invention;
[0035] Figure 2 It is a schematic diagram of the three-dimensional structure of the mainboard box of the present invention;
[0036] Figure 3 It is a schematic diagram of the three-dimensional structure of the regulating motor of the present invention;
[0037] Figure 4 The present invention Figure 3 Axis view of
[0038] Figure 5 The present invention Figure 4 Enlarged view of point A in the middle;
[0039] Figure 6is a schematic diagram of the three-dimensional structure of the control component of the present invention;
[0040] Figure 7 It is a flow chart of the invented target detection method;
[0041] Figure 8 It is a schematic diagram of the time synchronization strategy of the present invention;
[0042] Fig. 9 It is the spatial alignment flow chart of the present invention.
[0043] In the figure: 1, mounting seat; 2, outer cover; 21, side sliding hole; 22, scale line;
[0044] 3. Event camera; 4. Pulse camera;
[0045] 5. Mainboard box; 52. Guide support plate; 521. Sliding track; 522. Sliding member; 523. First connecting rod; 524. Second connecting rod; 53. Side bracket; 54. Sliding seat; 541. Insertion slot; 542. Side tightening slot;
[0046] 6. Adjusting motor; 61. Mounting bracket; 62. Threaded rod; 8. Control assembly; 81. Control pull plate; 82. Friction strip; 83. Clamping spring; 84. Scale pointer. DETAILED DESCRIPTION
[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0048] See also Figure 1-Figure 9 As shown, the present invention provides a high-speed target detection device based on pulse and event fusion data, comprising a mounting seat 1 and an outer cover 2, wherein an event camera 3 and a pulse camera 4 for capturing feature point images are arranged in the outer cover 2, a mainboard box 5 is arranged in the middle of the event camera 3 and the pulse camera 4, the mainboard box 5 is fixedly connected in the outer cover 2, and control components 8 for determining the relative positions of the event camera 3 and the pulse camera 4 are arranged on both sides of the mainboard box 5;
[0049] The control assembly 8 includes a control pull plate 81, a side sliding hole 21 is opened on the side wall of the outer cover 2, and a scale line 22 may be provided on the side wall of the outer cover 2 located on one side of the side sliding hole 21. The control pull plate 81 slides through the side sliding hole 21, and a scale pointer 84 is fixedly connected to one end of the side sliding hole 21 located outside the outer cover 2, and the tip of the scale pointer 84 points to the scale line 22.
[0050] The mounting base 1 can support the outer cover 2, and the outer cover 2 can store the event camera 3 and the pulse camera 4. When needed, the event camera 3 and the pulse camera 4 can slide out from the outer cover 2 to capture feature point images. The installation position relationship between the event camera and the pulse camera is different because these two cameras have differences in design, function and application scenarios, which leads to different installation positions. The positions of the event camera 3 and the pulse camera 4 are adjusted according to actual needs. The position relationship between the event camera 3 and the pulse camera 4 can be intuitively seen through the coordination of the scale pointer 84 and the scale line 22, which facilitates accurate adjustment.
[0051] Specifically, both sides of the mainboard box 5 are fixedly connected with side brackets 53, and the side walls of the side brackets 53 are slidably provided with sliding seats 54, and the event camera 3 and the pulse camera 4 are fixedly connected to the corresponding side walls of the sliding seats 54;
[0052] A guide support plate 52 is fixedly connected to the upper surface of the side bracket 53 near the event camera 3, a sliding track 521 is opened in the middle of the guide support plate 52, a sliding member 522 is slidably arranged in the middle of the sliding track 521, and a first connecting rod 523 and a second connecting rod 524 are fixedly connected on both sides of the sliding member 522, the first connecting rod 523 is connected to the sliding seat 54 near the event camera 3, and the second connecting rod 524 is connected to the sliding seat 54 near the pulse camera 4.
[0053] During actual use, the event camera 3 and the pulse camera 4 can drive the second connecting rod 524 to move synchronously through the movement of the sliding seat 54. The sliding member 522 moves along the sliding track 521 to ensure greater stability during the movement. The sliding member 522 drives the first connecting rod 523 connected to the other side to move synchronously. The first connecting rod 523 drives the pulse camera 4 to move through another connected sliding seat 54.
[0054] Specifically, a plug-in groove 541 is formed on the upper surface of the sliding seat 54, and the tops of the first connecting rod 523 and the second connecting rod 524 are both plugged into the corresponding plug-in groove 541, and a side abutment groove 542 is formed on the side wall of the sliding seat 54;
[0055] One end of the control pull plate 81 placed inside the outer cover 2 is inserted into the side tightening groove 542, and the end of the control pull plate 81 placed in the side tightening groove 542 is fixedly connected to the friction strip 82, and the end of the friction strip 82 close to the control pull plate 81 is fixedly connected to a tightening spring 83, and the end of the tightening spring 83 away from the friction strip 82 is fixedly connected to the inner wall of the side tightening groove 542.
[0056] The maximum static friction force between the clamping spring 83 and the first connecting rod 523 is much greater than the maximum static friction between the sliding seat 54 and the side bracket 53. When the positions of the event camera 3 and the pulse camera 4 need to be adjusted, the control pull plate 81 close to the side of the event camera 3 is pulled in the direction away from the event camera 3 to control the corresponding sliding seat 54 to move. During the movement, the sliding seat 54 will drive the control pull plate 81 to move synchronously. The positions of the event camera 3 and the pulse camera 4 can be intuitively observed by reading the scale lines 22 pointed to by the two sets of scale pointers 84 set on both sides, thereby achieving the purpose of accurately adjusting the positions of the event camera 3 and the pulse camera 4.
[0057] Specifically, the outer wall of the outer cover 2 is connected to an adjusting motor 6, and the adjusting motor 6 is fixedly connected to the outer cover 2 through a mounting bracket 61. The output end of the adjusting motor 6 is fixedly connected to a threaded rod 62, and the threaded rod 62 extends to the end of the side bracket 53 close to the event camera 3, and the threaded rod 62 is threadedly connected to the sliding seat 54.
[0058] By turning on the adjustment motor 6, the output end of the adjustment motor 6 can drive the threaded rod 62 to rotate. During the rotation, the threaded rod 62 can drive the event camera 3 and the pulse camera 4 to move to the outside of the outer cover 2 or enter the outer cover 2 for storage through the sliding seat 54. By pulling the control pull plate 81 close to the side of the event camera 3 toward the outside, the position of the event camera 3 can be adjusted separately. After adjusting the pulse camera 4 to the preset position, the control pull plate 81 close to the side of the pulse camera 4 can be pulled toward the outside to adjust the position of the event camera 3, which is convenient and quick.
[0059] A high-speed target detection method based on pulse and event fusion data, which uses the motion capture capability of the event camera 3 to achieve real-time detection and tracking of the target, and then uses the full-field light intensity information of the pulse camera 4 to complete the background reconstruction of the motion scene and the detailed texture reconstruction of the moving target, specifically includes the following steps:
[0060] Step 1: Create a high-speed target detection system. Two synchronous trigger signals of the high-speed target detection system are input to the event camera 3 and the pulse camera 4 as external triggers, respectively, to achieve time synchronization between the event camera 3 and the pulse camera 4.
[0061] Step 2: Reconstruct the synchronization frame based on the synchronization trigger signal of the event camera 3 and the pulse camera 4 to complete the spatial alignment calibration of the two;
[0062] Step 3: Calibrate the image of event camera 3 according to the result of spatial alignment calibration, and then perform pixel-level image fusion on the event data and pulse data after spatiotemporal alignment;
[0063] Step 4: Use fused data and target detection algorithms to achieve real-time detection and tracking of high-speed moving targets in motion scenes.
[0064] The high-speed target detection system includes an event camera 3, a pulse camera 4 and an FPGA. The event camera 3 is used for event imaging of moving targets, the pulse camera 4 is used for global texture detail reconstruction of moving scenes, and the FPGA is responsible for achieving time synchronization between the two cameras.
[0065] Step 1: Use FPGA to generate a periodic pulse signal, and input the pulse signal as an external trigger signal to event camera 3 and pulse camera 4. Figure 3 As shown, the event camera 3 and the pulse camera 4 reconstruct an event image and a pulse image respectively from the first rising edge to the second rising edge of the pulse signal.
[0066] The event image reconstructed by the event camera 3 and the pulse image reconstructed by the pulse camera 4 are synchronized in time.
[0067] Among them, the event image is an event frame image obtained by integrating the light intensity change (CD) event with the adjacent rising edge timestamps of the trigger event output by the event camera 3 as the start timestamp and the end timestamp; the pulse image is a pulse frame image obtained by texture decoding with the adjacent rising edge timestamps of the pulse signal as the start timestamp and the end timestamp combined with the pulse delay texture decoding (TFL) algorithm.
[0068] Step 2: Spatial alignment process Figure 4 As shown, firstly, the time-synchronized event camera 3 and pulse camera 4 are used to collect feature point images and detect the pixel coordinates of the feature points, and then the affine transformation matrix is calculated according to the pixel coordinates of the feature points and the matrix is saved for spatial alignment calibration;
[0069] The feature point image capture is performed by using a flashing checkerboard calibration plate or a flashing dot calibration plate to perform feature point pattern imaging of the event camera 3 and the pulse camera 4; the affine transformation matrix is used to construct a pixel mapping relationship between the event frame image and the pulse frame image, and the mapping relationship formula is:
[0070]
[0071] in, is the event camera 3 feature point The homogeneous coordinates of ; is the pulse feature point The homogeneous coordinates of ; , , , , , are the parameters in the affine transformation matrix.
[0072] According to the mapping relationship of the mapping relationship formula, it can be concluded that at least three pairs of feature point coordinates are required to solve the six parameters in the affine transformation matrix. Therefore, the captured feature point pattern must contain at least three pairs of feature points, and in order to ensure the accuracy of the parameter solution, more than three pairs of feature points are used for parameter optimization.
[0073] Step 3: spatially calibrate the event frame image according to the affine transformation matrix saved in step 2, and then fuse the calibrated event frame image with the pulse frame image at the pixel level to obtain a fused image.
[0074] The spatial calibration is to perform an affine transformation on the event frame image based on the size of the pulse frame image as the reference size. After the transformation, the pixel values whose pixel coordinate values exceed the reference size range are directly discarded, while the blank areas within the reference size range are filled with zeros.
[0075] Step 4: Input the fused image data into the target detection algorithm to detect moving targets, and realize real-time detection and tracking of high-speed moving targets in the motion scene.
[0076] The target detection algorithm may be a frame difference method based on a traditional image processing algorithm, or a YOLO series algorithm based on deep learning.
[0077] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of the present invention is defined by the attached claims and their equivalents.
Claims
1. A high-speed target detection device based on pulse and event fusion data, comprising a mounting base (1) and an outer cover (2), Features: An event camera (3) and a pulse camera (4) for capturing feature point images are arranged in the outer cover (2); a mainboard box (5) is arranged in the middle of the event camera (3) and the pulse camera (4); the mainboard box (5) is fixedly connected in the outer cover (2); and control components (8) for determining the relative positions of the event camera (3) and the pulse camera (4) are arranged on both sides of the mainboard box (5); The control assembly (8) comprises a control pull plate (81), a side sliding hole (21) is formed on a side wall of the outer cover (2), a scale line (22) may be formed on the side wall of the outer cover (2) on one side of the side sliding hole (21), the control pull plate (81) slides through the side sliding hole (21), a scale pointer (84) is fixedly connected to one end of the side sliding hole (21) outside the outer cover (2), and the tip of the scale pointer (84) points to the scale line (22); Both sides of the mainboard box (5) are fixedly connected to side brackets (53), the side walls of the side brackets (53) are slidably provided with sliding seats (54), and the event camera (3) and the pulse camera (4) are fixedly connected to the corresponding side walls of the sliding seats (54); A guide support plate (52) is fixedly connected to the upper surface of the side bracket (53) on the side close to the event camera (3); a sliding track (521) is provided in the middle of the guide support plate (52); a sliding member (522) is slidably provided in the middle of the sliding track (521); a first connecting rod (523) and a second connecting rod (524) are fixedly connected to both sides of the sliding member (522); the first connecting rod (523) is connected to the sliding seat (54) on the side close to the event camera (3); and the second connecting rod (524) is connected to the sliding seat (54) on the side close to the pulse camera (4); The upper surface of the sliding seat (54) is provided with an insertion groove (541), the tops of the first connecting rod (523) and the second connecting rod (524) are both inserted into the corresponding insertion grooves (541), and the side wall of the sliding seat (54) is provided with a side tightening groove (542); One end of the control pull plate (81) placed inside the outer cover (2) is inserted into the side tightening groove (542), and one end of the control pull plate (81) placed in the side tightening groove (542) is fixedly connected to a friction strip (82), and the end of the friction strip (82) close to the control pull plate (81) is fixedly connected to a tightening spring (83), and one end of the tightening spring (83) away from the friction strip (82) is fixedly connected to the inner wall of the side tightening groove (542); An adjusting motor (6) is connected to the outer wall of the outer cover (2); the adjusting motor (6) is fixedly connected to the outer cover (2) via a mounting bracket (61); a threaded rod (62) is fixedly connected to the output end of the adjusting motor (6); the threaded rod (62) extends to an end of a side bracket (53) close to a side of the event camera (3); and the threaded rod (62) is threadedly connected to a sliding seat (54).
2. A high-speed target detection method based on pulse and event fusion data, comprising the device according to claim 1, Features: The method utilizes the motion capture capability of an event camera (3) to realize a high-speed target detection device, and then utilizes the full-field light intensity information of a pulse camera (4) to complete the background reconstruction of the motion scene and the detailed texture reconstruction of the motion target, and specifically includes the following steps: Step 1: Create a high-speed target detection system. Two synchronous trigger signals of the high-speed target detection system are respectively input to the event camera (3) and the pulse camera (4) as external triggers to achieve time synchronization of the event camera (3) and the pulse camera (4); Step 2: Reconstructing the synchronization frame based on the synchronization trigger signals of the event camera (3) and the pulse camera (4) to complete the spatial alignment calibration of the two; Step 3: Calibrate the image of the event camera (3) according to the result of the spatial alignment calibration, and then perform pixel-level image fusion on the event data and pulse data after the temporal and spatial alignment; Step 4: Use fused data and target detection algorithms to achieve real-time detection and tracking of high-speed moving targets in motion scenes.
3. A high-speed target detection method based on pulse and event fusion data according to claim 2, Features: The high-speed target detection system includes an event camera (3), a pulse camera (4) and an FPGA; Event cameras (3) are used for event imaging of moving targets; Pulse cameras (4) are used to reconstruct global texture details of moving scenes; The FPGA is responsible for achieving time synchronization between the two cameras.
4. A high-speed target detection method based on pulse and event fusion data according to claim 3, Features: The specific process of step one is: A periodic pulse signal is generated by using FPGA, and the pulse signal is input into the event camera (3) and the pulse camera (4) as an external trigger signal; The event camera (3) and the pulse camera (4) reconstruct an event image and a pulse image respectively from the first rising edge to the second rising edge of the pulse signal; The event image reconstructed by the event camera (3) and the pulse image reconstructed by the pulse camera (4) are synchronized in time; The event image is an event frame image obtained by integrating the light intensity change event using the adjacent rising edge timestamps of the trigger event output by the event camera (3) as the start timestamp and the end timestamp; The pulse image is a pulse frame image obtained by using the adjacent rising edge timestamps of the pulse signal as the start timestamp and the end timestamp and performing texture decoding in combination with a pulse delay texture decoding algorithm.
5. A high-speed target detection method based on pulse and event fusion data according to claim 4, Features: The specific process of step 2 is as follows: Firstly, a time-synchronized event camera (3) and a pulse camera (4) are used to collect feature point images and detect pixel coordinates of the feature points; Then, the affine transformation matrix is calculated according to the pixel coordinates of the feature points and the matrix is saved for spatial alignment calibration; The feature point image is captured by using a scintillation checkerboard calibration plate or a scintillation dot calibration plate to perform feature point pattern imaging of the event camera (3) and the pulse camera (4); The affine transformation matrix is used to construct the pixel mapping relationship between the event frame image and the pulse frame image. The mapping relationship formula is: in, is the feature point of the event camera (3) The homogeneous coordinates of ; is the pulse feature point The homogeneous coordinates of ; are the parameters in the affine transformation matrix.
6. A high-speed target detection method based on pulse and event fusion data according to claim 5, Features: The specific process of step three is: Performing spatial calibration on the event frame image according to the affine transformation matrix saved in step 2; Then the calibrated event frame image and the pulse frame image are fused at the pixel level to obtain a fused image.
7. A high-speed target detection method based on pulse and event fusion data according to claim 6, Features: The spatial calibration is to perform affine transformation on the event frame image based on the size of the pulse frame image as the reference size. After the transformation, the pixel values whose pixel coordinate values exceed the reference size range are directly discarded, and the blank areas within the reference size range are filled with zeros.
Citation Information
Patent Citations
Three-dimensional binocular camera platform experimental device
CN105262946A
Target region segmentation method and system for time-space event pulse stream
CN116416258A
Underwater frogman target detection method based on event camera
CN116682000A
Image generation method and device based on pulse camera, electronic equipment and medium
CN116887022A
Target tracking method and device, computer equipment and storage medium
CN117237604A
