Dynamic target tracking system and detection method based on FPGA

By utilizing the parallel computing capabilities and mathematical models of FPGA-based dynamic target tracking systems, the problems of slow processing speed, low real-time performance, and interference from multiple objects in existing dynamic target tracking systems are solved. This results in efficient and stable dynamic target recognition and tracking, with a miniaturized and low-power system.

CN114612523BActive Publication Date: 2024-11-08SOUTHEAST UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210229821.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-10
Publication Date
2024-11-08
Estimated Expiration
2042-03-10

AI Technical Summary

Technical Problem

Existing dynamic target tracking systems suffer from problems such as unsatisfactory processing speed, low real-time performance, poor scalability and portability, large recognition errors, and bulky equipment, especially in situations with multiple objects interfering with the target.

Method used

An FPGA-based dynamic target tracking system is adopted, including data signal acquisition, preprocessing, target recognition and servo control modules. The parallel computing capability of FPGA is used for frame difference processing and servo control, and similarity determination is performed by combining mathematical models to reduce interference and improve stability.

Benefits of technology

It achieves efficient dynamic target recognition and tracking, reduces interference from multiple objects, improves system stability and real-time performance, and is small-scale, low-power, and easy to move.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114612523B_ABST
    Figure CN114612523B_ABST
Patent Text Reader

Abstract

The invention discloses a dynamic target tracking system based on FPGA and a detection method thereof. A CMOS camera collects real-time video data, a VDMA performs image buffering, and then data processing is performed at a programmable logic PL end: a frame difference processing module completes target detection through two adjacent frames of images, and further processes image data at the same time, and draws the minimum circumscribed rectangle of a moving object in the image; then a mathematical model is established, and effective frame determination is performed on the data result obtained by the frame difference processing module; if the determination result is true, a steering gear control module calculates a PWM wave duty ratio required for a two-degree-of-freedom steering gear to rotate to a target position, and transmits the PWM wave duty ratio to a steering gear driving board through serial port communication, and simultaneously updates model data in the effective frame determination module; if the determination result is no, the next video data is recalculated; a display module outputs the processed image data; and the image processing speed is improved, the dynamic target is tracked in real time, and the interference of other objects in the picture is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of dynamic target tracking, and in particular relates to a dynamic target detection method and tracking system based on FPGA. Background Art

[0002] With the rapid development of computer technology, computer processing of visual information has attracted widespread attention. Computer vision is a highly interdisciplinary subject that integrates image processing, computer graphics, pattern recognition, artificial intelligence, artificial neural networks, computers, psychology, physics and mathematics. The purpose of computer vision research is to enable computers to perceive the geometric information of objects in the environment, including its shape, position, posture, movement, etc., and to describe, store, identify and understand it. As an important branch of the field of computer vision, the research and application of video target tracking methods are increasingly widely used in science and technology, national defense construction, aerospace, medicine and health, and various fields of the national economy. Therefore, the research on target tracking technology has great practical value and broad development prospects.

[0003] The existing dynamic target tracking system has the following problems: 1) Due to the lack of parallel computing capabilities, the processing speed is not ideal; 2) The real-time performance is low and the target is easily lost; 3) The scalability and portability are not ideal; 4) The recognition error is large, and when there are multiple objects in the picture, the interference processing of other objects needs to be optimized; 5) The equipment is relatively bulky and not easy to move.

[0004] As a high-performance, low-power programmable chip, Field-Programmable Gate Array (FPGA) can be applied to small-batch streaming applications and is very suitable for processing the inference stage of deep learning, which can greatly shorten the development cycle. With the improvement of the accuracy and real-time performance of moving target recognition, the computational complexity and memory requirements of deep learning algorithms have also increased dramatically. Current general-purpose processors cannot meet their computing needs. Therefore, using FPGA for moving target recognition can better achieve real-time detection speed and low-power operation state, thus adapting to the actual application scenarios of moving target recognition. Summary of the invention

[0005] Technical problem: The purpose of the present invention is to optimize the problems existing in the above-mentioned prior art and provide a dynamic target detection method and tracking system based on FPGA, so as to achieve the purpose of improving image processing speed, tracking dynamic targets in real time, and reducing interference from other objects in the picture.

[0006] Technical solution: A dynamic target tracking system based on FPGA of the present invention comprises a data signal acquisition module, a data signal preprocessing module, a target recognition module, a steering gear control module and a data signal display module;

[0007] The data signal acquisition module includes a CMOS camera, a data stream conversion module, a VDMA data cache module and an acquisition module control unit; the CMOS camera realizes the original video data acquisition function and inputs the acquired video data into the data stream conversion module; the data stream conversion module realizes the function of converting RGB888 data into axi4 data stream format, which is convenient for data transmission at the PL end of the PYNQ-Z2 development board; the VDMA data cache module realizes the image cache function and can store two frames of images for subsequent image processing; the acquisition module control unit is implemented at the PS end of the processing system to complete the camera initialization and DDR3 memory control function;

[0008] The data signal preprocessing module extracts image data from the VDMA and converts it into grayscale image data;

[0009] The target recognition module includes a frame difference operation module and a valid frame determination module; the frame difference operation module performs frame difference operation and binarization processing on the grayscale image, and fills holes and smoothes boundaries through closing operations; the valid frame determination module determines the results obtained by the frame difference operation module by establishing a mathematical model;

[0010] The steering gear control module includes a duty cycle calculation module and a serial communication module; the duty cycle calculation module calculates the duty cycle required by the steering gear according to the data determined by the valid frame to achieve object tracking; the serial communication module realizes communication with the steering gear drive board to complete the control of the two-degree-of-freedom steering gear; the two-degree-of-freedom steering gear can rotate in the X and Y directions, and the rotation angle is controlled by the input PWM wave duty cycle;

[0011] The data signal display module includes an external rectangular frame processing module and an image conversion module, which can mark the motion area with an external rectangular frame and convert the RGB888 image data into a DVI video format, so as to intuitively understand the dynamic target position and subsequent debugging;

[0012] First, the data signal acquisition module inputs real-time video data, then the data format is converted through the data signal preprocessing module, and then the target recognition module finds the effective motion area. Finally, the servo control module calculates the PWM wave duty cycle required for the servo rotation and transmits it to the servo driver board. At the same time, the data signal display module will output the data processed by the target recognition module to facilitate system debugging.

[0013] in,

[0014] The detection method comprises the following steps:

[0015] Step S1: The camera module collects real-time video data, and the data stream conversion module converts the RGB888 data into the axi4 data stream format. The data is transmitted at the PL end of the PYNQ-Z2 development board, and two consecutive frames of images are cached in the VDMA data cache module, followed by frame difference processing;

[0016] Step S2: dilate and then erode the image data after frame difference processing to achieve the functions of filling small holes in objects, connecting adjacent objects and smoothing boundaries;

[0017] Step S3: Find the boundary in the image processed by step S2 and draw the circumscribed rectangular frame of the moving object, calculate the center of the moving object; the obtained moving area parameter (X 12 ,X 22 ,X 32 ,X 42 ) and the target parameter (X 11 ,X 21 ,X 31 ,X 41 );

[0018] Step S4: Establish a similarity measurement mathematical model, and determine the similarity with the original tracking target based on the shape, area and center of the circumscribed rectangular frame. If the determination result is true, proceed to step S5; if not, return to step S1;

[0019] Step S5: Update the length, width and center data of the circumscribed rectangular frame of the original tracked object, and calculate the servo duty cycle based on the object center position obtained in S4, and transmit it to the servo driver board via serial communication to adjust the servo angle.

[0020] The step S1 comprises the following specific steps:

[0021] Step S11: Enable the two EMIO interfaces of the FPGA development board to simulate the SCCB timing to control the reading and writing of the SCCB bus, thereby completing the configuration of the camera register;

[0022] Step S12: Splice the 8-bit camera input data into 16-bit RGB565 data, and use the Videoin to axi4-stream IP core provided by VIVADO to convert the RGB888 data into axi4 data stream format, so as to facilitate data transmission on the PL side of the PYNQ-Z2 development board;

[0023] Step S13: Cache two consecutive frames of images in the onboard DDR3 of the PYNQ-Z2 development board;

[0024] Step S14: read the Nth frame and the N+1th frame image from the VDMA, convert the two frames of image into grayscale images, subtract the two frames of image, and perform binarization processing:

[0025]

[0026] Among them, P1(x,y) represents the difference between the grayscale images of the N+1th frame and the Nth frame; (x,y) represents the horizontal and vertical coordinates of the pixel point; Y(x,y) represents the brightness component of the grayscale image in the YCbCr format; T represents the set threshold.

[0027] The step S2 first dilates and then corrodes to complete the closing operation; it can fill small holes in the object, connect adjacent objects, smooth the object boundary without significantly changing the object area, and suppress details smaller than the structural unit.

[0028] The expansion defines set A as the coordinate position of the pixel point with a value of 255 after binarization, and the structure element B is a 3*3 rectangle. The origin of the structure element B is continuously translated. If the intersection of B and A is not empty, the origin position of B is recorded and recorded as set C;

[0029] The erosion defines the structural element D as Keep translating the origin of the structural element D, and record the set in which D is still included in C, recorded as set E, where E is the result of the closing operation.

[0030] The step S3 comprises the following specific steps:

[0031] Step S31: performing area detection on the result of step S2, removing connected areas whose areas are smaller than a threshold value T2, and obtaining a complete motion area;

[0032] Step S32: traverse the complete motion area obtained in step S31 to obtain the total number of image pixels;

[0033] Step S33: Project the image horizontally, record the number of pixels in each row of the image on the X-axis, and find the left and right boundaries of the motion area. Subtract the x values ​​of the right and left boundaries to get the length of the circumscribed rectangle of the motion area.

[0034] Step S34: performing weighted accumulation on the number of X-axis pixel points recorded in step S33, that is, multiplying the number of pixel points in each row by the corresponding X-axis coordinate and adding them;

[0035] Step S35: Calculate the horizontal center point of the motion area by dividing the weighted accumulated value of step S34 by the total number of image pixels traversed in step S32, and the result is the coordinate x of the horizontal center point of the motion area;

[0036] Step S36: vertically project the image, record the number of pixels in each row of the image on the Y axis, and find the upper and lower boundaries of the motion area. Subtract the y values ​​of the upper and lower boundaries to get the width of the circumscribed rectangle of the motion area.

[0037] Step S37: performing weighted accumulation on the number of Y-axis pixel points recorded in step S36, that is, multiplying the number of pixel points in each row by the corresponding Y-axis coordinate and adding them;

[0038] Step S38: Calculate the vertical center point of the motion area. Divide the weighted accumulated value of step S37 by the total number of image pixels traversed in step S32. The result is the coordinate y of the vertical center point of the motion area.

[0039] The step S4 comprises the following specific steps:

[0040] Step S41: Using the motion region parameters (X) obtained in step S3 12 ,X 22 ,X 32 ,X 42 ) and the target parameter (X 11 ,X 21 ,X 31 ,X 41 ) is used as the similarity measure, and the original target variable is recorded as (X 11 ,X 21 ,X 31 ,X 41 ), the motion region variable obtained in step S3 is (X 12 ,X 22 ,X 32 ,X 42 ), calculate the cosine similarity:

[0041]

[0042] Where (X 11 ,X 21 ) represents the original target center coordinate position, X 31 Indicates the length of the original target's circumscribed rectangle, X 41 is the width of the original target's circumscribed rectangle, (X 12 ,X 22 ) represents the coordinate position (x, y) of the center of the motion area obtained in step S3, X 32 represents the length of the circumscribed rectangle of the motion area obtained in step S3, X 42 is the width of the circumscribed rectangle of the motion area obtained in step S3; i1 represents the original target i-th variable, X i2 represents the i-th variable of the motion area;

[0043] If r 12 If the moving area obtained in step S3 is greater than the threshold value T3, it can be considered that the moving area obtained in step S3 is similar to the original target, and the process goes to step S42. Otherwise, it is determined that the result obtained in step S3 is an invalid image, and the process goes back to step S1.

[0044] Step S42: Calculate the area of ​​the motion area circumscribed rectangular frame obtained in step S3 based on its length and width, and compare it with the area of ​​the original target circumscribed rectangular frame to make an area determination:

[0045]

[0046] If r2 is less than the threshold value T4, the area determination is considered to be true and the process proceeds to step S5. Otherwise, the result obtained in step S3 is considered to be an invalid image and the process returns to step S1.

[0047] The step S5 comprises the following specific steps:

[0048] Step S51: updating the length, width and center position data of the circumscribed rectangular frame of the original tracked object based on the result obtained in step S3;

[0049] Step S52: The servo control consists of two parts: numerical conversion and serial communication control. The numerical conversion completes the conversion from the rotation angle required for camera tracking determined by the center position of the moving target to the duty cycle of the PWM wave output by the driver board. Since the camera viewing angle used by the system is small, a curve is used to fit the relationship between the pixel position in the image and the servo rotation angle, that is, the relationship between the duty cycle and the horizontal and vertical coordinates of the center position.

[0050]

[0051] Among them, A1, A2, C1, and C2 are the corresponding curve parameters of duty cycle and pixel position obtained by fitting multiple sets of data measured by the host computer software and the servo drive board;

[0052] Step S53: Using serial communication, the coordinate position (X 12 ,X 22 ) Substitute the duty cycle conversion value calculated by formula (4) and the time required to rotate the specified angle set by yourself into the driver board to achieve tracking of the dynamic target.

[0053] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0054] (1) The present invention is built on the Xilinx PYNQ-Z2 expandable platform based on the ZYNQ architecture, and completes frame difference processing, valid frame determination, image processing output, servo control and serial port communication on the PL side, and completes CMOS camera register initialization and VDMA initialization on the PS side. It not only fully utilizes the powerful parallel computing capabilities and logic resources of PYNQ-Z2, but also combines the advantages of ARM core in building operating systems and implementing complex algorithms, thereby improving the computing speed and reducing the system size;

[0055] (2) The present invention establishes a mathematical model, combines similarity determination with area determination, eliminates invalid data, reduces interference when tracking a single target with multiple objects in the background, and improves the stability of the system.

[0056] (3) The present invention realizes a miniaturized moving object tracking system with low power consumption, small size and portability.

[0057] (4) The hardware price of the present invention is advantageous compared to that of PC. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is the hardware accelerated data flow graph of the present invention.

[0059] Figure 2 It is a flow chart of moving object detection and determination of the present invention. DETAILED DESCRIPTION

[0060] In order to better understand the invention, the present invention is further described below in conjunction with examples and drawings.

[0061] The present invention is a dynamic target detection method and tracking system based on FPGA, which comprises a PYNQ-Z2 development board, a power cord, an OV5640CMOS camera, a liquid crystal display, a steering gear drive board, a two-degree-of-freedom steering gear and an HDMI cable;

[0062] The PYNQ-Z2 is a development platform that supports the PYNQ open source framework. In addition to supporting traditional ZYNQ development methods, it also supports Python for APSoC programming. The PYNQ-Z2 development board is based on the ZYNQ XC7Z020 FPGA and is equipped with Ethernet, HDMI input / output, MIC input, audio output, Arduino interface, Raspberry Pi interface, 2 Pmods, onboard DDR3, user LEDs, buttons and switches.

[0063] The ASPSoC chip includes a programmable logic PL end and a processing system PS end. The PS end is composed of a dual-core ARMCortex-A9 processor and two Neon coprocessors, etc. It also integrates a large number of peripherals and memory controllers. The PL end is a programmable logic device that realizes functions such as frame difference processing, valid frame judgment, and servo control.

[0064] The power cord is used to power the PYNQ-Z2 development board;

[0065] The system hardware acceleration data flow diagram is as follows Figure 1 As shown;

[0066] The OV5640CMOS camera is connected to the PMOD interface of the PYNQ-Z2 development board through a self-welded adapter board. Two EMIO interfaces simulate the SCCB timing to control the reading and writing of the SCCB bus, thereby completing the configuration of the camera register.

[0067] The Tx and Rx interfaces of the servo drive board are connected to the GPIO port of the PYNQ-Z2 development board to realize serial communication between the PYNQ-Z2 development board and the servo drive board;

[0068] The HDMI cable connects the HDMI OUT interface of the PYNQ-Z2 development board with the LCD display;

[0069] The DDR3 memory controller is connected to the ASPSoC chip and the SD Card interface;

[0070] The ASPSoC chip is connected to the HDMI OUT interface;

[0071] The processing system PS end is connected to the DDR3 memory controller;

[0072] The AXI data transmission bus is used to connect the PL and PS ends. The AXI bus realizes data communication between the ARM and the FPGA, effectively improving the transmission speed between the PL and the PS ends;

[0073] The programmable logic PL end includes a VDMA data storage module, a frame difference processing module, a valid frame determination module, a steering gear control module, and a display output module;

[0074] The VDMA data storage module is connected to the AXI data transmission bus and is used to store and access video stream data;

[0075] The frame difference processing module is used to process VDMA image data;

[0076] The valid frame determination module is used to determine the valid frame according to the result of the frame difference processing module and exclude invalid data;

[0077] The steering gear control module is connected to the steering gear driver board through the GPIO interface, and is used to calculate the steering gear duty cycle and communicate with the steering gear driver board through the serial port;

[0078] The display output module is used to output the processed image to facilitate system debugging;

[0079] A dynamic target detection method based on FPGA includes the following steps:

[0080] S1, CMOS camera collects real-time video data, and then performs frame difference processing. First, enable the two EMIO interfaces of the FPGA development board, and let it simulate the SCCB timing to control the reading and writing of the SCCB bus, and then complete the configuration of the camera register. In this embodiment, the output pixel of OV5640 is set to 640*480, and the image frame rate is 40 frames / second;

[0081] Then, the 8-bit camera input data is spliced ​​into 16-bit RGB565 data, and the VIVADO built-in Video in to axi4-stream IP core is used to convert the RGB888 data into axi4 data stream format, which is convenient for data transmission on the PL side of the PYNQ-Z2 development board, and two consecutive frames of images are cached in the onboard DDR3 of the PYNQ-Z2 development board. Finally, the Nth frame and the N+1th frame of the image are read from the VDMA and converted into grayscale images.

[0082]

[0083]

[0084] Subtract the two frames and perform binarization:

[0085]

[0086] Among them, P1(x,y) represents the difference between the grayscale images of the N+1th frame and the Nth frame; (x,y) represents the horizontal and vertical coordinates of the pixel point; Y(x,y) represents the brightness component of the grayscale image in the YCbCr format; T represents the set threshold (T=100).

[0087] S2, dilate first and then corrode to complete the closing operation. Fill the small holes in the object, connect the adjacent objects, smooth the object boundaries without significantly changing the object area, and suppress the details smaller than the structural unit to obtain the image Image1.

[0088] S3, find the boundary in the image after processing in step S2 and draw the circumscribed rectangular frame of the moving object, and calculate the center of the moving object. First, perform area detection on the result Image1 of step S2, remove the connected area with an area less than the threshold T2, and obtain the complete moving area Image. Then traverse the obtained complete moving area Image to obtain the total number of image pixels. Then project the image horizontally, record the number of pixels in each row of the image on the X-axis, and find the left and right boundaries of the moving area. Then perform weighted accumulation on the number of pixels on the X-axis, that is, multiply the number of pixels in each row by the corresponding X-axis coordinates, and add them. Finally, calculate the horizontal center point of the moving area, divide the weighted accumulation value of the horizontal projection by the total number of image pixels traversed, and the result is the horizontal center point coordinate x of the moving area. Project the image vertically, record the number of pixels in each row of the image on the Y-axis, and find the upper and lower boundaries of the moving area. Then perform weighted accumulation on the recorded number of pixels on the Y-axis, that is, multiply the number of pixels in each row by the corresponding Y-axis coordinates, and add them. Finally, the vertical center point of the moving area is calculated, and the weighted cumulative value of the vertical projection is divided by the total number of traversed image pixels. The result is the vertical center point coordinate y of the moving area;

[0089] S4, establish a mathematical model, and determine the similarity with the original tracking target based on the shape, area and center of the circumscribed rectangular frame. First, use the cosine of the angle between the motion area parameter obtained in step S3 and the target parameter as the similarity measure, and record the original target variable as (X 11 ,X 21 ,X 31 ,X 41 ), the motion region variable obtained in step S3 is (X 12 ,X 22 ,X 32 ,X 42 ), calculate the cosine similarity:

[0090]

[0091] Where (X 11 ,X 21 ) represents the original target center coordinate position, X 31 Indicates the length of the original target's circumscribed rectangle, X 41 is the width of the original target's circumscribed rectangle, (X 12 ,X 22 ) represents the coordinate position of the center of the motion area obtained in step S3, X 32 represents the length of the circumscribed rectangle of the motion area obtained in step S3, X 42 is the width of the circumscribed rectangle of the motion area obtained in step S3;

[0092] If r 12If the value is greater than the threshold value T3, it can be considered that the motion region obtained in step S3 is similar to the original target. Otherwise, it is determined that the result obtained in step S3 is an invalid image and the process returns to step S1.

[0093] Next, the area is calculated from the length and width of the circumscribed rectangular frame of the motion region obtained in step S3, and compared with the area of ​​the circumscribed rectangular frame of the original target to make an area determination:

[0094]

[0095] If r2 is less than the threshold value T4, the area determination is considered to be true, and the process goes to step S5. Otherwise, the result obtained in determination S3 is an invalid image, and the process goes back to step S1.

[0096] S5, update the length, width and center data of the original tracked object's external rectangular frame, and calculate the servo duty cycle based on the object's center position obtained in S4, transmit it to the servo driver board via serial communication, and adjust the servo angle.

[0097] First, the length, width and center position data of the circumscribed rectangular frame of the original tracked object are updated based on the result obtained in step S3.

[0098] Next, the servo control is carried out, which mainly consists of two parts: numerical conversion and serial communication control. The numerical conversion completes the conversion from the rotation angle required for camera tracking determined by the center position of the moving target to the duty cycle of the PWM wave output by the driver board. Since the system uses a small camera viewing angle, a curve can be used to fit the relationship between the pixel position in the image and the servo rotation angle, that is, the relationship between the duty cycle and the horizontal and vertical coordinates of the center position:

[0099]

[0100] Among them, A1, A2, C1, and C2 are obtained by fitting multiple sets of data measured by the host computer software and the servo drive board. In this embodiment, the fitting result is:

[0101]

[0102] Finally, serial communication is used to transmit the calculated angle conversion value and the time required to rotate the specified angle to the driver board to achieve tracking of dynamic targets.

[0103] The following are specific embodiments of utilizing the present system. In the following embodiments, the system has the same image processing process, and different functions are realized by changing the device installed at the top of the two-degree-of-freedom servo. However, the scheme of the present invention is not limited to the following applications, and they will not be described one by one here.

[0104] Embodiment 1:

[0105] The two-degree-of-freedom servo is controlled by a PWM wave output by a servo drive board, and a USB camera is installed on the top, which is connected to a computer, and the tracking result is observed by the computer;

[0106] The USB camera is LRCP1080P_OV2710, photosensitive chip 1 / 2.7 inch; Pixel Size: 3um◇3um;

[0107] (1) The camera module collects real-time video data, and the data stream conversion module converts the RGB888 data into the axi4 data stream format to facilitate data transmission on the PL side of the PYNQ-Z2 development board. Two consecutive frames of images are cached in the VDMA data cache module, and then frame difference processing is performed.

[0108] (2) The image data after frame difference processing is first expanded and then eroded to fill the black holes in the object and connect the adjacent objects with the smooth boundary;

[0109] (3) Find the boundary in the image processed by step (2) and draw the circumscribed rectangular frame of the moving object, and calculate the center of the moving object;

[0110] (4) Establish a mathematical model and determine the similarity between the shape, area and center of the circumscribed rectangular frame and the original tracking target. If the determination result is true, proceed to step (5); if not, return to step (1);

[0111] (5) Update the length, width and center data of the circumscribed rectangular frame of the original tracked object, and calculate the servo duty cycle based on the object center position obtained in (4), transmit it to the servo driver board via serial communication, and adjust the servo angle.

[0112] The object is always located in the center of the video recorded by the USB camera on the top of the servo. Real-time monitoring of the moving target can be observed on the computer connected to the USB camera.

[0113] Embodiment 2:

[0114] The two-degree-of-freedom servo is controlled by a PWM wave output by a servo drive board, and a laser is installed on the top to ensure that the laser spot is always located above the moving target;

[0115] (1) The camera module collects real-time video data, and the data stream conversion module converts the RGB888 data into the axi4 data stream format to facilitate data transmission on the PL side of the PYNQ-Z2 development board. Two consecutive frames of images are cached in the VDMA data cache module, and then frame difference processing is performed.

[0116] (2) The image data after frame difference processing is first expanded and then eroded to fill the black holes in the object and connect the adjacent objects with the smooth boundary;

[0117] (3) Find the boundary in the image processed by step (2) and draw the circumscribed rectangular frame of the moving object, and calculate the center of the moving object;

[0118] (4) Establish a mathematical model and determine the similarity between the shape, area and center of the circumscribed rectangular frame and the original tracking target. If the determination result is true, proceed to step (5); if not, return to step (1);

[0119] (5) Update the length, width and center data of the circumscribed rectangular frame of the original tracked object, and calculate the servo duty cycle based on the object center position obtained in (4), transmit it to the servo driver board via serial communication, and adjust the servo angle.

[0120] No matter what trajectory the moving target moves in, the laser on the top of the servo always shines on the target, realizing real-time tracking of the moving target.

[0121] Embodiment 3:

[0122] The two-degree-of-freedom servo is controlled by a PWM wave output by a servo drive board, and a pen is mounted on the top to enable handwriting in the air.

[0123] (1) The camera module collects real-time video data and tracks hand information. The data stream conversion module converts RGB888 data into axi4 data stream format to facilitate data transmission on the PL side of the PYNQ-Z2 development board. Two consecutive frames of images are cached in the VDMA data cache module, and then frame difference processing is performed.

[0124] (2) The image data after frame difference processing is first expanded and then eroded to fill the black holes in the object and connect the adjacent objects with the smooth boundary;

[0125] (3) Find the boundary in the image processed by step (2) and draw the circumscribed rectangular frame of the moving object, and calculate the center of the moving object;

[0126] (4) Establish a mathematical model and determine the similarity between the shape, area and center of the circumscribed rectangular frame and the original tracking target. If the determination result is true, proceed to step (5); if not, return to step (1);

[0127] (5) Update the length, width and center data of the circumscribed rectangular frame of the original tracked object, and calculate the servo duty cycle based on the object center position obtained in (4), transmit it to the servo driver board via serial communication, and adjust the servo angle.

[0128] The COMS camera tracks hand movement information, and the two-degree-of-freedom servo drives the pen on top to achieve handwriting in the air.

Claims

1. A dynamic target detection method for a dynamic target tracking system based on FPGA, characterized in that: The detection method comprises the following steps: Step S1: The camera module collects real-time video data, and the data stream conversion module converts the RGB888 data into the axi4 data stream format. The data is transmitted at the PL end of the PYNQ-Z2 development board, and two consecutive frames of images are cached in the VDMA data cache module, followed by frame difference processing; Step S2: dilate and then erode the image data after frame difference processing to achieve the functions of filling small holes in objects, connecting adjacent objects and smoothing boundaries; Step S3: Find the boundary in the image processed by step S2 and draw the circumscribed rectangular frame of the moving object, calculate the center of the moving object; the obtained moving area parameter (X 12 ,X 22 ,X 32 ,X 42 ) and the target parameter (X 11 ,X 21 ,X 31 ,X 41 ); Step S4: Establish a similarity measurement mathematical model, and determine the similarity with the original tracking target based on the shape, area and center of the circumscribed rectangular frame. If the determination result is true, proceed to step S5; if not, return to step S1; Step S41: Using the motion region parameters (X) obtained in step S3 12 ,X 22 ,X 32 ,X 42 ) and the target parameter (X 11 ,X 21 ,X 31 ,X 41 ) is used as the similarity measure, and the original target variable is recorded as (X 11 ,X 21 ,X 31 ,X 41 ), the motion region variable obtained in step S3 is (X 12 ,X 22 ,X 32 ,X 42 ), calculate the cosine similarity: Where (X 11 ,X 21 ) represents the original target center coordinate position, X 31 Indicates the length of the original target's circumscribed rectangle, X 41 is the width of the original target's circumscribed rectangle, (X 12 ,X 22 ) represents the coordinate position (x, y) of the center of the motion area obtained in step S3, X 32 represents the length of the circumscribed rectangle of the motion area obtained in step S3, X 42 is the width of the circumscribed rectangle of the motion area obtained in step S3; i1 represents the original target i-th variable, X i2 represents the i-th variable of the motion area; If r 12 If it is greater than the threshold value T3, it is considered that the motion region obtained in step S3 is similar to the original target, and the process goes to step S42. Otherwise, it is determined that the result obtained in step S3 is an invalid image, and the process goes back to step S1. Step S42: Calculate the area of ​​the motion area circumscribed rectangular frame obtained in step S3 based on its length and width, and compare it with the area of ​​the original target circumscribed rectangular frame to make an area determination: If r2 is less than the threshold value T4, the area determination is considered to be true, and the process goes to step S5. Otherwise, the result obtained in determination S3 is an invalid image, and the process goes back to step S1. Step S5: Update the length, width and center data of the circumscribed rectangular frame of the original tracked object, and calculate the servo duty cycle based on the object center position obtained in S4, and transmit it to the servo driver board via serial communication to adjust the servo angle.

2. The dynamic target detection method according to claim 1, characterized in that: The step S1 comprises the following specific steps: Step S11: Enable the two EMIO interfaces of the FPGA development board to simulate the SCCB timing to control the reading and writing of the SCCB bus, thereby completing the configuration of the camera register; Step S12: Splice the 8-bit camera input data into 16-bit RGB565 data, and use the Video in to axi4-stream IP core provided by VIVADO to convert the RGB888 data into axi4 data stream format, so as to facilitate data transmission on the PL side of the PYNQ-Z2 development board; Step S13: Cache two consecutive frames of images in the onboard DDR3 of the PYNQ-Z2 development board; Step S14: read the Nth frame and the N+1th frame image from the VDMA, convert the two frames of image into grayscale images, subtract the two frames of image, and perform binarization processing: Among them, P1(x,y) represents the difference between the grayscale images of the N+1th frame and the Nth frame; (x,y) represents the horizontal and vertical coordinates of the pixel point; Y(x,y) represents the brightness component of the grayscale image in the YCbCr format; T represents the set threshold.

3. The dynamic target detection method according to claim 1, characterized in that: The step S2 first dilates and then corrodes to complete the closing operation; it can fill small holes in the object, connect adjacent objects, smooth the object boundary without significantly changing the object area, and suppress details smaller than the structural unit. The expansion defines set A as the coordinate position of the pixel point with a value of 255 after binarization, and the structure element B is a 3*3 rectangle. The origin of the structure element B is continuously translated. If the intersection of B and A is not empty, the origin position of B is recorded and recorded as set C; The erosion defines the structural element D as Keep translating the origin of the structural element D, and record the set in which D is still included in C, recorded as set E, where E is the result of the closing operation.

4. The dynamic target detection method according to claim 1, characterized in that: The step S3 comprises the following specific steps: Step S31: performing area detection on the result of step S2, removing connected areas whose areas are smaller than a threshold value T2, and obtaining a complete motion area; Step S32: traverse the complete motion area obtained in step S31 to obtain the total number of image pixels; Step S33: Project the image horizontally, record the number of pixels in each row of the image on the X-axis, and find the left and right boundaries of the motion area. Subtract the x values ​​of the right and left boundaries to get the length of the circumscribed rectangle of the motion area. Step S34: performing weighted accumulation on the number of X-axis pixel points recorded in step S33, that is, multiplying the number of pixel points in each row by the corresponding X-axis coordinate and adding them; Step S35: Calculate the horizontal center point of the motion area by dividing the weighted accumulated value of step S34 by the total number of image pixels traversed in step S32, and the result is the coordinate x of the horizontal center point of the motion area; Step S36: vertically project the image, record the number of pixels in each row of the image on the Y axis, and find the upper and lower boundaries of the motion area. Subtract the y values ​​of the upper and lower boundaries to get the width of the circumscribed rectangle of the motion area. Step S37: performing weighted accumulation on the number of Y-axis pixel points recorded in step S36, that is, multiplying the number of pixel points in each row by the corresponding Y-axis coordinate and adding them; Step S38: Calculate the vertical center point of the motion area. Divide the weighted accumulated value of step S37 by the total number of image pixels traversed in step S32. The result is the coordinate y of the vertical center point of the motion area.

5. The dynamic target detection method according to claim 1, characterized in that: The step S5 comprises the following specific steps: Step S51: updating the length, width and center position data of the circumscribed rectangular frame of the original tracked object based on the result obtained in step S3; Step S52: The servo control consists of two parts: numerical conversion and serial communication control. The numerical conversion completes the conversion from the rotation angle required for camera tracking determined by the center position of the moving target to the duty cycle of the PWM wave output by the driver board. Since the camera viewing angle used by the system is small, a curve is used to fit the relationship between the pixel position in the image and the servo rotation angle, that is, the relationship between the duty cycle and the horizontal and vertical coordinates of the center position. Among them, A1, A2, C1, and C2 are the corresponding curve parameters of duty cycle and pixel position obtained by fitting multiple sets of data measured by the host computer software and the servo drive board; Step S53: Using serial communication, the coordinate position (X 12 ,X 22 ) Substitute the duty cycle conversion value calculated by formula (4) and the time required to rotate the specified angle set by yourself into the driver board to achieve tracking of the dynamic target.

6. A dynamic target tracking system based on FPGA for the dynamic target detection method according to claim 1, characterized in that: The system includes a data signal acquisition module, a data signal preprocessing module, a target recognition module, a steering gear control module and a data signal display module; The data signal acquisition module includes a CMOS camera, a data stream conversion module, a VDMA data cache module and an acquisition module control unit; the CMOS camera realizes the original video data acquisition function and inputs the acquired video data into the data stream conversion module; the data stream conversion module realizes the function of converting RGB888 data into axi4 data stream format, which is convenient for data transmission at the PL end of the PYNQ-Z2 development board; the VDMA data cache module realizes the image cache function and can store two frames of images for subsequent image processing; the acquisition module control unit is implemented at the PS end of the processing system to complete the camera initialization and DDR3 memory control function; The data signal preprocessing module extracts image data from the VDMA and converts it into grayscale image data; The target recognition module includes a frame difference operation module and a valid frame determination module; the frame difference operation module performs frame difference operation and binarization processing on the grayscale image, and fills holes and smoothes boundaries through closing operations; the valid frame determination module determines the results obtained by the frame difference operation module by establishing a mathematical model; The steering gear control module includes a duty cycle calculation module and a serial communication module; The duty cycle calculation module calculates the duty cycle required by the servo according to the data determined by the valid frame, so as to realize the tracking of the object; the serial communication module realizes the communication with the servo drive board, and completes the control of the two-degree-of-freedom servo; the two-degree-of-freedom servo can rotate in the X and Y directions, and the rotation angle is controlled by the input PWM wave duty cycle; The data signal display module includes an external rectangular frame processing module and an image conversion module, which can mark the motion area with an external rectangular frame and convert the RGB888 image data into a DVI video format, so as to intuitively understand the dynamic target position and subsequent debugging; First, the data signal acquisition module inputs real-time video data, then the data format is converted through the data signal preprocessing module, and then the target recognition module finds the effective motion area. Finally, the servo control module calculates the PWM wave duty cycle required for the servo rotation and transmits it to the servo driver board. At the same time, the data signal display module will output the data processed by the target recognition module to facilitate system debugging.