Target detection method, system and terminal considering relative motion of target

Abstract

This invention belongs to the field of image processing technology and discloses a method, system, and terminal for continuous image target detection considering relative target motion. It utilizes an image target detector to detect targets in an image at time k, obtaining the corresponding detection result Y. k ; Regarding the detector's detection result Y k Y k Temporal matching is performed with the detection results obtained by the algorithm at the previous time step; based on the average brightness index g of the image at time k. k Estimate the reliability parameter p of the test results k Calculate the noise matrix R(k) and the predicted bounding box size b″ of the detection results at time k, respectively. k The Kalman filter is applied to the position of the object at time k, and the bounding box size of the object at time k is estimated by weighted fusion to form the result of image detection at time k. This invention can estimate the motion parameters and size variation patterns of targets in images based on target detection results in continuous images, providing effective data support for subsequent target localization and actual motion state estimation.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to a continuous image target detection method, system and terminal considering target relative motion. BACKGROUND

[0002] At present, as an important branch of computer vision, image target detection and recognition has important significance for computer to understand the implicit layer information in natural scene images, and is also a hot issue in the field of machine learning. With the excellent performance of AlexNe algorithm proposed by Krizhevsky et al. in 2012 in image target detection and recognition problems, various deep learning algorithms represented by deep convolutional neural network (DCNN) gradually caused a boom in the field of computer vision, followed by the wide application of related technologies in public safety, medical health, security monitoring, automatic driving and other fields. Among them, the "one-stage" method represented by YOLO uses the idea of regression to complete detection and recognition at the same time, and has shown excellent performance in real-time and accuracy. In July 2022, YOLOv7 proposed by the official team of YOLO has reached the highest accuracy of 56.8% AP of real-time target detector under the condition of 30FPS on GPU V100, and its speed and accuracy have exceeded the known target detector in the range of 5-160FPS.

[0003] With the continuous improvement of the performance of real-time target detection algorithm, this technology can be used for real-time and accurate target detection and recognition in continuous images, thereby providing data support for target tracking positioning and motion state estimation. However, when using deep neural network to detect visible light continuous images, the following defects exist: (1) the algorithm only detects and recognizes the target in the current frame image, without considering the detection and recognition results in the historical frame image, so once the current image is disturbed or the detection fails, the final detection will fail; (2) the algorithm only detects the position and size of the boundary box of the target in the image, and cannot estimate the motion state of the target in the image and the change law of the size parameter, so the data provided has limited support effect for subsequent target positioning and motion state estimation. Therefore, it is urgent to design a new continuous image target detection method considering target relative motion.

[0004] Through the above analysis, the problems and defects of the prior art are:

[0005] (1) When only using a deep neural network-based image target detector to detect continuous images, the detector only detects and recognizes the target in the current frame image, without considering the detection and recognition results in the historical frame image, so once the current image is disturbed or the detection fails, the final detection will fail;

[0006] (2) Existing continuous image target detection algorithms or video image target detection algorithms do not consider the relative motion between the target and the observation device, and cannot take into account the scale change of the target in the image caused by the change of the target's position relative to the observation center. In this context, simply correcting the detection result of the current frame based on the historical detection results will have a negative impact on the final result.

[0007] (3) Existing continuous image target detection algorithms based on deep neural networks only detect the position of the target and the size of the bounding box in the image. They cannot estimate the motion state of the target in the image and the variation law of the size parameter. The data provided has limited support for subsequent target localization and motion state estimation. Summary of the Invention

[0008] To address the problems existing in the prior art, this invention provides a continuous image target detection method, system, and terminal that takes into account the relative motion of the target.

[0009] This invention is implemented as follows: a continuous image target detection method considering relative target motion, comprising: detecting the image at time k using an image target detector to obtain the corresponding detection result Y. k ; Regarding the detector's detection result Y k Y k Compared with the detection results obtained by the algorithm at the previous time step Perform time-series matching; based on the average brightness index g of the image at time k. k Estimate the reliability parameter p of the test results k Calculate the noise matrix R(k) and the predicted bounding box size b of the detection results at time k, respectively. k "; Kalman filtering is applied to the position of the object at time k, and weighted fusion estimation is performed on the bounding box size of the object at time k to form the result of image detection at time k.

[0010] Furthermore, the continuous image target detection method considering the relative motion of the target includes the following steps:

[0011] Step 1: Obtain the raw detection data and perform time-series matching of the detection results;

[0012] Step two: Perform filtered estimation of object position and prediction of object size parameters;

[0013] Step 3: Perform current frame detection confidence estimation and image target fusion detection.

[0014] Furthermore, continuous image target detection methods specifically include:

[0015] Step 1: Obtain raw test data.

[0016] The YOLOv7 algorithm was used to detect and recognize targets in continuously captured images, and the target detection results were obtained.

[0017] Step 2: Temporal matching of detection results

[0018] For the detection result Y of the i-th object detected at time n in The detection result Y of the j-th object detected at time n+1. jn+1 Calculate the intersection-union ratio (IoU) between the bounding boxes of the two entities. ij Remember all Y jn+1 IoU ij The largest detection result is Y imn+1 The corresponding IoU value is denoted as IoU. im .

[0019] For each Y in All of them are judged as follows:

[0020] (1) Judgment of rejection of test results

[0021] If IoU im <λ d If so, it is considered that no match for Y can be found in the detection results at time n+1. in The matching test results will Y in Remove.

[0022] Where, λ d The rejection threshold can be freely set based on experience.

[0023] (2) Matching judgment of detection results

[0024] If Y in element c in and Y imn+1 element c imn+1 satisfy:

[0025] c in =c imn+1

[0026] Then it is believed that Y in and Y imn+1 The detection results of the same object generated at different times will be matched together.

[0027] (3) Judgment of missed detection of the detection object

[0028] If Y in element c in and Y imn+1 element cimn+1 satisfies:

[0029] c in ≠ c imn+1

[0030] If the object i is considered to be missed at the n+1 moment due to occlusion or other reasons, the object i is retained.

[0031] (4)Supplement judgment of detection results

[0032] After completing the matching judgment, if there is a detection result Y jn+1 cannot find a matching Y in , considering that a single frame of image may have false detection, if p in+1 satisfies the threshold condition:

[0033] p in+1 ≥ λ p

[0034] The detection result Y jn+1 is considered to be a new object at the n+1 moment for supplement. Wherein λ p is a supplement threshold value set freely according to experience.

[0035] Step 3: Current frame detection confidence estimation

[0036] The average brightness value of the whole image is selected as an index to measure the lighting condition, and for color images, the weighted average of the three components of RGB is used as the basis for calculating the brightness value of a single pixel:

[0037] g(n) = 0.299r(n) + 0.578g(n) + 0.114b(n)

[0038] The average brightness g(n) of the detection result corresponding to the image at n moment is obtained by the above formula.

[0039] According to the fitting function

[0040]

[0041] The value p(n) corresponding to it is obtained by bringing g(n), which reflects the accuracy of the detector in detecting and identifying targets in the image, as an important basis for measuring the detection confidence of the image at this moment.

[0042] Step 4: Filtering estimation of object position

[0043] Assuming that the mAP index of the image detector under normal lighting conditions is P, for the target detection result at n moment, R(n) satisfies:

[0044]

[0045] On this basis, the real position coordinates of a certain object in the image at time n are recorded as [x n ,y n ] T Considering that there is relative motion between the target and the observation device, the velocity and acceleration of the target in the x direction at time n are recorded as v xn and a xn , and the velocity and acceleration in the y direction are recorded as v yn and a yn . Since the motion states in the x and y directions can be estimated separately in the actual filtering process, the motion state of the target in the x direction at time n is recorded as X n =[x n ,v xn ,a xn ] T The transfer equation of the system state at time n and time n+1 is obtained by combining the transfer law of the target motion state:

[0046] X n+1 =Φ n,n+1 X n +v(n)

[0047] where v(n) is the system noise at time n, Φ n,n+1 is the state transition matrix of the system state from time n to time n+1, and for the uniform acceleration motion assumption, we have:

[0048]

[0049] where Δt is the time interval between the system state transition from time n to time n+1.

[0050] At time n+1, the detector obtains the image target detection result, and the observation information Z n+1 of the observed target coordinates in the image is obtained, which satisfies the observation equation between the real motion state X n of the target at time n+1:

[0051] Z n+1 =Θ n+1 X n +r(n+1)

[0052] where r(n+1) is the observation noise of the system at time n+1, Θ n+1 is the observation matrix of the system at time n+1, and for the case involved in the present application, Θ n+1 satisfies:

[0053]

[0054] On the basis of constructing the above system equation, the Kalman filtering theory is used to filter and estimate the target parameters:

[0055]

[0056]

[0057]

[0058]

[0059]

[0060] P(n+1|n+1)=[I-K n+1 Θ n+1 ]P(n+1|n)

[0061] After the above operation, the Kalman filter estimation of the target motion state vector is formed, and the estimation information of the i-th target position at time n is obtained And

[0062] Step 5: Prediction of object size parameter

[0063] Let the confidence index of the detection result of the image at time n before 20 frames be p n-20 , p n-19 , …, p n-1 , and take the average value as the confidence index of the size prediction result of the bounding box:

[0064]

[0065] Record the 20 groups of detection results obtained from time n-20 to n-1 as the dependent variable Y=[b wn-20 , b wn-18 , …, b wn-1 ] T , and record the corresponding time from time n-20 to n-1 as the independent variable X=[t n-20 , t n-18 , …, t n-1 ] T , and perform second-order Gaussian function fitting on X and Y to obtain the function of b w with time, denoted as:

[0066]

[0067] Then, t n is brought into the above function to obtain the predicted value of b w at time n at time n-1:

[0068]

[0069] Taking the width dimension as an example, let the width of the target bounding box obtained by the detector at time n be b' wn , and the width of the target bounding box obtained by the fitting function be b The estimated value of the width of the target bounding box at this time is the weighted average of the above two:

[0070]

[0071] The calculation of the height dimension is similar to the width.

[0072] Through the above process, the estimated value of the width of the i-th target bounding box at time n is b , and the estimated value of the height is h

[0073] Step 6: Result output

[0074] Let the detection result of the i-th target at time n after processing satisfy:

[0075]

[0076] where c in is the type of the target, is the coordinate of the target detection box in the X direction, is the coordinate of the target detection box in the Y direction, is the height of the detection box, is the width of the detection box.

[0077] Further, the acquisition of the original detection data in step 1 includes: using the YOLO algorithm to perform target detection and recognition in continuous image shooting, obtaining the detection result at time k, denoted as Y ik = [p ik , c ik , b xik , b yik , b h ik , b wik ] T ; where p ik is the confidence of the object appearing in the bounding box, c ik is the type number of the detected object, b xik is the horizontal coordinate of the center of the bounding box, b yik is the vertical coordinate of the center of the bounding box, b hik is the height of the bounding box, and b wik is the width of the bounding box.

[0078] Furthermore, the temporal matching of the detection results in step one includes: after the detector obtains the detection results based on the image at time k+1, since the number of objects in the detection results at both time k and time k+1 is greater than 1, a temporal matching operation is performed on the objects in the two detection results:

[0079] For the detection result Y of the i-th object obtained at time k... ik And the detection result Y of the j-th object detected at time k+1. jk+1 Calculate the IoU value between the bounding boxes of the two. ij For a certain Y ik Remember all Y jk+1 IoU ij The largest detection result is Y imk+1 The corresponding IoU value is denoted as IoU. im .

[0080] For each Y ik The following judgments are made for all cases:

[0081] (1) Judgment of rejection of test results

[0082] If IoU im <λ d If so, it is considered that no match for Y can be found in the detection results at time k+1. ik The matching test results will Y ik Remove; where λ d The rejection threshold is set freely based on experience.

[0083] (2) Matching judgment of detection results

[0084] If Y ik element c ik and Y imk+1 element c imk+1 satisfy:

[0085] c ik =c imk+1 ;

[0086] Then it is believed that Y ik and Y imk+1 The detection results of the same object generated at different times will be matched together.

[0087] (3) Judgment of missed detection of the detection object

[0088] If Y ik element c ik and Y imk+1 element c imk+1 satisfy:

[0089] cik ≠c imk+1 ;

[0090] If object i is not detected at k+1 time due to occlusion or other reasons, object i is retained.

[0091] (4)Supplement judgment of detection results

[0092] After completing the matching judgment, if there are detection results Y jk+1 that cannot find matching Y ik , considering that there may be false detection in a single image, if p ik+1 satisfies the threshold condition:

[0093] p ik+1 ≥λ p

[0094] The detection result Y jk+1 is considered as a new object at k+1 time for supplement. Wherein λ p is a supplement threshold value set freely according to experience.

[0095] Further, the filtering estimation of object position in step two is realized by using the classical Kalman filter.

[0096] According to the labeling information and the detection results of the training samples by the detector, the variance statistical information r x and r y of the parameters b h and b w in the detection result Y=[p,c,b T ,b x ,b y ,b bx ]is obtained by data comparison and statistical analysis. by

[0097] Let the real position coordinates of a certain object in the image at time n be [x n ,y n ] T , considering that there is relative motion between the target and the observation device, let the speed and acceleration of the target in the x direction at time n be v xn and a xn , and the speed and acceleration in the y direction be v yn and a yn . Since the motion states in the x and y directions are estimated separately in the actual filtering process, let the motion state of the target in the x direction at time n be X n =[x n ,v xn ,a xn ] T, combining the transfer law of target motion state, the transfer equation of system state at n time and n+1 time is obtained:

[0098] X n+1 =Φ n,n+1 X n +v(n);

[0099] Wherein, v(n) is the system noise at n time, Φ n,n+1 is the state transition matrix of system state from n time to n+1 time, for uniform acceleration motion assumption, then:

[0100]

[0101] Wherein, Δt is the time interval of system state from n time to n+1 time.

[0102] At n+1 time, the detector obtains the image target detection result, and obtains the observation information Z n+1 of the observed target coordinate in the image, the information satisfies the observation equation between the true motion state X n of the target at n+1 time:

[0103] Z n+1 =Θ n+1 X n +r(n+1);

[0104] Wherein, r(n+1) is the observation noise of system at n+1 time, Θ n+1 is the observation matrix of system at n+1 time, Θ n+1 satisfies:

[0105]

[0106] On the basis of constructing system equation, the Kalman filtering theory is used to filter and estimate the target parameters:

[0107]

[0108]

[0109]

[0110]

[0111]

[0112] P(n+1|n+1)=[I-K n+1 Θ n+1 ]P(n+1|n)。

[0113] Further, the prediction of the object size parameter in step two includes:

[0114] The observation information of the observation device on the target is analyzed for the change law with the movement of the target, and the prediction of the size parameter of the target boundary box in the subsequent image is formed according to the detection result of the continuous shooting image combined with the corresponding law.

[0115] A second-order Gaussian function is selected to represent the change law of the observation size of the target with time, and the expression is:

[0116]

[0117] The detection results in the 20 images closest to the current frame are used as the basis for predicting the size of the target boundary box in the future frame, and the specific operation is as follows: when the detector has completed the detection of the target in the image at time n, 20 sets of detection results including the detection result at time n are used to predict the size parameters b hn+1 and b wn+1 of the target boundary box at time n+1.

[0118] For the prediction of b wn+1 , 20 sets of detection results obtained at times n-19 to n are recorded as the dependent variable Y=[b wn-19 , b wn-18 , …, b wn ] T , and the corresponding times at times n-19 to n are recorded as the independent variable X=[t n-19 , t n-18 , …, t n ] T , the X and Y are fitted with a second-order Gaussian function to obtain the change function of b w with time, denoted as:

[0119]

[0120] The t n+1 is brought into the change function of b w with time to obtain the prediction value of b w at time n+1 at time n:

[0121]

[0122] Further, the current frame detection confidence estimation in step three includes:

[0123] The influence of the change of the imaging effect of the image under different light conditions on the detection effect of the final detector is analyzed, and the average brightness value of the entire image is selected as an index for measuring the light condition; and for color images, the weighted average of the RGB three components is used as the basis for calculating the brightness value of a single pixel:

[0124] g = 0.299r + 0.578g + 0.114b;

[0125] By designing experiments to record the mAP value of the detection result under different ambient brightness conditions, and recording the average brightness value g under the corresponding conditions, the influence between the average brightness value of the image and the detection effect of the detector is determined.

[0126] The data is fitted by combining the change law of the parameter mAP with g, and after data fitting, g and mAP also approximately obey the second-order Gaussian distribution:

[0127]

[0128] For the actually obtained continuous image to be detected, when the average brightness value of the image at time n is g n , the corresponding value p n is obtained by bringing the fitting function, and the size of the parameter reflects the accuracy of the detector in detecting and identifying the target in the image, so it is an important basis for measuring the detection credibility of the image at time n.

[0129] Further, the image target fusion detection in step three includes: completing the filtering estimation of the target position, the estimation of the target boundary box size, and the estimation of the credibility of the current frame detection result according to the existing detection results in the continuous shooting images, and estimating the position and size parameters of the target in the current image by comprehensively estimating various information;

[0130] (1) Estimation of target position

[0131] When the noise matrix of the observation device is determined through experiments and statistics as R, and the average brightness parameter g n at time n is estimated to obtain the credibility index p n of the detection result, and the noise matrix R(n) of the detection result at time n is obtained by combining p k and R; under normal lighting conditions, the mAP index of the image detector is P, and R(n) satisfies:

[0132]

[0133] The target position filtering estimation considering the target detection credibility is realized by adjusting the noise matrix parameter at time n.

[0134] (2) Estimation of target size

[0135] Let the credibility indexes of the detection results of the previous 20 images at time n be p n-20 , p n-19 , …, p n-1 , and the average value is taken as the credibility index of the size prediction result of the boundary box:

[0136]

[0137] The target bounding box width obtained by the detector at the n moment is b' wn The target bounding box width obtained by the fitting function is The estimated value of the target bounding box width at the n moment is the weighted average of the two:

[0138]

[0139] Another object of the present application is to provide a target detection system considering target relative motion of continuous images applying the target detection method of continuous images considering target relative motion, the target detection system considering target relative motion of continuous images comprising:

[0140] An original detection data acquisition module is configured to detect and identify targets in continuous images by using a YOLO algorithm to obtain detection results at the k moment;

[0141] A detection result time sequence matching module is configured to perform time sequence matching operation on objects in the detection results at the k moment and the k+1 moment after the detector obtains the detection results according to the image at the k+1 moment;

[0142] An object position filtering estimation module is configured to realize filtering estimation of object positions by using Kalman filtering;

[0143] An object size parameter prediction module is configured to predict the size parameters b hn+1 and b wn+1 of the target bounding box at the n+1 moment by using 20 groups of detection results including the detection result at the n moment;

[0144] A current frame detection confidence estimation module is configured to select the average brightness value of the whole image as an index for measuring the light condition, and calculate the single pixel brightness value by using the weighted average of the RGB three components;

[0145] An image target fusion detection module is configured to realize target position filtering estimation considering target detection confidence by adjusting the noise matrix parameters at the n moment, and realize target size estimation by using a fitting function.

[0146] Another object of the present application is to provide a computer device, which comprises a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the steps of the target detection method of continuous images considering target relative motion.

[0147] Another object of the present application is to provide a computer readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method for target detection in continuous images considering relative motion of the target.

[0148] Another object of the present application is to provide an information data processing terminal for implementing the system for target detection in continuous images considering relative motion of the target.

[0149] In combination with the above technical solutions and the technical problems solved, the technical solutions of the present application have the following advantages and positive effects:

[0150] First, in view of the technical problems existing in the prior art and the difficulty in solving the problems, the technical solutions of the present application and the results and data obtained during the research and development are combined to analyze in detail and profoundly how the technical solutions solve the technical problems and bring some creative technical effects after solving the problems. The specific description is as follows:

[0151] The present application provides a method for target detection in continuous images considering relative motion of the target, which can predict the target detection result of the target that has not yet appeared according to the target detection and recognition result in the existing continuous image, and further accurately estimate the real position and size of the target by combining the newly generated detection and recognition result.

[0152] Second, from the perspective of the product as a whole, the technical solutions of the present application have the following technical effects and advantages:

[0153] (1) The present application further accurately estimates the position and size of the target in the image according to the historical results in the continuous image on the basis of the existing image target detector, thereby eliminating the influence of environmental light and obstacle shielding and other factors on the detection and improving the accuracy of the target detection result in the image.

[0154] (2) The method designed by the present application can estimate the motion parameters and size change rule of the target in the image according to the target detection result in the continuous image, and the related parameters can provide effective data support for subsequent target positioning and actual motion state estimation.

[0155] Third, as the creative evidence of the invention of the claims of the present application, the expected income and commercial value of the technical solution of the present application after transformation are:

[0156] The technical solution of the present application can be used in security monitoring, traffic monitoring and other fields with high dependence on video image target detection and recognition after transformation.

[0157] If applied to security monitoring, the corresponding technology can effectively solve the continuous detection and accurate positioning of the target under the condition of multiple occlusions and multiple missed detections, and further improve the accuracy of target bounding box detection, which is an important basis for reflecting the shape characteristics of typical targets such as personnel and vehicles.

[0158] If applied to the field of traffic monitoring, the corresponding technology can further realize the detection and estimation of target motion on the basis of effectively solving the occlusion problem, thereby providing important data support for traffic accident identification, key target positioning tracking and target motion trajectory prediction. BRIEF DESCRIPTION OF DRAWINGS

[0159] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiments of the present application will be briefly introduced as follows. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0160] Figure 1 is a flowchart of the continuous image target detection method considering target relative motion provided by the embodiments of the present application;

[0161] Figure 2 is a principle diagram of the continuous image target detection method considering target relative motion provided by the embodiments of the present application;

[0162] Figure 3 is a simulation schematic diagram of the target observation size change trend over time provided by the embodiments of the present application;

[0163] Figure 4 is a comparison diagram of the fitting effect of the simulation data of the second-order Gaussian function provided by the embodiments of the present application;

[0164] Figure 5 is a graph of the change of mAP with g value under the test conditions provided by the embodiments of the present application;

[0165] Figure 6 is a comparison diagram of the fitting effect of the test data of the second-order Gaussian function provided by the embodiments of the present application;

[0166] Figure 7 is a continuous image of the key frame of the test scene provided by the embodiments of the present application;

[0167] Figure 8 is the direct detection result of the standard model YOLOv7 provided by the embodiment of the application;

[0168] Figure 9 is the detection result after the algorithm is designed and processed provided by the embodiment of the application. DETAILED DESCRIPTION

[0169] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0170] In view of the problems in the prior art, the present application provides a continuous image target detection method, system and terminal considering target relative motion, which will be described in detail below with reference to the accompanying drawings.

[0171] I. Explanation of embodiments. In order to enable those skilled in the art to fully understand how the present application is specifically implemented, this part is an explanation of the embodiments of the technical solutions of the claims.

[0172] As shown in Figure 1 , the continuous image target detection method considering target relative motion provided by the embodiment of the application includes the following steps:

[0173] S101, predicting the position of the target in the current frame image according to the detection and recognition result of the detector in the existing frame image;

[0174] S102, predicting the size of the target in the current frame image according to the detection and recognition result of the detector in the existing frame image;

[0175] S103, accurately estimating the position and size of the target in the current frame by comprehensively considering the prediction results of the position and size of the target in the current frame image, the detection and recognition result of the detector in the current frame image and the confidence of target detection and recognition in the current frame image.

[0176] In the embodiment of the present application, the continuous image target detection method specifically includes:

[0177] Step 1: Obtain the original detection data.

[0178] In the continuously captured images, YOLOv7 algorithm is used for target detection and recognition to obtain the target detection result.

[0179] Step 2: Time sequence matching of detection result

[0180] For the detection result Y of the i-th object detected at time n in, and the detection result Y of the jth object detected at the n+1 moment jn+1 , calculate the intersection over union value IoU between the two bounding boxes ij , record all Y jn+1 , let IoU ij max be the maximum detection result imn+1 , and the corresponding IoU value is denoted as IoU im .

[0181] For each Y in , the following judgment is made:

[0182] (1) Elimination judgment of detection result

[0183] If IoU im <λ d , it is considered that there is no detection result in the detection result at the n+1 moment that matches Y in , and Y in is eliminated.

[0184] Where λ d is a freely set elimination threshold value according to experience.

[0185] (2) Matching judgment of detection result

[0186] If the elements c in in Y in and the elements c imn+1 in Y imn+1 satisfy:

[0187] c in =c imn+1

[0188] then Y in and Y imn+1 are considered to be detection results of the same object at different moments, and they are matched with each other.

[0189] (3) Missing detection judgment of detection object

[0190] If the elements c in in Y in and the elements c imn+1 in Y imn+1 satisfy:

[0191] c in ≠c imn+1

[0192] then it is considered that object i is missed at the n+1 moment due to occlusion or other reasons, and object i is retained.

[0193] (4) Supplement judgment of detection result

[0194] After completing the matching judgment, if there is a detection result Y jn+1 No matching Y found in Considering the possibility of false detections in a single frame image, if p in+1 Threshold condition is met:

[0195] p in+1 ≥λ p

[0196] Then, depending on the test result Y jn+1 To supplement the objects added at time n+1. Where λ p The supplementary threshold is set freely based on experience.

[0197] Step 3: Current frame detection confidence estimation

[0198] The average brightness value of the entire image is selected as an indicator of lighting conditions, while for color images, the weighted average of the RGB components is used as the basis for calculating the brightness value of a single pixel.

[0199] g(n)=0.299r(n)+0.578g(n)+0.114b(n)

[0200] The average brightness g(n) of the detection result corresponding to the image at time n is obtained from the above formula.

[0201] Based on the fitting function

[0202]

[0203] Substituting g(n) into the value p(n) yields the corresponding value. The magnitude of this parameter reflects the accuracy of the detector in detecting and recognizing targets in the image, serving as an important basis for measuring the reliability of image detection at that moment.

[0204] Step 4: Filtered estimation of object location

[0205] Assuming normal illumination conditions, the mAP (meta-pass absorption) of the image detector is P. For the target detection result at time n, let R(n) satisfy:

[0206]

[0207] Based on this, let the true position coordinates of a specific object in the image at time n be [x n ,y n ] T Considering the relative motion between the target and the observation equipment, let the velocity and acceleration of the target in the x-direction at time n be vn and vn, respectively. xn and a xn The velocity and acceleration in the y-direction are v yn and ayn Since the motion states in the x and y directions can be estimated separately during the actual filtering process, without loss of generality, let the motion state of the target in the x direction at time n be X. n =[x n ,v xn ,a xn ] T Combining the transmission law of the target's motion state, the transmission equations of the system state at time n and time n+1 are obtained:

[0208] X n+1 =Φ n,n+1 X n +v(n)

[0209] Where v(n) is the system noise at time n, Φ n,n+1 Let be the state transition matrix of the system from time n to time n+1. Assuming uniform acceleration, then:

[0210]

[0211] Where Δt is the time interval between the transition of the system state from time n to time n+1.

[0212] At time n+1, after the detector obtains the image target detection result, it obtains the observation information Z of the coordinates of the observed target in the image. n+1 This information is related to the target's true motion state X at time n+1. n The observation equation is satisfied between them:

[0213] Z n+1 =Θ n+1 X n +r(n+1)

[0214] Where r(n+1) is the observation noise of the system at time n+1, Θ n+1 Let Θ be the observation matrix at time n+1 of the system. For the case involved in this invention, Θ n+1 satisfy:

[0215]

[0216] Based on the above system equations, Kalman filtering theory is used to estimate the target parameters:

[0217]

[0218]

[0219]

[0220]

[0221]

[0222] P(n+1|n+1)=[I-K n+1 Θ n+1 ]P(n+1|n)

[0223] After the above operation, the Kalman filter estimation of the target motion state vector is formed, and the estimation information of the i-th target position at time n is obtained With

[0224] Step 5: Prediction of object size parameters

[0225] Let the confidence index of the detection result of the image of the previous 20 frames at time n be p n-20 , p n-19 , …, p n-1 , and take the average value as the confidence index of the size prediction result of the bounding box:

[0226]

[0227] Record the 20 sets of detection results obtained from time n-20 to time n-1 as the dependent variable Y=[b wn-20 , b wn-18 , …, b wn-1 ] T , and record the corresponding time from time n-20 to time n-1 as the independent variable X=[t n-20 , t n-18 , …, t n-1 ] T , and perform second-order Gaussian function fitting on X and Y to obtain the function of b w with time, denoted as:

[0228]

[0229] Then, t n is brought into the above function to obtain the prediction value of b w at time n-1 at time n:

[0230]

[0231] Taking the width size as an example, let the target bounding box width obtained by the detector at time n be b′ wn , and the target bounding box width obtained by the fitting function be Take the estimation value of the target bounding box width at this time as the weighted average of the above two:

[0232]

[0233] The calculation of height is similar to that of width.

[0234] Through the above process, the estimated value of the width of the i-th target bounding box at time n is obtained. and height estimates

[0235] Step 6: Output the results

[0236] Let the detection result of the i-th target at time n after processing satisfy:

[0237]

[0238] Among them, c in For the type of target, Let X be the coordinates of the target detection box in the X direction. Let Y be the coordinates of the target detection box in the Y direction. The height of the detection frame, The width of the detection frame.

[0239] As a preferred embodiment, such as Figure 2 As shown, the continuous image target detection method considering relative target motion provided in this embodiment of the invention specifically includes the following steps:

[0240] Step 1: Obtain raw test data.

[0241] The YOLO algorithm is used for target detection and recognition in continuously captured images to obtain the detection results at time k. Let Yi be the detection result of the i-th object. ik =[p ik ,c ik ,b xik ,b yik ,b hik ,b wik ] T , where p ik c is the confidence level for an object to appear within the bounding box. ik b is the type number of the detected object. xik b is the x-coordinate of the center of the bounding box yik b is the ordinate of the center of the bounding box hik b is the height of the bounding box wik This represents the width of the bounding box.

[0242] Step 2: Temporal matching of detection results

[0243] After the detector obtains the detection results based on the image at time k+1, considering that the number of objects in the detection results at both time k and time k+1 may be greater than 1, a temporal matching operation needs to be performed on the objects in the two detection results:

[0244] For the detection result Y of the i-th object obtained at time k... ik And the detection result Y of the j-th object detected at time k+1. jk+1 Calculate the IoU (Intersection over Union) value between the bounding boxes of the two entities. ij For a certain Y i k Remember all Y jk+1 IoU ij The largest detection result is Y imk+1 The corresponding IoU value is denoted as IoU. im .

[0245] For each Y ik The following judgments are made for all cases:

[0246] (1) Judgment of rejection of test results

[0247] If IoU im <λ d If so, it is considered that no match for Y can be found in the detection results at time k+1. ik The matching test results will Y ik Remove.

[0248] Where, λ d The rejection threshold can be freely set based on experience.

[0249] (2) Matching judgment of detection results

[0250] If Y ik element c ik and Y imk+1 element c imk+1 satisfy:

[0251] c ik =c imk+1

[0252] Then it is believed that Y ik and Y imk+1 The detection results of the same object generated at different times will be matched together.

[0253] (3) Judgment of missed detection of the detection object

[0254] If Y ik element c ik and Y imk+1 element c imk+1 satisfy:

[0255] c ik ≠c imk+1

[0256] If object i is not detected at k+1 time due to occlusion or other reasons, object i is retained.

[0257] (4)Supplemental judgment of detection result

[0258] After the matching judgment is completed, if there is a detection result Y jk+1 that cannot find a matching Y ik , considering that there may be false detection in a single image, if p ik+1 satisfies the threshold condition:

[0259] p ik+1 ≥λ p

[0260] the detection result Y jk+1 is considered as a new object at k+1 time for supplement. Wherein λ p is a supplemental threshold value set freely according to experience.

[0261] ③Filtering estimation of object position

[0262] Considering that the core content of the present application is to further accurately estimate the position and size information of the target on the basis of target detection and identification of the detector for continuous images, the emphasis is not on the improvement of the filtering estimation algorithm itself, therefore the classical Kalman filter is used to realize the filtering estimation of the object position.

[0263] According to the existing labeling information and the detection result of the training sample by the detector, the variance statistical information r x and r y of the parameters b bx and b by in the detection result Y=[p,c,b x ,b y ,b h ,b w ] T of the detector for any object can be obtained through data comparison and statistical analysis.

[0264] On this basis, the real position coordinates of a certain object in the image at n time are recorded as [x n ,y n ] T , considering that there is relative motion between the target and the observation device, the velocity and acceleration of the target in the x direction at n time are recorded as v xn and a xn , and the velocity and acceleration of the target in the y direction at n time are recorded as v yn and a yn. Since the motion state in x and y directions can be estimated respectively in the actual filtering process, the motion state of the target in x direction at time n is denoted as X n = [x n , v xn , a xn ] T , and the transfer equation of the system state at time n and n+1 is obtained by combining the transfer law of the target motion state:

[0265] X n+1 = Φ n,n+1 X n + v(n)

[0266] wherein v(n) is the system noise at time n, Φ n,n+1 is the state transition matrix of the system state from time n to n+1, and for the uniform acceleration motion assumption, it has:

[0267]

[0268] wherein Δt is the time interval of the system state from time n to n+1.

[0269] At time n+1, the detector obtains the image target detection result, and the observation information Z n+1 of the observed target coordinate in the image is obtained, which satisfies the observation equation between the true motion state X n of the target at time n+1:

[0270] Z n+1 = Θ n+1 X n + r(n+1)

[0271] wherein r(n+1) is the observation noise of the system at time n+1, Θ n+1 is the observation matrix of the system at time n+1, and for the case involved in the present application, Θ n+1 satisfies:

[0272]

[0273] On the basis of constructing the above system equation, the Kalman filtering theory is used to filter and estimate the target parameters:

[0274]

[0275]

[0276]

[0277]

[0278]

[0279] P(n+1|n+1) = [I - K n+1 Θ n+1 ]P(n+1|n)

[0280] After the above operation, the Kalman filtering estimation of the target motion state vector is formed, and the estimation accuracy of the actual coordinates of the target is improved.

[0281] ④Prediction of object size parameters

[0282] In addition to the position of the target, the size parameters b h and b w of the target detection bounding box are also important parameters in the image target detection work. In order to further ensure the accuracy of target detection, when estimating the size parameters of the object detection bounding box, it is necessary to first study the change law of the observation information of the observation device on the target with the movement of the target, and then combine the corresponding law to form the prediction of the size parameters of the target bounding box in the subsequent image according to the continuous image detection results that have been generated, as an important basis for subsequent target position and bounding size estimation.

[0283] Assuming that the target moves at a constant speed relative to the observation center, through numerical simulation, the change of the observation size of the target with time is shown in Figure 3 .

[0284] Considering that the above curve changes obviously nonlinearly, a data fitting method is adopted to represent the change of the observation size of the target with time in uniform motion. Through testing, it is found that the second-order Gaussian function has good fitting effect, and the schematic diagram is shown in Figure 4 .

[0285] According to the fitting result, the confidence of the fitting function reaches 99.97%, so the second-order Gaussian function is selected in the present application to represent the change law of the observation size of the target with time, and its general expression is:

[0286]

[0287] Since there are 6 unknown parameters in the second order Gaussian function, at least 6 groups of observation data are needed to determine the function of the observation size changing with time in actual operation. Meanwhile, more data does not necessarily mean better results in the fitting calculation, because the fitting function is established based on the assumption that the target moves at a constant speed. If the time span is too long and the target speed changes, the assumption will not hold, and the results of the fitting will no longer be accurate. Considering that the frame rate of most video observation devices is generally in the range of 15-60 FPS, the method mentioned in the present application uses the detection results in the 20 frames closest to the current frame as the basis for predicting the size of the target bounding box in the future frame. The specific operation is as follows:

[0288] Assuming that the detector has completed the detection of the target in the image at time n, the next 20 groups of detection results including the detection result at time n will be used to predict the size parameters b hn+1 and b wn+1 of the target bounding box at time n+1. Taking the prediction of b wn+1 as an example:

[0289] Record the 20 groups of detection results obtained from time n-19 to time n as the dependent variable Y=[b wn-19 , b wn-18 , …, b wn ] T , and record the corresponding times from time n-19 to time n as the independent variable X=[t n-19 , t n-18 , …, t n ] T , perform second order Gaussian function fitting on X and Y to obtain the function of b w changing with time, denoted as:

[0290]

[0291] Then bring t n+1 into the above function to obtain the predicted value of b w at time n+1 at time n:

[0292]

[0293] ⑤Current frame detection confidence estimation

[0294] Generally, the detection effect of visible light image target detector is affected by many factors such as observation equipment performance, detector learning model structure, training sample, training times, environmental light condition and the like, so it is difficult to give an accurate estimation of the reliability of target detection. However, for the same detector which has completed training, the main influencing factor comes from the imaging effect of the image itself, and the continuous observation image comes from the same observation equipment, and the quality of the image can be considered constant, so the main factor affecting the reliability of the detection result becomes the environmental factor.

[0295] The present application mainly considers the influence of the change of image imaging effect under different light conditions on the detection effect of the final detector, selects the average brightness value of the whole image as an index for measuring the light condition, and for a color image, adopts the weighted average of the three components of RGB as the basis for calculating the brightness value of a single pixel:

[0296] g = 0.299r + 0.578g + 0.114b

[0297] In order to determine the influence between the average brightness value of the image and the detection effect of the detector, the mAP value of the detection result under different environmental brightness conditions is recorded through the design experiment, and the average brightness value g under the corresponding condition is recorded, and the obtained result is as shown in Figure 5

[0298] The above data is fitted by combining the change law of the parameter mAP with g, and after data fitting, g and mAP also approximately obey the second-order Gaussian distribution:

[0299]

[0300] The fitting result of the above function is as shown in Figure 6

[0301] According to the fitting result, the confidence of the fitting function reaches 96.30%.

[0302] In this way, for the actually obtained continuous image to be detected, assuming that the average brightness value of the image at time n is g n , the fitting function will obtain the corresponding value p n by being brought into the fitting function, and the size of the parameter reflects the accuracy of the detector in detecting and identifying the target in the image, so it can be used as an important basis for measuring the reliability of the image detection at this time.

[0303] ⑥Image target fusion detection

[0304] ​​After the filtering estimation of the target position, the estimation of the size of the target bounding box and the estimation of the credibility of the current frame detection result are completed according to the existing detection results in the continuous shooting images, the position and size parameters of the target in the current image can be further estimated by comprehensively considering the above various information:

[0305] (1) Estimation of the target position

[0306] According to the Kalman filtering principle, the credibility of observation information at different times is mainly based on the observation noise matrix R(n). Considering that the observation noise matrix of the same observation device should be constant in theory under normal circumstances, the average brightness of the image will affect the detection result of the image detector due to the influence of the light intensity. Assuming that the noise matrix of the observation device is R through experimental statistics, and the average brightness parameter g(n) of the image at time n is recorded, n The credibility index p of the detection result is estimated n , and the noise matrix R(n) of the detection result at this time is obtained by combining p k and R:

[0307] Assuming that the mAP index of the image detector under normal light conditions is P, and R(n) satisfies:

[0308]

[0309] In this way, the target position filtering estimation considering the credibility of target detection can be realized by adjusting the noise matrix parameter at time n.

[0310] (2) Estimation of the target size

[0311] The prediction result of the target detection box size at time n is derived from the detection results of the past 20 images, and the prediction result will be affected by the detection credibility of the previous 20 images.

[0312] Let the credibility indexes of the detection results of the previous 20 images at time n be p n-20 , p n-19 , …, p n-1 , and the average value is the credibility index of the prediction result of the bounding box size:

[0313]

[0314] Taking the width size as an example, let the target bounding box width obtained by the detector at time n be b′ wn , and the target bounding box width obtained by the fitting function be The estimated value of the target bounding box width at this time is the weighted average of the above two values:

[0315]

[0316] The calculation of the height dimension is similar to the width, and thus is not described herein.

[0317] The continuous image target detection system considering the relative motion of the target provided by the embodiments of the present application comprises:

[0318] An original detection data acquisition module is configured to perform target detection and identification in the continuous image by using a YOLO algorithm to obtain a detection result at time k;

[0319] A detection result time sequence matching module is configured to perform a time sequence matching operation on the objects in the detection results at time k and time k+1 after the detector obtains the detection result according to the image at time k+1;

[0320] An object position filtering estimation module is configured to perform filtering estimation of the object position by using a Kalman filter;

[0321] An object size parameter prediction module is configured to predict the size parameters b hn+1 and b wn+1 of the target bounding box at time n+1 by using 20 groups of detection results including the detection result at time n;

[0322] A current frame detection confidence estimation module is configured to select the average brightness value of the entire image as an index for measuring the light condition, and calculate the brightness value of a single pixel by using the weighted average of the RGB three components;

[0323] An image target fusion detection module is configured to perform target position filtering estimation considering the target detection confidence by adjusting the noise matrix parameters at time n, and perform target size estimation by using a fitting function.

[0324] II. Application Embodiments. In order to prove the creativity and technical value of the technical solutions of the present application, this part is an application embodiment of the technical solutions of the claims on a specific product or related technology.

[0325] The following takes an actual video image as an object to give an embodiment of the image target detection involved in the present application:

[0326] A camera is used to capture the target in the test scene, and the imaging device parameters are shown in Table 1.

[0327] Table 1: Imaging device parameter table

[0328]

[0329] The test scene is built to simulate the typical scene of continuous shooting in a natural scene. In order to adapt to the input picture size requirement of the detector, the video images shot by the imaging device are appropriately cropped, and the resolution is 854*480 and the target scene part region is completely contained as the original input video of the input image target detector. The key frame continuous images in the original image are as shown in Figure 7

[0330] The obtained video input image detector (YOLOv7) is detected by using a standard model (YOLOv7). After detection, the image target detection result corresponding to the key frame is as shown in Figure 8

[0331] On this basis, the algorithm designed in the application is used to process the continuously shot video images, and then the position of the target detection frame and the result of target classification are output in the original image, wherein the detection result corresponding to the key frame is as shown in Figure 9

[0332] Comparing Figures 7-9 The three groups of images and target detection results can know that:

[0333] 1. Under the test conditions, the YOLOv7 algorithm is used to detect the video images, which can accurately detect the targets in the images, but in the case of target mutual occlusion, unstable light conditions and the like, the target detection result in part of the frame images will be affected, thereby reducing the accuracy of the detection result;

[0334] 2. After the detection result is processed by using the algorithm designed in the application, the problems of missed detection, classification error and false detection in the detection result of the original image target detector are effectively inhibited, and the final detection result is closer to the true situation.

[0335] According to the above embodiment under the test conditions, it can be known that the algorithm designed in the application can effectively process the continuous shooting image target detection result, and further improve the accuracy of target detection and classification.

[0336] III. Evidence of the effect of the embodiment. The embodiment of the application has achieved some positive effects in the research and development or use process, and indeed has great advantages compared with the prior art. The following content is described in combination with the data, charts and the like of the test process.

[0337] In order to further verify the effectiveness of the algorithm, the test scene is designed to verify the improvement significance of the algorithm designed in the application for the continuous image target detection effect under different conditions:

[0338] ​​​The imaging device described in Table 1 is used to take pictures of the target scene, and during the shooting process, the brightness of the ambient light, the position relationship between the targets and other ways are adjusted to simulate the influence of factors such as changes in ambient light in actual natural scenes, obstacles blocking and other factors on the target detection process in video images. Through shooting, the original video containing the target scene under different conditions is obtained.

[0339] The image target in the original video is labeled to obtain the manually labeled label as a measure of the image target detector.

[0340] The YOLOv7 algorithm is used to detect the image target in the original video to obtain the detection result of the image detector, and the result is recorded as a comparison group, and the algorithm designed by the application is used to continue processing based on the result to obtain a new result.

[0341] The detection results before and after processing are compared with the manually labeled label to measure the difference between the two groups of results and the manually labeled label.

[0342] The above operation is repeated several times to obtain the precision, recall, mean average precision (mAP) and average time (AT) before and after processing as shown in Table 2.

[0343] Table 2 Comparison of detection results before and after processing

[0344]

[0345] From the experimental data and experimental results, it can be seen that the algorithm designed by the application can effectively reduce the influence of factors such as changes in ambient light and obstacles blocking on the target detection accuracy on the basis of the original image target detector, and further improve the target detection accuracy in continuous shooting images.

[0346] It should be noted that embodiments of the present application can be realized by hardware, software, or a combination of software and hardware. The hardware portion can be realized by a special logic; the software portion can be stored in a memory and executed by a proper instruction execution system, such as a microprocessor or a specially designed hardware. A person of ordinary skill in the art can understand that the above-mentioned apparatus and method can be realized by computer executable instructions and / or included in processor control codes, such as a carrier medium, such as a magnetic disk, CD or DVD-ROM, a programmable memory, such as a read-only memory (firmware), or a data carrier, such as an optical or electronic signal carrier. The apparatus of the present application and its modules can be realized by a hardware circuit, such as a very large scale integrated circuit or a gate array, a semiconductor, such as a logic chip, a transistor, or a programmable hardware device, such as a field programmable gate array, a programmable logic device, or the like, by software executed by various types of processors, or by a combination of the above-mentioned hardware circuit and software, such as firmware.

[0347] The above description is merely a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any modification, equivalent replacement, and improvement within the technical range disclosed by the present application, and within the spirit and principle of the present application, should be covered within the protection scope of the present application.

Claims

1. A continuous image target detection method considering relative target motion, characterized in that, Step 1: Obtain the raw detection data and perform time-series matching of the detection results; Step two: Perform filtered estimation of object position and prediction of object size parameters; Step 3: Estimate the confidence level of the current frame detection and perform image target fusion detection; The temporal matching of the detection results in step one includes: after the detector obtains the detection results based on the image at time k+1, a temporal matching operation is performed on the objects in the two detection results: For the detection result Y of the i-th object obtained at time k... ik And the detection result Y of the j-th object detected at time k+1. jk+1 Calculate the IoU value between the bounding boxes of the two. ij For a certain Y ik Remember all Y jk+1 IoU ij The largest detection result is Y imk+1 The corresponding IoU value is denoted as IoU. im ; For each Y ik All of them are judged as follows: (1) Judgment of rejection of test results If IoU im <λ d If so, it is considered that no match for Y can be found in the detection results at time k+1. ik The matching test results will Y ik Eliminate; where λ d The rejection threshold is set freely based on experience; (2) Matching judgment of detection results If Y ik element c ik and Y imk+1 element c imk+1 satisfy: c ik =c imk+1 ; Then it is believed that Y ik and Y imk+1 Match the detection results of the same object generated at different times; (3) Judgment of missed detection of the detection object If Y ik element c ik and Y imk+1 element c imk+1 satisfy: c ik ≠c imk+1 ; Then it is considered that object i was missed at time k+1 due to occlusion or other reasons, and object i is retained; (4) Supplementary judgment of test results After completing the matching judgment, if there is a detection result Y jk+1 No matching Y found ik If so, it is considered as an object newly added at time k+1 and is added accordingly; The object position estimation in step two is implemented using the classic Kalman filter; Step 3, the current frame detection confidence estimation, includes: The average brightness value of the entire image is selected as an indicator of lighting conditions; while for color images, the weighted average of the three RGB components is used as the basis for calculating the brightness value of a single pixel. ； By designing experiments to record the mAP values ​​of detection results under different ambient brightness conditions, and recording the average brightness value g under the corresponding conditions, the influence between the average brightness value of the image and the detection effect of the detector is determined. By combining the variation of parameter mAP with g, a function fit was performed on the data. After data fitting, g and mAP also approximately follow a second-order Gaussian distribution. ； For the actual obtained continuous image to be detected, when the average brightness value of the image at time n is g n Substituting this into the fitting function yields the corresponding value p. n ; The image target fusion detection in step three includes: estimating the target position by filtering based on the existing detection results in the continuously captured images, estimating the target bounding box size and the confidence level of the detection results in the current frame, and estimating the position and size parameters of the target in the current image by integrating various information; (1) Estimation of target location When the noise matrix of the observation equipment is determined to be R through experimental statistics, let g be the average brightness parameter of the image at time n. n The reliability index p of the estimated test results is obtained. n Combined with p n The noise matrix R(n) of the detection results at time n is obtained from R; under normal illumination, the mAP index of the image detector is P, and R(n) satisfies: ； The target position filtering estimation considering the target detection reliability is achieved by adjusting the noise matrix parameters at time n. (2) Estimation of target size Let p be the confidence index of the detection results of the 20 frames of images before time n. n-20 p n-19 ... p n-1 The average value is used as a metric to measure the reliability of the bounding box size prediction results. ； Let the width of the target bounding box obtained by the detector at time n be . The width of the target bounding box obtained by fitting the function is The estimated value of the target bounding box width at time n is taken as a weighted average of the two estimates: 。 2. The continuous image target detection method considering relative target motion as described in claim 1, characterized in that, Step one, obtaining the raw detection data, includes: performing target detection and recognition using the YOLO algorithm in continuously captured images, obtaining the detection results at time k, and denoting the detection result of the i-th object as Y. ik =[p ik ,c ik ,b xik ,b yik ,b hik ,b wik ] T ; where p ik c represents the confidence level that an object appears within the bounding box. ik b is the type number of the detected object. xik b is the x-coordinate of the bounding box center. yik b is the ordinate of the center of the bounding box hik b is the height of the bounding box wik This represents the width of the bounding box.

3. A continuous image target detection system considering relative target motion, applying the continuous image target detection method considering relative target motion as described in any one of claims 1 to 2, characterized in that, Continuous image target detection systems that consider the relative motion of targets include: The raw detection data acquisition module is used to perform target detection and recognition in continuously captured images using the YOLO algorithm to obtain the detection results at time k. The detection result temporal matching module is used to perform temporal matching operations on the objects in the two detection results at time k and time k+1 after the detector obtains the detection result based on the image at time k+1. The object position filtering estimation module is used to achieve filtering estimation of the object position using Kalman filtering; The object size parameter prediction module is used to predict the size parameter b of the target bounding box at time n+1 using 20 sets of detection results, including the detection results at time n. hn+1 and b wn+1 Make predictions; The current frame detection confidence estimation module is used to select the average brightness value of the entire image as an indicator of lighting conditions, and calculates the brightness value of a single pixel by weighted average of the three RGB components. The image target fusion detection module is used to achieve target position filtering estimation considering the target detection reliability by adjusting the noise matrix parameters at time n, and to estimate the target size by fitting a function.

4. A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the continuous image target detection method considering relative target motion as described in any one of claims 1 to 2.

5. An information data processing terminal, characterized in that, The information data processing terminal is used to implement the continuous image target detection system that takes into account the relative motion of the target as described in claim 3.

Images