Target detection method and device

Abstract

The present application provides a target detection method and device, which relates to the field of image processing technology. The method detects targets in an image by combining a deep learning algorithm and a Kalman filter, which can well consider the continuity of the target's motion in the previous and next frame images, and can effectively improve the false detection rate and missed detection rate. And by timely updating the parameters of the Kalman filter, it is easy to update the state change of the moving target in time, so that the detection accuracy of the target in the subsequent frame image is higher.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a target detection method and device. Background Art

[0002] With the development of autonomous driving technology, accurate and timely detection of obstacles around the road, such as vehicles and pedestrians, has become a basic requirement for the implementation of autonomous driving assistance systems. In order to perceive obstacles around the vehicle, the target detection algorithm based on image processing is generally used to detect obstacles around the vehicle. Although this method has a high detection efficiency, it still has a certain amount of false detection of the target and the detection accuracy is not high. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a target detection method and device to improve the problem of low detection accuracy in the prior art.

[0004] In a first aspect, an embodiment of the present application provides a target detection method, the method comprising:

[0005] Detecting the target in the current frame image by using a deep learning algorithm to obtain at least one first detection frame;

[0006] Predicting the target in the current frame image by using a Kalman filter to obtain at least one second detection frame, wherein the Kalman filter is a Kalman filter corresponding to each target obtained by detecting the target in the previous frame image of the current frame image;

[0007] Match each first detection frame with each second detection frame, and obtain a first matching detection frame in at least one first detection frame that matches the second detection frame;

[0008] The parameters of the Kalman filter corresponding to the target corresponding to the first matching detection frame are updated according to the first matching detection frame, so as to use the updated Kalman filter to predict the target in the next frame image.

[0009] In the above implementation process, by combining the deep learning algorithm and the Kalman filter to detect the target in the image, the continuity of the target's movement in the previous and next frame images can be well considered, which can effectively improve the false detection rate and missed detection rate. And by timely updating the parameters of the Kalman filter, the state change of the moving target can be updated in time, so that the detection accuracy of the target in the subsequent frame image is higher.

[0010] Optionally, updating a parameter of a Kalman filter corresponding to the target corresponding to the first matching detection frame according to the first matching detection frame includes:

[0011] Acquire, according to the first matching detection frame, a distance between a target corresponding to the first matching detection frame and a shooting device that shoots an image;

[0012] The parameters of the Kalman filter corresponding to the target corresponding to the first matching detection frame are updated according to the distance.

[0013] In the above implementation process, the parameters of the Kalman filter are updated according to the distance, so that the relevant parameters of the Kalman filter can be dynamically adjusted according to the distance of the target, so that the Kalman filter can adapt to the impact of changes in the detection frame of targets at different distances on target detection, so as to improve the target detection accuracy.

[0014] Optionally, updating a parameter of a Kalman filter corresponding to the target corresponding to the first matching detection frame according to the distance includes:

[0015] Acquire, according to the distance, an update coefficient for updating a state transfer matrix of a Kalman filter corresponding to the target corresponding to the first matching detection frame; wherein the update coefficient is negatively correlated with the distance;

[0016] The state transfer matrix is ​​updated using the update coefficients.

[0017] In the above implementation process, the update coefficient is negatively correlated with the distance, so that the Kalman filter can adapt to the detection of long-distance or short-distance targets.

[0018] Optionally, updating a parameter of a Kalman filter corresponding to the target corresponding to the first matching detection frame according to the distance includes:

[0019] Acquire, according to the distance, an update parameter for updating the measurement noise of the Kalman filter corresponding to the target corresponding to the first matching detection frame; wherein the update parameter is positively correlated with the distance;

[0020] The measurement noise is updated using the update parameter.

[0021] In the above implementation process, by updating the measurement noise, the measurement noise can be dynamically adjusted when detecting targets at different distances, so as to improve the accuracy of detecting targets at different distances.

[0022] Optionally, the calculation formula of the update parameter is:

[0023] r=d 2 / p;

[0024] Wherein, r represents the update parameter, d represents the distance, and p represents the preset coefficient. The update parameter increases with the increase of the distance, so that when performing target detection, it can adapt to the detection of targets with different moving speeds at different distances, and can effectively improve the problem of low detection accuracy caused by the jitter of the detection frame.

[0025] Optionally, updating a parameter of a Kalman filter corresponding to the target corresponding to the first matching detection frame according to the first matching detection frame includes:

[0026] Counting the detection value of the first matching detection frame within a preset time period;

[0027] Calculating the variance of the detection value;

[0028] Acquire an update parameter for updating the measurement noise of the Kalman filter corresponding to the target corresponding to the first matching detection frame according to the variance, wherein the variance is positively correlated with the update parameter;

[0029] The measurement noise is updated using the update parameter.

[0030] In the above implementation process, by obtaining update parameters according to the variance, the changes of the detection box over a period of time can be comprehensively considered to more accurately adjust the measurement noise.

[0031] Optionally, updating a parameter of a Kalman filter corresponding to the target corresponding to the first matching detection frame according to the first matching detection frame includes:

[0032] Obtaining the size of the first matching detection frame;

[0033] The parameters of the Kalman filter corresponding to the target corresponding to the first matching detection frame are updated according to the size.

[0034] In the above implementation process, the parameters of the Kalman filter are updated according to the size of the detection frame, so that the influence of the size of the target in the image on the target detection accuracy can be considered to improve the detection accuracy.

[0035] Optionally, updating a parameter of a Kalman filter corresponding to the target corresponding to the first matching detection frame according to the first matching detection frame includes:

[0036] Obtaining a pixel ratio of pixels corresponding to the first matching detection frame in the current frame image;

[0037] The parameters of the Kalman filter corresponding to the target corresponding to the first matching detection frame are updated according to the pixel proportion.

[0038] In the above implementation process, the parameters of the Kalman filter are updated according to the pixel ratio of the detection frame, so that the influence of the size of the target in the image on the target detection accuracy can be considered to improve the detection accuracy.

[0039] Optionally, after matching each first detection frame with each second detection frame, the method further includes:

[0040] Acquire a first unmatched detection frame in the at least one first detection frame that does not match the second detection frame;

[0041] A Kalman filter corresponding to the target corresponding to the first unmatched detection box is newly created according to the first unmatched detection box.

[0042] In the above implementation process, a new Kalman filter is created for the unmatched detection frame, which can increase the detection of newly appeared targets.

[0043] Optionally, after obtaining a first unmatched detection frame in the at least one first detection frame that does not match the second detection frame, the method further includes:

[0044] The target corresponding to the first unmatched detection frame is marked with a new target identifier.

[0045] In the above implementation process, a new target identifier is marked for the new target, and when the target is subsequently tracked, other information can be further combined to determine whether the target is a falsely detected target.

[0046] Optionally, after matching each first detection frame with each second detection frame, the method further includes:

[0047] Acquire a second unmatched detection frame in the at least one second detection frame that does not match the first detection frame;

[0048] Counting the number of matching failures of the second unmatched detection frame;

[0049] When the number of matching failures of the second unmatched detection frame is greater than a preset value, the second unmatched detection frame is deleted.

[0050] In the above implementation process, by counting the number of matching failures of the unmatched second unmatched detection frame, it is determined whether the target corresponding to the second unmatched detection frame disappears in the current frame image according to the number of matching failures, thereby avoiding the problem of false detection of the target.

[0051] Optionally, after matching each first detection frame with each second detection frame, the method further includes:

[0052] Acquire a second unmatched detection frame in the at least one second detection frame that does not match the first detection frame;

[0053] Counting the number of matching failures of the second unmatched detection frame;

[0054] When the number of matching failures of the second unmatched detection frame is less than a preset value and the identifier of the target corresponding to the second unmatched detection frame is not a new target identifier, the Kalman prediction value of the second unmatched detection frame is output.

[0055] In the above implementation process, by counting the number of matching failures of the second unmatched detection frame, when the number of matching failures is less than a preset value, it means that the target corresponding to the second unmatched detection frame is a real target, and the predicted value of the detection frame can be output to provide data reference for the vehicle's automatic driving, thereby avoiding outputting detection values ​​for misdetected targets and causing interference to the vehicle's automatic driving. This method can make up for the missed detection of the deep learning algorithm.

[0056] Optionally, after acquiring a first matching detection frame in the at least one first detection frame that matches the second detection frame, the method further includes:

[0057] Counting the number of successful matches of the first matching detection frame;

[0058] When the number of successful matches is greater than a preset value, a target identifier is marked on the target corresponding to the first matching detection frame, and a Kalman update value of the first matching detection frame is output.

[0059] In the above implementation process, for the first matching detection frame that is successfully matched, when the number of successful matches is greater than a preset value, it indicates that the target corresponding to the first matching detection frame is a real target, thereby avoiding outputting detection values ​​for misdetected targets and causing interference to the vehicle's automatic driving.

[0060] In a second aspect, an embodiment of the present application provides a target detection device, the device comprising:

[0061] A first detection module, configured to detect a target in a current frame image by using a deep learning algorithm to obtain at least one first detection frame;

[0062] A second detection module is used to predict the target in the current frame image through a Kalman filter to obtain at least one second detection frame, wherein the Kalman filter is a Kalman filter corresponding to each target obtained by detecting the target in the previous frame image of the current frame image;

[0063] A matching module, configured to match each first detection frame with each second detection frame, and obtain a first matching detection frame in the at least one first detection frame that matches the second detection frame;

[0064] An updating module is used to update the parameters of the Kalman filter corresponding to the target corresponding to the first matching detection frame according to the first matching detection frame, so as to use the updated Kalman filter to predict the target in the next frame image.

[0065] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method provided in the first aspect are performed.

[0066] In a fourth aspect, an embodiment of the present application provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the method provided in the first aspect are performed.

[0067] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the embodiments of the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0069] Figure 1 A schematic diagram of the structure of an electronic device for executing a target detection method provided in an embodiment of the present application;

[0070] Figure 2 A flow chart of a target detection method provided in an embodiment of the present application;

[0071] Figure 3 A structural block diagram of a target detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0072] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present application.

[0073] An embodiment of the present application provides a target detection method, which obtains a matched first detection frame by matching a first detection frame obtained by performing target detection using a deep learning algorithm with a second detection frame obtained by performing target detection using a Kalman filter, and then uses the first matching detection frame to update the parameters of the Kalman filter. In this way, the parameters of the Kalman filter can be updated in a timely manner after each frame of the image is detected, so that when the Kalman filter performs target detection on the next frame image, it can well consider the continuity of the target's motion in the previous and next frame images, and the detection is more accurate. In combination with the results of the deep learning algorithm detection, the target in the image can be detected more accurately, thereby avoiding the problem of low detection accuracy when only using the deep learning algorithm for detection.

[0074] Please refer to Figure 1 , Figure 1 A structural diagram of an electronic device for executing a target detection method provided in an embodiment of the present application, the electronic device may include: at least one processor 110, such as a CPU, at least one communication interface 120, at least one memory 130 and at least one communication bus 140. Among them, the communication bus 140 is used to realize direct connection and communication between these components. Among them, the communication interface 120 of the device in the embodiment of the present application is used to communicate signaling or data with other node devices. The memory 130 can be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 130 can optionally also be at least one storage device located away from the aforementioned processor. Computer-readable instructions are stored in the memory 130. When the computer-readable instructions are executed by the processor 110, the electronic device executes the following Figure 2 In the method process shown, for example, the memory 130 can be used to store images and detected detection frames, and the processor 110 can be used to run a deep learning algorithm and a Kalman filter, match the detection frames, and update the Kalman filter based on the matching results.

[0075] Understandably, Figure 1 The structure shown is for illustration only, and the electronic device may also include Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0076] Please refer to Figure 2 , Figure 2 A flow chart of a target detection method provided in an embodiment of the present application, the method comprising the following steps:

[0077] Step S110: Detect the target in the current frame image through a deep learning algorithm to obtain at least one first detection frame.

[0078] Among them, the deep learning algorithm can refer to machine learning based on artificial neural networks, mainly including convolutional neural networks, recurrent neural networks, recursive neural networks, etc. In the embodiment of the present application, in order to improve the effect of target detection, the deep learning algorithm can adopt the YOLO network, which has a good effect on target detection.

[0079] In the field of autonomous driving, the target in the image can refer to any obstacle, and the categories of targets include vehicles, pedestrians, bicycles, etc. In order to avoid collision between the vehicle and obstacles, it is necessary to detect obstacles around the vehicle. The above-mentioned electronic device can be understood as a device for target detection on an autonomous driving vehicle. The camera on the autonomous driving vehicle captures images and can send the images to the processor in the electronic device, which analyzes the images.

[0080] It is understandable that the embodiments of the present application are not limited to the field of autonomous driving, and the target detection method provided by the embodiments of the present application can also be used in other fields, such as the field of security monitoring of drones.

[0081] The processor can run a deep learning algorithm to detect the target in the image through the deep learning algorithm to obtain at least one first detection frame for the target. The YOLO network can be trained in advance, a large number of images can be collected during the training process, and the detection frames of each target can be marked. The YOLO network is trained using the training images, and then the trained YOLO network can be used for target detection.

[0082] After the YOLO network detects the target, it outputs the first detection frame corresponding to each target. The information of the first detection frame may include but is not limited to: the category of the detection frame, such as vehicle, pedestrian, bicycle, etc., and may also include the coordinates of the detection frame in the image, the width and height of the detection frame, the confidence of the detection frame, etc.

[0083] Step S120: predicting the target in the current frame image by using a Kalman filter to obtain at least one second detection frame.

[0084] Due to the limited computing power of electronic devices, when deep learning algorithms are deployed in electronic devices, the model needs to be quantized or compressed. However, quantization or compression will damage the detection accuracy of the model, resulting in low detection accuracy when the electronic device runs the deep learning algorithm to detect the target. Therefore, the embodiment of the present application also combines the Kalman filter to predict the target in the current frame image.

[0085] The Kalman filter is a Kalman filter corresponding to each target obtained by detecting the target in the previous frame image of the current frame image. For example, when detecting the first frame image, the target can be detected by a deep learning algorithm. At this time, the first detection frame corresponding to each target in the first frame image is obtained. After the first detection frame is obtained, each first detection frame corresponds to a target. At this time, a Kalman filter can be initialized for each first detection frame, that is, the parameters in the Kalman filter are initialized, such as the state transfer matrix, measurement noise, etc. These parameters are determined according to the first detection frame obtained by the deep learning algorithm detection. For example, after the first frame image is detected by the deep learning algorithm, three first detection frames are obtained. At this time, three Kalman filters are initialized, and then when the second frame image is detected by the Kalman filter, the three Kalman filters can be used to detect the target in the second frame image. Since the Kalman filter algorithm predicts the state of the target at the current moment based on the state of the target at the previous moment, when detecting the target in the second frame image, three second detection frames can be obtained by detecting the three initialized Kalman filters.

[0086] It should be noted that when the current frame image is the first frame image (i.e., the first frame image), its Kalman filter is an initialized filter. At this time, since it is impossible to detect the target in the first frame image according to the Kalman filter, it is impossible to match the first detection frame and the second detection frame. Therefore, the current frame image in the embodiment of the present application is an image other than the first frame image.

[0087] Step S130: Match each first detection frame with each second detection frame to obtain a first matching detection frame that matches the second detection frame in the at least one first detection frame.

[0088] In order to avoid missed detection or false detection, the first detection frame can be matched with the second detection frame. For example, at least one first detection frame includes a1, b1, c1, and at least one second detection frame includes a2, b2, c2, d3. At this time, the second detection frame has one more than the first detection frame, indicating that the target detected by the deep learning algorithm may have been missed. It may also be because a target in the previous frame image disappeared in the current frame image, that is, disappeared within the current shooting field of view. Therefore, by matching the detection frames, missed detection or target disappearance can be judged.

[0089] When matching, the first detection frame and the second detection frame are matched in pairs. There are three types of matching situations, as follows:

[0090] (1) A first detection frame in the first detection frame that does not match the second detection frame.

[0091] (2) The first detection boxes in the first detection box that match the second detection box.

[0092] (3) The second detection boxes in the second detection box that do not match the first detection box.

[0093] For example, if a1 matches a2, b1 matches b2, and the rest of the detection boxes do not match, then the first matching detection boxes in the first detection box that match the second detection box include a1 and b1.

[0094] Among them, the method of matching two detection boxes can be: calculate the IOU of the two detection boxes, and when the IOU is greater than the preset value, it is considered that the two detection boxes match. Or it can also calculate the IOU of the width and height of the two detection boxes. If the IOU is greater than the set value, and the absolute value of the difference in the abscissas of the two detection boxes is less than the preset set value, such as |x1 - x2| < N, then it is considered that the two detection boxes match.

[0095] In addition, when two targets are relatively close, the detection boxes of different targets may be matched when performing detection box matching. Therefore, to avoid this situation, when performing detection box matching, two detection boxes of the same category can also be matched, that is, the targets selected by the two matched detection boxes are the same, so that the same target can be tracked and detected.

[0096] Step S140: Update the parameters of the Kalman filter corresponding to the target corresponding to the first matching detection box, so as to predict the target in the next frame of image by using the updated Kalman filter.

[0097] The first matching detection box refers to the relatively accurate detection box output by the deep learning algorithm for the target. Therefore, the parameters of the Kalman filter corresponding to it can be updated by using the first matching detection box. For example, the Kalman filter corresponding to the target in the first matching detection box a1 is X1, and the Kalman filter corresponding to the target in the first matching detection box b1 is X2. Then, the relevant parameters of the Kalman filters X1 and X2 are updated. After the update, the updated Kalman filter can be used to predict the target in the next frame of image. For example, when calculating the predicted position of the target in the next frame of image, the parameters of the updated Kalman filter are used for calculation. In this way, the Kalman filter can be continuously updated based on the matching detection boxes, so that the subsequent target detection can be more accurate.

[0098] In the above implementation process, by combining the deep learning algorithm and the Kalman filter to detect the target in the image, the continuity of the target's movement in the previous and next frame images can be well considered, which can effectively improve the false detection rate and missed detection rate. And by timely updating the parameters of the Kalman filter, the state change of the moving target can be updated in time, so that the detection accuracy of the target in the subsequent frame image is higher.

[0099] In some implementations, there are several ways to update the parameters of the Kalman filter, and several of them are introduced below.

[0100] Method 1, obtaining the distance between the target corresponding to the first matching detection frame and the shooting device that shoots the image according to the first matching detection frame, and then updating the parameters of the Kalman filter corresponding to the target corresponding to the first matching detection frame according to the distance.

[0101] The distance between the target and the camera device can be predicted by a neural network model based on the first matching detection frame. For example, the neural network model is trained in advance, and the distance between different detection frames and the camera device is used as label data to train the neural network model. In this way, the distance between the target and the camera device can be quickly output by the neural network model. The neural network model can be a convolutional neural network model, etc.

[0102] The shooting device may refer to a camera on an autonomous driving vehicle, so that after obtaining the distance between the camera and the target, the parameters of the corresponding Kalman filter may be dynamically adjusted to adapt to the influence of the speed of change of the detection frame at different distances of the target on the target detection.

[0103] In some implementations, updating the parameters of the Kalman filter includes updating the state transfer matrix and the measurement noise.

[0104] The time update equation of the Kalman filter is:

[0105]

[0106]

[0107] The state update equation of the Kalman filter is:

[0108]

[0109]

[0110]

[0111] in, They represent the posterior state estimates at time k-1 and k respectively, which are one of the results of filtering, that is, the updated result, that is, the Kalman update value;

[0112] It represents the prior state estimate at time k, which is the intermediate calculation result of filtering, that is, the result at time k predicted based on the optimal estimate at the previous moment, which is the result of the prediction equation and can be called the Kalman prediction value;

[0113] P k-1 , P k They represent the posterior estimated covariance at time k-1 and time k respectively, which is one of the results of filtering;

[0114] Represents the prior estimated covariance at time k, which is the intermediate calculation result of filtering;

[0115] H represents the transfer distance from the state variable to the observation, which represents the relationship connecting the state and the observation. In the Kalman filter, it is a linear relationship, which is responsible for converting the m-dimensional measurement value to the n-dimensional one so that it conforms to the mathematical form of the state variable. It is one of the prerequisites for filtering. In the embodiment of the present application, H is set to the unit matrix.

[0116] z k Refers to the observed value, which is the input of the filter. In the embodiment of the present application, it refers to the detection value obtained by the deep learning algorithm detection;

[0117] K k Represents the Kalman filter parameters, which are the intermediate calculation results of the filter;

[0118] A represents the state transfer matrix, which is actually a conjecture model of the target state transition;

[0119] Q represents the process excitation noise covariance, which is used to represent the error between the state transition matrix and the actual process. In the embodiment of the present application, it can be set to a constant value according to the requirements;

[0120] R represents the measurement noise;

[0121] B is the matrix that transforms input into state;

[0122] It represents the residual between the actual observation and the predicted observation, and together with the Kalman coefficient, it corrects the Kalman prediction value to obtain the Kalman update value.

[0123] Since the detection of moving targets by the Kalman filter is generally affected by the state transfer matrix and measurement noise on the accuracy, in the embodiment of the present application, the parameters involved in the update are the state transfer matrix A and the measurement noise R.

[0124] In the above implementation process, the parameters of the Kalman filter are updated according to the distance, so that the relevant parameters of the Kalman filter can be dynamically adjusted according to the distance of the target, so that the Kalman filter can adapt to the impact of changes in the detection frame of targets at different distances on target detection, so as to improve the target detection accuracy.

[0125] In some embodiments, an update coefficient for updating a state transfer matrix of a Kalman filter corresponding to a target corresponding to a first matching detection frame may be obtained based on the distance, wherein the update coefficient is negatively correlated with the distance, and then the state transfer matrix may be updated using the update coefficient.

[0126] For example, assuming that the detection box moves in a uniform linear motion in the image coordinate system, then x k =A·x k-1 x k =[x,y,w,h,v x ,v y ,v w ,v h ] T , x k Represents the state of the target at time k, including the coordinates of the detection box, the box height, and the change speed of its four edges, where the state transfer matrix A is:

[0127]

[0128] Since the position and size of the detection frame do not actually change at a uniform speed, updating the state transfer matrix can refer to replacing d in the state transfer matrix with t Multiply by an update coefficient, so that the farther the distance between the target and the shooting device, the smaller the update coefficient, which conforms to the characteristics that the nearby target moves fast and the detection frame changes fast, and the distant target moves slowly and the detection frame changes slowly. Especially for targets with fast and uniform speed in both the horizontal and vertical coordinates, it is beneficial to improve the detection accuracy of the Kalman filter for targets with these characteristics.

[0129] In some implementations, the update coefficient calculation formula may be: s=n / d, and the specific value of n may be set based on actual experience.

[0130] It can be understood that the calculation formula of the update coefficient can also be other formulas, which can be set according to actual needs, as long as the update coefficient is negatively correlated with the distance, so that the Kalman filter can adapt to the detection of close targets and distant targets.

[0131] In some embodiments, when updating the measurement noise, an update parameter of the measurement noise of the Kalman filter corresponding to the target corresponding to the first matching detection frame can be updated according to the distance, and the update parameter is positively correlated with the distance, and then the measurement noise is updated using the update parameter.

[0132] In order to prevent the detection frame of distant targets from shaking and the detection frame of nearby targets from failing to keep up during tracking, the farther the target, the greater the measurement noise. The specific method is to multiply the measurement noise by an update parameter r. The update parameter is dynamically adjusted according to the distance of the target. For example, the farther the target, the larger the update parameter r.

[0133] In some implementations, the calculation formula for the update parameter r may be as follows:

[0134] r=d 2 / p;

[0135] Among them, r represents the update parameter, d represents the distance, and p represents the preset coefficient. The specific value of the preset coefficient can be set according to actual conditions.

[0136] In some embodiments, to address the problem of detection frame jitter, another method can be used to update the measurement noise, such as counting the detection values ​​of the first matching detection frame within a preset time period, and then calculating the variance of the detection values, and obtaining an update parameter for updating the measurement noise of the Kalman filter corresponding to the target corresponding to the first matching detection frame based on the variance, where the variance is positively correlated with the update parameter, and then using the update parameter to update the measurement noise.

[0137] Among them, the detection value of the first matching detection frame may refer to the coordinates of the first matching detection frame in the image, such as the center point coordinates, and the preset time period may refer to the detection value of the detection frame corresponding to the first matching detection frame in the image before the current frame image, and its detection value refers to the detection value detected by the Kalman filter. For example, in the first frame image, the Kalman filter corresponding to a certain detection frame is the Kalman filter corresponding to the first matching detection frame, and then the detection value of the detection frame is obtained. That is to say, the detection value of the detection frame corresponding to the Kalman filter corresponding to the first matching detection frame in the frame image within the preset time period can be found based on the Kalman filter, and then the variance of the detection value is calculated to count the discreteness of the detection frame, so that the measurement noise can be dynamically adjusted according to the variance, such as the larger the variance, the larger the update parameter, and the smaller the variance, the smaller the update parameter.

[0138] The formula for calculating the update parameter may be: update parameter=m*variance, and the specific value of m may be set according to the actual situation. Of course, in this embodiment, the calculation formula for the update parameter may also be other formulas, as long as the update parameter is positively correlated with the variance.

[0139] The updating of the measurement noise may be performed by multiplying the updated parameter by the original measurement noise, so as to obtain the updated measurement noise.

[0140] In the above implementation process, by obtaining update parameters according to the variance, the changes of the detection box over a period of time can be comprehensively considered to more accurately adjust the measurement noise.

[0141] Method 2: obtaining the size of the first matching detection frame, and then updating the parameters of the Kalman filter corresponding to the target corresponding to the first matching detection frame according to the size.

[0142] The parameter update of the Kalman filter includes the update of the state transfer matrix and the update of the measurement noise. The update method is similar to the above method 1. For example, the update coefficient for updating the state transfer matrix can also be obtained according to the size, and then the state transfer matrix is ​​updated using the update coefficient, wherein the update coefficient is positively correlated with the size. In other words, the larger the size of the first matching detection frame, the larger the update coefficient, and the smaller the size, the smaller the update coefficient, so that it can adapt to the detection of targets at different distances.

[0143] In this way, as an example, the calculation formula of the update coefficient becomes: update coefficient = size * k, and the specific value of k can be set according to actual conditions.

[0144] When updating, the updated state transfer matrix can be obtained by multiplying dt in the state transfer matrix by the update coefficient.

[0145] It is also possible to obtain update parameters for updating the measurement noise according to the size, and then use the update parameters to update the measurement noise. In this case, the update parameters are negatively correlated with the size. That is, the larger the size, the smaller the update parameters, and the smaller the size, the larger the update parameters. This can avoid the impact of the jitter of the detection frame on the target when detecting distant targets.

[0146] At this time, as an example, the calculation formula of the update parameter becomes: update parameter=h / size, and the specific value of h can be set according to actual conditions.

[0147] During updating, the measurement noise may be multiplied by the update parameter to obtain the updated measurement noise.

[0148] In the above implementation process, the parameters of the Kalman filter are updated according to the size of the detection frame, so that the influence of the size of the target in the image on the target detection accuracy can be considered to improve the detection accuracy.

[0149] Method 3: obtaining the pixel ratio of the pixels corresponding to the first matching detection frame in the current frame image, and then updating the parameters of the Kalman filter corresponding to the target corresponding to the first matching detection frame according to the pixel ratio.

[0150] The parameter update of the Kalman filter includes the update of the state transfer matrix and the update of the measurement noise. The update method is similar to the above method 2. For example, the update coefficient for updating the state transfer matrix can be obtained according to the pixel ratio, and then the state transfer matrix is ​​updated using the update number, where the update coefficient is positively correlated with the pixel ratio. In other words, the larger the pixel ratio, the larger the update coefficient, and the smaller the pixel ratio, the smaller the update coefficient, which can adapt to the detection of targets at different distances.

[0151] At this time, as an example, the calculation formula of the update coefficient becomes: update coefficient = pixel ratio * f, and the specific value of f can be set according to actual conditions.

[0152] When updating, the updated state transfer matrix can be obtained by multiplying dt in the state transfer matrix by the update coefficient.

[0153] It is also possible to obtain an update parameter for updating the measurement noise according to the pixel ratio, and then use the update parameter to update the measurement noise. At this time, the update parameter is negatively correlated with the pixel ratio. That is, the larger the pixel ratio, the smaller the update parameter, and the smaller the pixel ratio, the larger the update parameter. This can avoid the impact of the detection frame jitter on the target when detecting distant targets.

[0154] At this time, as an example, the calculation formula of the update parameter becomes: update parameter = g / pixel ratio, and the specific value of g can be set according to actual conditions.

[0155] During updating, the measurement noise may be multiplied by the update parameter to obtain the updated measurement noise.

[0156] In the above implementation process, the parameters of the Kalman filter are updated according to the pixel ratio of the detection frame, so that the influence of the size of the target in the image on the target detection accuracy can be considered to improve the detection accuracy.

[0157] It is understandable that in practical applications, there are other ways to update the parameters of the Kalman filter, such as updating the parameters of the Kalman filter according to the number of pixels of the first matching detection frame, or updating the parameters of the Kalman filter according to the coordinates of the first matching detection frame, or updating the parameters of the Kalman filter according to the change of the coordinates of the first matching detection frame, etc., which are not listed here. It is understandable that other unlisted methods should also be included in the protection scope of this application.

[0158] In the above implementation process, by dynamically adjusting the parameters of the Kalman filter according to the distance between the target and the autonomous driving vehicle, appropriate parameters can be selected for target detection according to the distance between different targets and the autonomous driving vehicle, thereby improving the target detection accuracy and target tracking effect.

[0159] In some embodiments, in order to avoid the problem of poor target tracking effect caused by false detection or missed detection of the target, after matching the first detection frame with the second detection frame as mentioned above, at least one first unmatched detection frame that does not match the second detection frame can be obtained, and then a Kalman filter corresponding to the target corresponding to the first unmatched detection frame can be created based on the first unmatched detection frame.

[0160] For example, if there is a first unmatched detection frame in the first detection frame that does not match the second detection frame, it means that a new target may appear in the current frame image, and a new Kalman filter can be created for the target, that is, a Kalman filter can be initialized according to the relevant information of the first unmatched detection frame. In this way, when detecting the target in the next frame image, the newly added target is detected by the Kalman filter, and it can be known whether the target is misdetected by matching the detection frames in subsequent frame images.

[0161] In order to identify each target, for each detection box, the member variables shown in Table 1 below can be:

[0162] Table 1

[0163]

[0164]

[0165] In order to distinguish the newly added targets mentioned above, the target corresponding to the first unmatched detection frame can also be marked with a new target identifier. Since the new target may also be a target that is misdetected by the deep learning algorithm, in order to distinguish the new target from other targets that are not misdetected, the new target identifier can be different from the identifiers of other targets that are not misdetected. For example, the target identifier of a target that is not misdetected ranges from 0 to 4095, while the new target identifier is -1. In this way, in subsequent judgments, it is possible to determine whether it is a misdetected target based on the identifier and other information.

[0166] In order to record the matching status of each detection frame, so that under certain conditions, it is possible to judge whether the detection frame is falsely detected or missed, after the first unmatched detection frame is obtained as mentioned above, the number of successful matches of the first unmatched detection frame is recorded as 0, and the number of failed matches is recorded as 0.

[0167] In the above implementation process, a new Kalman filter is created for the unmatched detection frame, which can increase the detection of newly appeared targets.

[0168] In some embodiments, the second detection frame is predicted by a Kalman filter. Since the Kalman filter refers to the Kalman filter corresponding to each target in the previous frame image, if there are several second detection frames in the previous frame image, there are also the same number of second detection frames in the current frame image. However, if some targets disappear in the current frame image, there will be redundant detection frames predicted by the Kalman filter, which may be misdetected targets. In order to detect this situation, at least one second unmatched detection frame that does not match the first detection frame can be obtained, and the number of matching failures of the second unmatched detection frame is counted. When the number of matching failures of the second unmatched detection frame is greater than a preset value, the second unmatched detection frame is deleted.

[0169] Among them, counting the number of matching failures of the second unmatched detection frame can be understood as counting the number of matching failures of the second unmatched detection frame in the historical frame image. For example, if the Kalman filter corresponding to the second unmatched detection frame is x1, then the number of matching failures of the detection frame predicted by x1 in each frame image is counted. When each matching fails, the number of matching failures is increased by 1. If the matching still fails in the current frame image, the target may have disappeared, and there is no need to detect the target. Therefore, when the number of matching failures is greater than the preset value, the detection frame can be deleted, and the Kalman filter x1 can be deleted at the same time. Among them, the preset value can be flexibly set according to actual needs, such as being set to 1 or 2.

[0170] In the above implementation process, by counting the number of matching failures of the unmatched second unmatched detection frame, it is determined whether the target corresponding to the second unmatched detection frame disappears in the current frame image according to the number of matching failures, thereby avoiding the problem of false detection of the target.

[0171] In some cases, in order to compensate for the situation where the deep learning algorithm misses the target, at least one second unmatched detection frame that does not match the first detection frame can be obtained, and the number of matching failures of the second unmatched detection frame can be counted. When the number of matching failures of the second unmatched detection frame is less than a preset value and the identifier of the target corresponding to the second unmatched detection frame is not a new target identifier, the Kalman prediction value of the second unmatched detection frame is output.

[0172] Although the second unmatched detection frame fails to match successfully, if the number of matching failures is less than a preset value (the preset value may be the same as or different from the above preset value, and may be flexibly set according to actual conditions), and the target is not a new target, it can be determined that the target in the second unmatched detection frame may be a target that has not been detected by the deep learning algorithm, that is, a missed target. At this time, the Kalman prediction value of the second unmatched detection frame can be output.

[0173] The Kalman prediction value refers to the prediction value obtained by the Kalman filter, that is, For the matched second detection frame, a more accurate Kalman update value can be obtained through the Kalman filter in combination with the first detection frame detected by the deep learning algorithm, and then the Kalman update value is output.

[0174] The detection value of the output detection frame can provide data references such as early warning or AEB control signals for the autonomous driving vehicle. For example, the autonomous driving vehicle can know the direction and distance of each target according to the detection value of each detection frame, and then adjust its own motion state according to this information.

[0175] In some embodiments, for a successfully matched detection frame, in order to ensure the accuracy of target detection, the detection value of the detection frame may be output under certain conditions, such as counting the number of successful matches of the first matching detection frame, and when the number of successful matches is greater than a preset value, marking the target identifier of the target corresponding to the first matching detection frame, and outputting the Kalman update value of the first matching detection frame.

[0176] It can be understood that when the number of successful matches of the first matching detection frame is greater than a preset value (the preset value may be different from or the same as the above two preset values, and may be flexibly set according to actual conditions), it indicates that there is no false detection of the target in the first matching detection frame, and the Kalman update value of the first matching detection frame can be output. The output Kalman update value can provide data reference for the vehicle's autonomous driving.

[0177] The Kalman update value refers to an update value obtained by correcting the predicted value obtained by the Kalman filter algorithm using the detection value obtained by the deep learning algorithm. The correction process will not be described in detail here, and reference can be made to the implementation process of the Kalman filter algorithm in the prior art.

[0178] In the above implementation process, for the first matching detection frame that is successfully matched, when the number of successful matches is greater than a preset value, it indicates that the target corresponding to the first matching detection frame is a real target, thereby avoiding outputting detection values ​​for misdetected targets and causing interference to the vehicle's automatic driving.

[0179] The specific process of the above target detection is described below through a specific embodiment.

[0180] For the first frame image, the detection frames 1, 2, and 3 are obtained through the deep learning algorithm. A Kalman filter is initialized for each detection frame. Since the prediction of the Kalman filter needs to be based on the previous state of the target, the initial state of the target is the detection frame detected by the deep learning algorithm.

[0181] For the sake of ease of description, the second detection frame obtained by Kalman filter detection is referred to as detection frame A, and the first detection frame obtained by deep learning algorithm detection is referred to as detection frame B.

[0182] For the second frame image, the detection box B: a1, b1, c1 is obtained through the deep learning algorithm detection, and the detection box A: x1, y1, z1 is obtained through the initialized Kalman filter prediction obtained after the first frame image detection. If Kalman filter 1 corresponds to the prediction x1, Kalman filter 2 corresponds to the prediction y1, and Kalman filter 3 corresponds to the prediction z1.

[0183] Next, the detection box A is matched with the detection box in the detection box B. If the matching results obtained include: a1 matches x1, b1 matches y1, and c1 and z1 do not match.

[0184] For detection frame a1: Use detection frame a1 to update the parameters of Kalman filter 1 corresponding to x1, set the number of matching failures corresponding to a1 and x1 to 0, and the number of matching successes to 1.

[0185] For detection frame b1: Use detection frame b1 to update the parameters of Kalman filter 2 corresponding to y1, set the number of matching failures corresponding to b1 and y1 to 0, and the number of matching successes to 1;

[0186] For detection frame c1: At this time, it is a newly added detection frame, so a new target c is created, and the Kalman filter 4 corresponding to the target c is initialized using the detection frame c1, and the id of the target c is set to -1 (the new target identifier of the new target), the number of failed matches is 0, the number of successful matches is 0, and the detection frame c1 is added to the detection frame C (for use in the detection of the next frame of the image);

[0187] For detection box z1: add 1 to the number of failed matches of detection box z1.

[0188] The Kalman filter includes: Kalman filter 1 (x1), Kalman filter 2 (y1), Kalman filter 3 (z1) and Kalman filter 4 (c1). For these detection frames, a detection frame pool C can be established, that is, the detection frames in C include x1, y1, z1, c1. The statistics of each detection frame are:

[0189] a1 (mached_n: 1, fali_mached_n: 0); b1 (mached_n: 1, fali_mached_n: 0); c1 (mached_n: 0, fali_mached_n: 1); x1 (mached_n: 1, fali_mached_n: 0); y1 (mached_n: 1, fali_mached_n: 0); z1 (mached_n: 0, fali_mached_n: 1).

[0190] Continue to detect the third frame image. The detection frames obtained by the deep learning algorithm include: a2, b2, d2, c2. The detection frame in A obtained after the second frame image detection is used to predict the detection frame A through the Kalman filter: x2, y2, z2, r2. At this time, x2 is predicted by Kalman filter 1, y2 is predicted by Kalman filter 2, z2 is predicted by Kalman filter 3, and r2 is predicted by Kalman filter 4.

[0191] After matching the detection frames, the matching results obtained include: a2 matches x2, b2 matches y2, and the remaining d2, c2, z2, and r2 do not match.

[0192] After matching, Kalman filter 1 and Kalman filter 2 are updated using the above method, and then the statistics of each detection box are recorded as follows:

[0193] a2(mached_n:2, fali_mached_n:0), here is because a1 matches x1, and the Kalman filter corresponding to x1 predicts x2, so it is equivalent to a2 and a1 here are detection boxes for the same target; b2(mached_n:2, fali_mached_n:0), the targets corresponding to b1 and b2 are the same target; c2(mached_n:0, fali_mached_n:2), the targets corresponding to c1 and c2 are the same target; d2(mached_n:0, fali_mached_n:1); x2(mached_n:2, fali_mached_n:0); y2(mached_n:2, fali_mached_n:0); z2(mached_n:0, fali_mached_n:2); r2(mached_n:0, fali_mached_n:1).

[0194] At this time, for the detection frames obtained by the deep learning algorithm, if the number of successful matches of a2 and b2 is greater than the preset value (if the preset value is 1), the corresponding target identifiers can be set for a2 and b2, and then the Kalman update values ​​corresponding to the two detection frames can be output. The detection values ​​of the detection frames c2 and d2 are not output. At this time, c2 and d2 can create a new Kalman filter 5 and Kalman filter 6 and add them to the detection frame pool C.

[0195] Then, for z2 in the detection frame pool C, if the number of matching failures is greater than a preset value (if the preset value is 1), it means that the target in the detection frame corresponding to z2 has disappeared, and the detection frame z2 can be deleted from the detection frame pool C.

[0196] Alternatively, for z2 in the detection frame pool C, the number of matching failures is less than the preset value (if the preset value is 3), and the target identifier of the target corresponding to z2 is not -1, that is, it is not a new target identifier, then the Kalman prediction value of the detection frame z2 can be output.

[0197] For r2 with a new target identifier, when the number of matching failures is greater than a preset value (if the preset value is 2), it indicates that the target corresponding to r2 may be a false detection, and it can be deleted from the detection pool C.

[0198] In addition, in the subsequent frame detection process, if the number of successful matches of the new target r2 is greater than the set value, the new target identifier can be updated to a normal target identifier, that is, the target is identified as a real target, not a falsely detected target.

[0199] After detecting the third frame image, the detection frames contained in the detection pool C include: x2, y2, r2, and then the Kalman filters corresponding to these detection frames can be used to continue predicting the next frame image.

[0200] According to the above process, subsequent frame images can continue to be detected, so as to achieve tracking detection of the same target, and the targets that are misdetected or missed by the deep learning algorithm can be corrected through the Kalman filter to achieve accurate detection of the target, which can effectively improve the tracking effect of the target while improving the target detection accuracy.

[0201] Please refer to Figure 3 , Figure 3 This is a structural block diagram of a target detection device 200 provided in an embodiment of the present application. The device 200 may be a module, program segment or code on an electronic device. It should be understood that the device 200 is similar to the above Figure 2 The method embodiment corresponds to and can be executed Figure 2 The various steps involved in the method embodiment and the specific functions of the device 200 can be found in the above description. To avoid repetition, the detailed description is appropriately omitted here.

[0202] Optionally, the device 200 includes:

[0203] A first detection module 210 is used to detect the target in the current frame image by using a deep learning algorithm to obtain at least one first detection frame;

[0204] A second detection module 220 is used to predict the target in the current frame image by using a Kalman filter to obtain at least one second detection frame, wherein the Kalman filter is a Kalman filter corresponding to each target obtained by detecting the target in the previous frame image of the current frame image;

[0205] A matching module 230 is used to match each first detection frame with each second detection frame, and obtain a first matching detection frame in the at least one first detection frame that matches the second detection frame;

[0206] The updating module 240 is used to update the parameters of the Kalman filter corresponding to the target corresponding to the first matching detection frame according to the first matching detection frame, so as to use the updated Kalman filter to predict the target in the next frame image.

[0207] Optionally, the update module 240 is used to obtain the distance between the target corresponding to the first matching detection frame and the shooting device that shoots the image based on the first matching detection frame; and update the parameters of the Kalman filter corresponding to the target corresponding to the first matching detection frame based on the distance.

[0208] Optionally, the update module 240 is used to obtain an update coefficient for updating a state transfer matrix of a Kalman filter corresponding to a target corresponding to the first matching detection frame according to the distance; wherein the update coefficient is negatively correlated with the distance; and the state transfer matrix is ​​updated using the update coefficient.

[0209] Optionally, the update module 240 is used to obtain an update parameter for updating the measurement noise of the Kalman filter corresponding to the target corresponding to the first matching detection box according to the distance; wherein the update parameter is positively correlated with the distance; and the measurement noise is updated using the update parameter.

[0210] Optionally, the calculation formula of the update parameter is:

[0211] r=d 2 / p;

[0212] Among them, r represents the update parameter, d represents the distance, and p represents the preset coefficient.

[0213] Optionally, the update module 240 is used to count the detection values ​​of the first matching detection frame within a preset time period; calculate the variance of the detection values; obtain an update parameter for updating the measurement noise of the Kalman filter corresponding to the target corresponding to the first matching detection frame according to the variance, and the variance is positively correlated with the update parameter; and use the update parameter to update the measurement noise.

[0214] Optionally, the updating module 240 is used to obtain a size of the first matching detection frame; and update parameters of a Kalman filter corresponding to the target corresponding to the first matching detection frame according to the size.

[0215] Optionally, the update module 240 is used to obtain the pixel ratio of the pixels corresponding to the first matching detection frame in the current frame image; and update the parameters of the Kalman filter corresponding to the target corresponding to the first matching detection frame according to the pixel ratio.

[0216] Optionally, the device 200 further includes:

[0217] The new target detection module is used to obtain a first unmatched detection frame that is not matched with the second detection frame in the at least one first detection frame; and to create a Kalman filter corresponding to the target corresponding to the first unmatched detection frame according to the first unmatched detection frame.

[0218] Optionally, the new target detection module is used to mark the target corresponding to the first unmatched detection frame with a new target identifier.

[0219] Optionally, the device 200 further includes:

[0220] A detection frame deletion module is used to obtain a second unmatched detection frame that does not match the first detection frame among the at least one second detection frame; count the number of matching failures of the second unmatched detection frame; and delete the second unmatched detection frame when the number of matching failures of the second unmatched detection frame is greater than a preset value.

[0221] Optionally, the device 200 further includes:

[0222] A prediction value output module is used to obtain a second unmatched detection frame that is not matched with the first detection frame in the at least one second detection frame; count the number of matching failures of the second unmatched detection frame; when the number of matching failures of the second unmatched detection frame is less than a preset value and the identifier of the target corresponding to the second unmatched detection frame is not a new target identifier, output the Kalman prediction value of the second unmatched detection frame.

[0223] Optionally, the device 200 further includes:

[0224] An update value output module is used to count the number of successful matches of the first matching detection frame; when the number of successful matches is greater than a preset value, mark the target corresponding to the first matching detection frame with a target identifier, and output the Kalman update value of the first matching detection frame.

[0225] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0226] The present application provides a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the following is performed: Figure 2 The method process in the method embodiment shown is executed by the electronic device.

[0227] The present embodiment discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the methods provided by the above-mentioned method embodiments, for example, including: detecting the target in the current frame image through a deep learning algorithm to obtain at least one first detection frame; predicting the target in the current frame image through a Kalman filter to obtain at least one second detection frame, wherein the Kalman filter is a Kalman filter corresponding to each target obtained by detecting the target in the previous frame image of the current frame image; matching each first detection frame with each second detection frame to obtain a first matching detection frame in the at least one first detection frame that matches the second detection frame; and updating the parameters of the Kalman filter corresponding to the target corresponding to the first matching detection frame according to the first matching detection frame, so as to use the updated Kalman filter to predict the target in the next frame image.

[0228] In summary, the embodiments of the present application provide a method and device for target detection, which detects targets in images by combining a deep learning algorithm and a Kalman filter, so that the continuity of the target's motion in the previous and next frame images can be well considered, and the false detection rate and missed detection rate can be effectively improved. And by timely updating the parameters of the Kalman filter, the state changes of the moving target can be updated in time, so that the detection accuracy of the target in the subsequent frame image is higher.

[0229] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0230] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0231] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0232] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0233] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A target detection method, characterized in that: The method comprises: Detecting the target in the current frame image by using a deep learning algorithm to obtain at least one first detection frame; Predicting the target in the current frame image by using a Kalman filter to obtain at least one second detection frame, wherein the Kalman filter is a Kalman filter corresponding to each target obtained by detecting the target in the previous frame image of the current frame image; Match each first detection frame with each second detection frame, and obtain a first matching detection frame in at least one first detection frame that matches the second detection frame; updating the parameters of the Kalman filter corresponding to the target corresponding to the first matching detection frame according to the first matching detection frame, so as to predict the target in the next frame image by using the updated Kalman filter; The updating of the parameters of the Kalman filter corresponding to the target corresponding to the first matching detection frame according to the first matching detection frame includes: The parameters of the Kalman filter corresponding to the target corresponding to the first matching detection frame are updated according to the relevant parameters of the first matching detection frame, wherein the relevant parameters include the distance between the target corresponding to the first matching detection frame and the shooting device that shoots the image, the variance of the detection value of the first matching detection frame within a preset time period, the size of the first matching detection frame, the pixel ratio of the pixels corresponding to the first matching detection frame in the current frame image, the number of pixels of the first matching detection frame, the coordinates of the first matching detection frame and / or the change in the coordinates of the first matching detection frame.

2. The method according to claim 1, characterized in that The updating, according to the first matching detection frame, of parameters of a Kalman filter corresponding to the target corresponding to the first matching detection frame comprises: Acquire, according to the first matching detection frame, a distance between a target corresponding to the first matching detection frame and a shooting device that shoots an image; The parameters of the Kalman filter corresponding to the target corresponding to the first matching detection frame are updated according to the distance.

3. The method according to claim 2, characterized in that The updating of the parameters of the Kalman filter corresponding to the target corresponding to the first matching detection frame according to the distance includes: Acquire, according to the distance, an update coefficient for updating a state transfer matrix of a Kalman filter corresponding to the target corresponding to the first matching detection frame; wherein the update coefficient is negatively correlated with the distance; The state transfer matrix is ​​updated using the update coefficients.

4. The method according to claim 2, characterized in that: The updating of the parameters of the Kalman filter corresponding to the target corresponding to the first matching detection frame according to the distance includes: Acquire, according to the distance, an update parameter for updating the measurement noise of the Kalman filter corresponding to the target corresponding to the first matching detection frame; wherein the update parameter is positively correlated with the distance; The measurement noise is updated using the update parameter.

5. The method according to claim 4, characterized in that The calculation formula of the update parameter is: r=d 2 / p; Among them, r represents the update parameter, d represents the distance, and p represents the preset coefficient.

6. The method according to claim 1, characterized in that The updating, according to the first matching detection frame, of parameters of a Kalman filter corresponding to the target corresponding to the first matching detection frame comprises: Counting the detection value of the first matching detection frame within a preset time period; Calculating the variance of the detection value; Acquire an update parameter for updating the measurement noise of the Kalman filter corresponding to the target corresponding to the first matching detection frame according to the variance, wherein the variance is positively correlated with the update parameter; The measurement noise is updated using the update parameter.

7. The method according to any one of claims 1 to 6, characterized in that: After matching each first detection frame with each second detection frame, the method further includes: Acquire a first unmatched detection frame in the at least one first detection frame that does not match the second detection frame; A Kalman filter corresponding to the target corresponding to the first unmatched detection box is newly created according to the first unmatched detection box.

8. The method according to any one of claims 1 to 6, characterized in that: After matching each first detection frame with each second detection frame, the method further includes: Acquire a second unmatched detection frame in the at least one second detection frame that does not match the first detection frame; Counting the number of matching failures of the second unmatched detection frame; When the number of matching failures of the second unmatched detection frame is greater than a preset value, the second unmatched detection frame is deleted.

9. The method according to any one of claims 1 to 6, characterized in that: After matching each first detection frame with each second detection frame, the method further includes: Acquire a second unmatched detection frame in the at least one second detection frame that does not match the first detection frame; Counting the number of matching failures of the second unmatched detection frame; When the number of matching failures of the second unmatched detection frame is less than a preset value and the identifier of the target corresponding to the second unmatched detection frame is not a new target identifier, the Kalman prediction value of the second unmatched detection frame is output.

10. The method according to any one of claims 1 to 6, characterized in that: After obtaining the first matching detection frame that matches the second detection frame in the at least one first detection frame, the method further includes: Counting the number of successful matches of the first matching detection frame; When the number of successful matches is greater than a preset value, a target identifier is marked on the target corresponding to the first matching detection frame, and a Kalman update value of the first matching detection frame is output.

11. A target detection device, characterized in that: The device comprises: A first detection module, configured to detect a target in a current frame image by using a deep learning algorithm to obtain at least one first detection frame; A second detection module is used to predict the target in the current frame image through a Kalman filter to obtain at least one second detection frame, wherein the Kalman filter is a Kalman filter corresponding to each target obtained by detecting the target in the previous frame image of the current frame image; A matching module, configured to match each first detection frame with each second detection frame, and obtain a first matching detection frame in the at least one first detection frame that matches the second detection frame; An updating module, configured to update the parameters of the Kalman filter corresponding to the target corresponding to the first matching detection frame according to the first matching detection frame, so as to predict the target in the next frame image by using the updated Kalman filter; Among them, the update module is specifically used to update the parameters of the Kalman filter corresponding to the target corresponding to the first matching detection frame according to the relevant parameters of the first matching detection frame, wherein the relevant parameters include the distance between the target corresponding to the first matching detection frame and the shooting device that shoots the image, the variance of the detection value of the first matching detection frame within a preset time period, the size of the first matching detection frame, the pixel ratio of the pixels corresponding to the first matching detection frame in the current frame image, the number of pixels of the first matching detection frame, the coordinates of the first matching detection frame and / or the change in the coordinates of the first matching detection frame.

Images