Target tracking optimization method, device, computer equipment and storage medium
By identifying and occluding image frames, combining window filters and vector register technology, the problem of poor target tracking stability in automatic assisted driving is solved, and efficient target tracking is achieved in complex environments.
Patent Information
- Application Number
- CN202111613778.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-27
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-12-27
AI Technical Summary
In automatic assisted driving scenarios, target tracking technology is easily misidentified and lost due to occlusion factors caused by complex road conditions, resulting in poor tracking stability.
By performing object recognition processing on image frames, filtering effective position points, judging the occlusion situation, and using a memorable window filter for occlusion, combining SSD fast detector and RISC-V vector register to achieve multi-objective tracking and vector acceleration.
It improves the stability and efficiency of target tracking, reduces resource occupation and algorithm running time, and realizes continuous target tracking under complex road conditions.
Smart Images

Figure CN114463377B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of target tracking technology. More specifically, the present invention can provide a target tracking optimization method, device, computer equipment and storage medium. Background Art
[0002] With the development of Internet of Things and artificial intelligence technologies, automatic assisted driving technology has developed rapidly. In the actual application of automatic assisted driving technology, images of the vehicle's surroundings are acquired in real time through image acquisition devices such as cameras, and then the targets in the images of the surroundings are identified and tracked, so that the vehicle can be automatically controlled based on the identification and tracking results, or driving suggestions can be provided to the driver to achieve the purpose of automatic assisted driving. However, the actual road environment is often more complex than the simulated environment. If there is interference from occlusion factors when tracking the target, problems such as misidentification or loss of the tracked target often occur, and the stability of target tracking is poor. Therefore, target tracking technology urgently needs to be optimized and improved. Summary of the invention
[0003] In order to solve the problems of poor real-time and availability of traditional target tracking technology in automatic assisted driving scenarios, the embodiments of the present invention can provide a target tracking optimization method, device, computer equipment and storage medium to achieve technical purposes such as improving target tracking stability and solve one or more problems existing in the prior art.
[0004] To achieve the above technical objectives, the present invention provides a target tracking optimization method, which may include but is not limited to one or more of the following steps.
[0005] The target recognition process is performed on the currently received image frame to obtain sample response values of all position points in the image frame.
[0006] Position points whose sample response values are greater than the mean value of the response matrix are screened out from all the position points and are used as valid position points; the mean value of the response matrix is the mean value of the sample response values of all the position points.
[0007] Whether occlusion occurs is determined according to the coordinates of each valid position point and the coordinates of the position point with the maximum sample response value.
[0008] Further, determining whether occlusion occurs according to the coordinates of each valid position point and the coordinates of the position point with the maximum sample response value includes:
[0009] The sum of the distances between each valid position point and the position point with the maximum sample response value is determined by using the coordinates of the valid position points and the coordinates of the position point with the maximum sample response value.
[0010] According to the sum of the distances being greater than or equal to a first threshold, it is determined that occlusion has occurred.
[0011] According to the sum of the distances being less than the first threshold, it is determined that no occlusion occurs.
[0012] Further, determining that occlusion has occurred according to the sum of the distances being greater than or equal to a first threshold includes:
[0013] Based on the occlusion that has occurred, the sum of the distances is compared with a preset range to obtain a comparison result.
[0014] The occlusion level of the occlusion that has occurred is determined according to the comparison result.
[0015] Furthermore, the target tracking optimization method also includes:
[0016] According to the occlusion that has occurred, target position information of a first set number of image frames before the currently received image frame is obtained.
[0017] The target offset information is determined based on the preset offset vector and the target position information.
[0018] The target position information of the currently received image frame is determined according to the target offset information and the target position information of the previous image frame of the currently received image frame.
[0019] Further, the determining the target offset information based on the preset offset vector and the target position information includes:
[0020] The target position information of the first set number of image frames is used to determine the offset position information of two adjacent image frames.
[0021] The target offset information is determined by the preset offset and the offset position information of the two adjacent image frames.
[0022] Furthermore, the target tracking optimization method also includes:
[0023] Acquire feature information of multiple targets in the currently received image frame, and acquire track information of all targets in a second set number of image frames before the currently received image frame.
[0024] A correlation result between the target and the trajectory is determined based on the feature information and the trajectory information.
[0025] The matching result between the target and the track is determined according to the relationship between the correlation result and the second threshold.
[0026] In order to achieve the above technical objectives, the present invention can also provide a target tracking optimization device, which may include but is not limited to a target recognition processing module, a valid position screening module and an occlusion occurrence judgment module.
[0027] The target recognition processing module is used to perform target recognition processing on the currently received image frame to obtain sample response values of all position points in the image frame.
[0028] The effective position screening module is used to screen out the position points whose sample response values are greater than the response matrix mean from all the position points and use them as effective position points; the response matrix mean is the mean of the sample response values of all the position points.
[0029] The occlusion occurrence judgment module is used to determine whether occlusion occurs according to the coordinates of each valid position point and the coordinates of the position point with the maximum sample response value.
[0030] In order to achieve the above-mentioned technical objectives, the present invention can also provide a computer device, which may include a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the target tracking optimization method described in any embodiment of the present invention.
[0031] In order to achieve the above-mentioned technical objectives, the present invention may also provide a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the target tracking optimization method described in any embodiment of the present invention.
[0032] In order to achieve the above-mentioned technical objectives, the present invention can also provide a computer program product. When the instructions in the computer program product are executed by a processor, the steps of the target tracking optimization method described in any embodiment of the present invention are executed.
[0033] The beneficial effects of the present invention are as follows: based on the sample response value, the position point with the maximum sample response value and the effective position point are determined. The present invention can provide an occlusion detection mechanism to accurately determine whether the target is occluded; on the basis of effectively identifying the occlusion situation, a memorable window filter is used for occlusion processing, and the occlusion processing is performed in a manner of estimating the target position, which solves the occlusion problem under complex road conditions in practical applications and realizes continuous tracking of the target. It can be seen that the present invention can greatly improve the stability of target tracking, so as to better solve at least one problem existing in the prior art. Moreover, the present invention can use the SSD fast detector to replace the target detector of the KCF algorithm, and realize multi-target tracking through the data and trajectory association mechanism, which occupies less resources than traditional multi-threaded target tracking; in addition, the present invention can use the RISC-V extended vector instruction set, and through the characteristics of the vector register, the function of single instruction and multiple data is realized, thereby achieving the effect of vector acceleration. The present invention reduces the running time of the target tracking algorithm. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 A schematic flow chart of a target tracking optimization method in one or more embodiments of the present invention is shown.
[0035] Figure 2 A schematic diagram showing a detailed process of an occlusion processing algorithm in one or more embodiments of the present invention.
[0036] Figure 3 A detailed flowchart of a multi-target tracking algorithm in one or more embodiments of the present invention is shown.
[0037] Figure 4 A schematic diagram of performing a sample shift operation through a vector register in one or more embodiments of the present invention is shown.
[0038] Figure 5 A schematic diagram of a KCF tracking algorithm transplantation process for RISC-V in one or more embodiments of the present invention is shown.
[0039] Figure 6 A schematic diagram showing the composition of a target tracking optimization device in one or more embodiments of the present invention.
[0040] Figure 7 A schematic diagram of the internal structure of a computer device in one or more embodiments of the present invention is shown. DETAILED DESCRIPTION
[0041] The target tracking optimization method, device, computer equipment and storage medium provided by the present invention are explained and illustrated in detail below in conjunction with the drawings in the specification.
[0042] like Figure 1 As shown, and can be combined Figure 2 and Figure 3 One or more embodiments of the present invention may provide a target tracking optimization method, which includes but is not limited to at least one of the following steps, which are specifically described as follows.
[0043] First, the embodiment of the present invention performs target recognition processing on the currently received image frame to obtain sample response values of all position points in the image frame. The sample response values in the embodiment of the present invention can be obtained by express.
[0044] Secondly, the position points whose sample response values are greater than the mean of the response matrix are selected from all the position points and used as valid position points and the coordinates of the valid position points are determined. The coordinates of the valid position points are expressed as The response matrix mean is the mean of the sample response values at all locations.
[0045] The embodiment of the present invention expresses the response matrix mean value by the following formula:
[0046]
[0047] in, represents the Fourier transform of sample x, Parameters representing model updates.
[0048] Then, whether occlusion occurs is determined based on the coordinates of each valid position point and the coordinates of the position point with the maximum sample response value. The maximum sample response value in this embodiment can be obtained by Indicates that the coordinates of the location point with the maximum sample response value can be obtained by express.
[0049] Optionally, in this embodiment, determining whether occlusion occurs according to the coordinates of each valid position point and the coordinates of the position point with the maximum sample response value includes: using the coordinates of the valid position point and the coordinates of the location point with the maximum sample response value Determine the sum of the distances between each valid position point and the position point with the maximum sample response value; this embodiment uses the coordinates of the position point with the maximum sample response value Coordinates of valid locations The absolute value of the difference and sum it up, that is
[0050] According to the sum of the distances being greater than or equal to the first threshold, it is determined that occlusion has occurred. In this embodiment, the first threshold may be represented by ε1. If This indicates that occlusion has occurred.
[0051] According to the sum of the distances being less than the first threshold, that is Make sure no occlusion occurs.
[0052] Optionally, in an embodiment of the present invention, determining that occlusion has occurred based on the sum of the distances being greater than or equal to a first threshold may include: based on the occlusion having occurred, the present embodiment may output a signal that occlusion has occurred; comparing the sum of the distances with a preset range interval to obtain a comparison result, and determining the occlusion level of the occlusion that has occurred based on the comparison result. The preset range interval involved in the present invention may be set according to actual conditions, and the present invention is not limited thereto.
[0053] For the occlusion situation, the embodiment of the present invention can use a memorizable window filter to estimate the position of the target and perform occlusion processing. Then the target tracking optimization method provided by the embodiment of the present invention also includes the following steps, which are specifically described as follows.
[0054] According to the occlusion that has occurred, target position information of a first set number of image frames before the currently received image frame is obtained.
[0055] Determining target offset information based on a preset offset vector and target position information may specifically include the following steps.
[0056] When tracking starts, a window filter is created: the target position information of the first set number of image frames is used to determine the offset position information of two adjacent image frames. For the occlusion at the rth moment, the target position of the first r-1 frames in the embodiment of the present invention is specifically expressed as z=(z1, z2, z3, ..., z r-1 ), then this embodiment uses a window filter to represent the offset position information of two adjacent image frames, and the window filter can be represented as z′=(z2-z1, z3-z2, ..., z r -z r-1 ), then the preset offset vector corresponding to the present embodiment is g=(g1, g2, g3, ..., g r-1 ), by presetting the offset vector g to weight the window of the previous r-1 frames, where g1 to g r-1 Increasing.
[0057] Next, this embodiment determines the target offset information by using the preset offset and the offset position information of two adjacent image frames, specifically Δz=z′×g T , where g T represents the transpose of g.
[0058] The target position information of the currently received image frame is determined based on the target offset information and the target position information of the previous image frame of the currently received image frame. r-1 Determine the target position of frame r, z r =z r-1+Δz. And continue to track the target. If occlusion occurs again, continue to use the above method of the present invention. If there is no occlusion, continue to track the target. It can be seen that the present invention provides an occlusion detection mechanism through the response matrix mean, and uses a memorizable window filter to perform occlusion processing.
[0059] The present invention can also realize tracking of multiple targets. Specifically, the target tracking optimization method provided by the embodiment of the present invention also includes the following steps.
[0060] After the target tracking starts, this embodiment detects the input continuous frames through the SSD fast detector to obtain the trajectory information of all targets in the second set number of image frames before the currently received image frame, and obtains the feature information of multiple targets in the currently received image frame. This embodiment uses a set M to represent the feature information of each target detected by the SSD detection in the tth frame, and expresses the feature information through the formula: The set Z represents the set of all target trajectories at the t-1th frame, and the set of all target trajectories is expressed as
[0061] In this embodiment, k can be used to represent the number of targets detected in the t-th image frame. represents the feature information of the kth target at the tth frame; n represents the total number of trajectories that have been formed at the t-1th frame.
[0062] The correlation between the target and the trajectory is determined based on the feature information and the trajectory information, so as to calculate the correlation between the subsequent target and the generated trajectory. In this embodiment, the correlation between the target and the trajectory is calculated by using the data correlation representation method IoU (Intersection over Union), which can be calculated by the following formula: in, Indicates the position of the target tracked by the last frame of the j-th trajectory in the trajectory at frame t-1.
[0063] The matching result of the target and the trajectory is determined according to the relationship between the correlation result and the second threshold value, so as to realize multi-target tracking. The second threshold value of the embodiment of the present invention can be represented by ε2, and ε2 can be, for example, 0.5, but is not limited thereto. If the correlation result is greater than the second threshold value, it is determined that the target and the trajectory may be matched. If the correlation result is less than the second threshold value, it is determined that the target and the trajectory must not match, so as to perform the optimal matching of the target and the trajectory on this basis. The embodiment of the present invention can use the best row and column matching rule for judgment: the optimal match obtained in each row is also the optimal match of the corresponding column. Based on this matching method, the present invention can divide the targets into three categories: the target trajectory is successfully matched, the target is a new target, and the target is lost. This embodiment adopts the corresponding trajectory processing method for these situations: if the match is successful, the tracking continues; if a new target appears, it needs to be continuously detected in the subsequent tracking. If the target is detected in 5 consecutive frames, it is considered that the target needs to be tracked, and the target and trajectory matching is performed on the target in the next frame; if the target is lost, it is also necessary to detect in the subsequent 5 frames. If the target cannot be detected, it is considered that the target has left the field of view and does not need to be tracked. It can be seen from this that the present invention can implement a data association mechanism based on an SSD (Single Shot MultiBox Detector) fast detector to realize a multi-target tracking mechanism.
[0064] like Figure 5 As shown, the embodiment of the present invention can be implemented based on RISC-V (Reduced Instruction Set Computer-FIVE, the fifth generation of reduced instruction set). Facing the RISC-V instruction set, the present invention uses RISC-V ToolChain (tool chain) to cross-compile the OpenCV function library, realize the transplantation of the function library on the RISC-V architecture, and then realize the basic transplantation of the target tracking algorithm in the OpenCV function library on the RISC-V architecture. It can be seen that the present invention provides an optimization method for the target tracking algorithm based on RISC-V. The present invention first chooses to transplant the OpenCV computer vision library to the RISC-V platform, and adopts the mode of Spike simulator plus PK (proxy tool) proxy kernel to realize the RISC-V instruction set environment, and completes the relevant compilation and debugging through the RISC-V tool chain.
[0065] As a reduced instruction set, RISC-V has the characteristics of being completely open source, modularly designed, extensible and customizable. The RISC-V vector extension instruction set adds 32 vector registers relative to the scalar instruction set, and the bit width of each vector register is fixed. The purpose of the vector extension instruction is to realize the single instruction multiple data function of the RISC-V instruction set. The present invention can use this extension to perform vector acceleration on some modules in the transplanted tracking algorithm. The standard number of vector registers is 32, and the size of the vector register changes dynamically. The maximum size of the vector register is generally determined by the hardware characteristics of the processor. In an embodiment of the present invention, the length of the vector register can be selected by command calculation, vgrant=min(vlen, vreql), where vgrant represents the returned vector length, vlen represents the default length of the hardware processor, and vreql represents the input vector length. Based on the customizable characteristics, the present invention can be applied to the typical AIoT (AI+IoT, artificial intelligence Internet of Things) scenario of automatic assisted driving (Advanced Driving Assistance System). According to the hardware characteristics of RISC-V, the present invention adopts an extended instruction set to accelerate the target tracking algorithm, and optimizes the target tracking algorithm to adapt to the application scenarios of automated assisted driving, thereby realizing a target occlusion detection processing mechanism and a multi-target tracking function.
[0066] like Figure 4 As shown, the present invention provides access operations of sample data in vector registers. In the process of generating samples by cyclic matrix displacement of KCF algorithm, four different sample data can be stored in the same vector register by writing vector load instructions, that is, four sample data are stored in one vector register from low to high in sequence, and the vector register is respectively left-multiplied and right-multiplied. This command operates on four data, that is, a cyclic displacement operation is performed on four sample data, thereby realizing a single instruction multiple data mode. Based on the single instruction multiple data mode adopted by the present invention, multiple data are stored in the same vector register for use to reduce the amount of calculation of this module. V_S 2 vector register is used to save the offset of each data, and V_S 3 vector register is selected in atomic operation to provide source data. This operation can realize vector acceleration of RISC-V instruction set, and the acceleration effect will be doubled according to the number of data stored in the vector register according to the present invention. The above scheme of the present invention is applied in RISC-V environment to realize vector extension trajectory, compile the target tracking algorithm in the embodiment of the present invention, generate a binary file executable by RISC-V instruction set, and finally realize the transplantation and optimization of target tracking algorithm based on RISC-V instruction set.
[0067] like Figure 6As shown, based on the same inventive technical concept as the target tracking optimization method of the present invention, one or more embodiments of the present invention can also provide a target tracking optimization device.
[0068] The target tracking optimization device includes but is not limited to a target recognition processing module, an effective position screening module and an occlusion occurrence judgment module, which are specifically described as follows.
[0069] The target recognition processing module is used to perform target recognition processing on the currently received image frame to obtain sample response values of all position points in the image frame.
[0070] The effective position screening module is used to screen out the position points whose sample response values are greater than the mean value of the response matrix from all the position points and use them as effective position points; the mean value of the response matrix is the mean value of the sample response values of all the position points.
[0071] The occlusion occurrence judgment module is used to determine whether occlusion occurs according to the coordinates of each valid position point and the coordinates of the position point with the maximum sample response value.
[0072] Optionally, the occlusion occurrence judgment module can be used to determine the sum of the distances between each valid position point and the position point with the maximum sample response value by using the coordinates of the valid position points and the coordinates of the position point with the maximum sample response value; the occlusion occurrence judgment module is used to determine that occlusion has occurred based on the sum of the distances being greater than or equal to a first threshold; or the occlusion occurrence judgment module is used to determine that occlusion has not occurred based on the sum of the distances being less than the first threshold.
[0073] Optionally, the occlusion occurrence judgment module can be used to compare the sum of distances with a preset range interval based on the occlusion that has occurred to obtain a comparison result; the occlusion occurrence judgment module is used to determine the occlusion level of the occlusion that has occurred according to the comparison result.
[0074] The target tracking optimization device in the embodiment of the present invention may further include an occluded target recognition module, wherein the occluded target recognition module is used to obtain target position information of a first set number of image frames before the currently received image frame based on the occlusion that has occurred; the occluded target recognition module is used to determine target offset information based on a preset offset vector and target position information; the occluded target recognition module is used to determine the target position information of the currently received image frame based on the target offset information and the target position information of the previous image frame of the currently received image frame.
[0075] Optionally, the occluded target recognition module is used to determine the offset position information of two adjacent image frames using the target position information of the first set number of image frames, and the occluded target recognition module is used to determine the target offset information by a preset offset and the offset position information of two adjacent image frames.
[0076] The target tracking optimization device in the embodiment of the present invention can also include a multi-target tracking module, which is used to obtain feature information of multiple targets in a currently received image frame, and to obtain trajectory information of all targets in a second set number of image frames before the currently received image frame; the multi-target tracking module is used to determine the correlation result between the target and the trajectory based on the feature information and the trajectory information, and to determine the matching result between the target and the trajectory based on the relationship between the correlation result and the second threshold.
[0077] like Figure 7As shown, based on the same inventive technical concept as the target tracking optimization method of the present invention, one or more embodiments of the present invention can also provide a computer device, the computer device includes a memory and a processor, the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor can execute the steps of the target tracking optimization method in any embodiment of the present invention. Among them, the target tracking optimization method may include but is not limited to the following steps: First, target recognition processing is performed on the currently received image frame to obtain sample response values of all position points in the image frame. Secondly, position points whose sample response values are greater than the mean value of the response matrix are screened out from all position points, and are used as valid position points; the mean value of the response matrix is the mean value of the sample response values of all position points. Then, it is determined whether occlusion occurs based on the coordinates of each valid position point and the coordinates of the position point with the maximum sample response value. Optionally, the embodiment of the present invention determines whether occlusion occurs according to the coordinates of each valid position point and the coordinates of the position point with the maximum sample response value, including: using the coordinates of the valid position points and the coordinates of the position point with the maximum sample response value, determining the sum of the distances between each valid position point and the position point with the maximum sample response value; determining that occlusion has occurred according to the sum of the distances being greater than or equal to the first threshold; and determining that occlusion has not occurred according to the sum of the distances being less than the first threshold. Optionally, in the embodiment of the present invention, determining that occlusion has occurred according to the sum of the distances being greater than or equal to the first threshold includes: comparing the sum of the distances with a preset range interval based on the occlusion that has occurred to obtain a comparison result; and determining the occlusion level of the occlusion that has occurred based on the comparison result. Optionally, the target tracking optimization method provided by the embodiment of the present invention also includes: obtaining the target position information of a first set number of image frames before the currently received image frame based on the occlusion that has occurred; determining the target offset information based on the preset offset vector and the target position information; and determining the target position information of the currently received image frame based on the target offset information and the target position information of the previous image frame of the currently received image frame. Specifically, in an embodiment of the present invention, determining the target offset information based on a preset offset vector and target position information includes: determining the offset position information of two adjacent image frames using the target position information of a first set number of image frames; determining the target offset information by a preset offset and the offset position information of two adjacent image frames. Optionally, the target tracking optimization method provided in an embodiment of the present invention also includes: obtaining feature information of multiple targets in a currently received image frame, and obtaining trajectory information of all targets in a second set number of image frames before the currently received image frame; determining the correlation result between the target and the trajectory based on the feature information and the trajectory information; and determining the matching result between the target and the trajectory based on the relationship between the correlation result and the second threshold.
[0078] like Figure 7As shown, based on the same inventive technical concept as the target tracking optimization method of the present invention, one or more embodiments of the present invention can also provide a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, one or more processors execute the steps of the target tracking optimization method in any embodiment of the present invention. Among them, the target tracking optimization method may include but is not limited to the following steps: first, target recognition processing is performed on the currently received image frame to obtain sample response values of all position points in the image frame. Secondly, position points whose sample response values are greater than the mean value of the response matrix are screened out from all position points and used as valid position points; the mean value of the response matrix is the mean value of the sample response values of all position points. Then, whether occlusion occurs is determined according to the coordinates of each valid position point and the coordinates of the position point with the maximum sample response value. Optionally, the embodiment of the present invention determines whether occlusion occurs according to the coordinates of each valid position point and the coordinates of the position point with the maximum sample response value, including: using the coordinates of the valid position point and the coordinates of the position point with the maximum sample response value, determining the sum of the distances between each valid position point and the position point with the maximum sample response value; determining that occlusion has occurred according to the sum of the distances being greater than or equal to the first threshold; determining that occlusion has not occurred according to the sum of the distances being less than the first threshold. Optionally, in an embodiment of the present invention, determining that occlusion has occurred based on the sum of distances being greater than or equal to a first threshold value includes: comparing the sum of distances with a preset range interval based on the occlusion that has occurred to obtain a comparison result; and determining the occlusion level of the occlusion that has occurred based on the comparison result. Optionally, the target tracking optimization method provided in an embodiment of the present invention also includes: acquiring target position information of a first set number of image frames before the currently received image frame based on the occlusion that has occurred; determining target offset information based on a preset offset vector and target position information; and determining target position information of the currently received image frame based on the target offset information and the target position information of the previous image frame of the currently received image frame. Specifically, in an embodiment of the present invention, determining target offset information based on a preset offset vector and target position information includes: determining offset position information of two adjacent image frames using target position information of the first set number of image frames; and determining target offset information by a preset offset and the offset position information of two adjacent image frames. Optionally, the target tracking optimization method provided by an embodiment of the present invention also includes: obtaining feature information of multiple targets in a currently received image frame, and obtaining trajectory information of all targets in a second set number of image frames before the currently received image frame; determining a correlation result between the target and the trajectory based on the feature information and the trajectory information; and determining a matching result between the target and the trajectory based on a relationship between the correlation result and a second threshold.
[0079] Based on the same inventive technical concept as the target tracking optimization method of the present invention, one or more embodiments of the present invention can also provide a computer program product. When the instructions in the computer program product are executed by a processor, the steps of the target tracking optimization method in any embodiment of the present invention are executed. Among them, the target tracking optimization method may include but is not limited to the following steps: first, target recognition processing is performed on the currently received image frame to obtain sample response values of all position points in the image frame. Secondly, position points whose sample response values are greater than the mean value of the response matrix are screened out from all position points and used as valid position points; the mean value of the response matrix is the mean value of the sample response values of all position points. Then, whether occlusion occurs is determined according to the coordinates of each valid position point and the coordinates of the position point with the maximum sample response value. Optionally, the embodiment of the present invention determines whether occlusion occurs according to the coordinates of each valid position point and the coordinates of the position point with the maximum sample response value, including: using the coordinates of the valid position point and the coordinates of the position point with the maximum sample response value, determining the sum of the distances between each valid position point and the position point with the maximum sample response value; determining that occlusion has occurred according to the sum of the distances being greater than or equal to the first threshold; determining that occlusion has not occurred according to the sum of the distances being less than the first threshold. Optionally, in an embodiment of the present invention, determining that occlusion has occurred based on the sum of distances being greater than or equal to a first threshold value includes: comparing the sum of distances with a preset range interval based on the occlusion that has occurred to obtain a comparison result; and determining the occlusion level of the occlusion that has occurred based on the comparison result. Optionally, the target tracking optimization method provided in an embodiment of the present invention also includes: acquiring target position information of a first set number of image frames before the currently received image frame based on the occlusion that has occurred; determining target offset information based on a preset offset vector and target position information; and determining target position information of the currently received image frame based on the target offset information and the target position information of the previous image frame of the currently received image frame. Specifically, in an embodiment of the present invention, determining target offset information based on a preset offset vector and target position information includes: determining offset position information of two adjacent image frames using target position information of the first set number of image frames; and determining target offset information by a preset offset and the offset position information of two adjacent image frames. Optionally, the target tracking optimization method provided by an embodiment of the present invention also includes: obtaining feature information of multiple targets in a currently received image frame, and obtaining trajectory information of all targets in a second set number of image frames before the currently received image frame; determining a correlation result between the target and the trajectory based on the feature information and the trajectory information; and determining a matching result between the target and the trajectory based on a relationship between the correlation result and a second threshold.
[0080] The logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be embodied in any computer-readable storage medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, apparatus, or device and execute instructions), or in conjunction with such instruction execution system, apparatus, or device. For purposes of this specification, "computer-readable storage medium" may be any device that can contain, store, communicate, propagate, or transmit a program for use by an instruction execution system, apparatus, or device, or in conjunction with such instruction execution system, apparatus, or device. More specific examples of computer-readable storage media (a non-exhaustive list) include the following: an electrical connection with one or more wirings (electronic devices), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM, Erasable Programmable Read-Only Memory, or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM, Compact Disc Read-Only Memory). In addition, the computer-readable storage medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.
[0081] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiment, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA, Programmable Gate Array), a field programmable gate array (FPGA, Field Programmable Gate Array), etc.
[0082] In the description of this specification, the description with reference to the terms "this embodiment", "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.
[0083] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0084] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and simple improvements made to the essential contents of the present invention should be included in the protection scope of the present invention.
Claims
1. A target tracking optimization method, characterized in that: include: Performing target recognition processing on the currently received image frame based on the KCF algorithm to obtain sample response values of all position points in the image frame; Screen out the position points whose sample response values are greater than the mean value of the response matrix from all the position points and use them as valid position points; the mean value of the response matrix is the mean value of the sample response values of all the position points; Determining whether occlusion occurs according to the coordinates of each valid position point and the coordinates of the position point with the maximum sample response value, including: using the coordinates of the valid position points and the coordinates of the position point with the maximum sample response value, determining the sum of the distances between each valid position point and the position point with the maximum sample response value, determining that occlusion has occurred according to the sum of the distances being greater than or equal to a first threshold, and determining that occlusion has not occurred according to the sum of the distances being less than the first threshold; According to the occlusion that has occurred, obtaining target position information of a first set number of image frames before the currently received image frame; Determine target offset information based on a preset offset vector and the target position information; The target position information of the currently received image frame is determined according to the target offset information and the target position information of the previous image frame of the currently received image frame.
2. The target tracking optimization method according to claim 1, characterized in that: The determining that occlusion has occurred according to the sum of the distances being greater than or equal to a first threshold comprises: Based on the occlusion that has occurred, comparing the sum of the distances with a preset range interval to obtain a comparison result; The occlusion level of the occlusion that has occurred is determined according to the comparison result.
3. The target tracking optimization method according to claim 1, characterized in that: The determining of the target offset information based on the preset offset vector and the target position information comprises: Determine offset position information of two adjacent image frames using target position information of the first set number of image frames; The target offset information is determined by the preset offset and the offset position information of the two adjacent image frames.
4. The target tracking optimization method according to claim 1, characterized in that: Also includes: Acquire feature information of multiple targets in the currently received image frame, and acquire trajectory information of all targets in a second set number of image frames before the currently received image frame; Determine a correlation result between a target and a trajectory based on the feature information and the trajectory information; The matching result between the target and the track is determined according to the relationship between the correlation result and the second threshold.
5. A target tracking optimization device, characterized in that: include: A target recognition processing module is used to perform target recognition processing on the currently received image frame based on the KCF algorithm to obtain sample response values of all position points in the image frame; An effective position screening module, used to screen out the position points whose sample response values are greater than the mean value of the response matrix from all the position points and use them as effective position points; the mean value of the response matrix is the mean value of the sample response values of all the position points; An occlusion occurrence judgment module is used to determine whether occlusion occurs according to the coordinates of each valid position point and the coordinates of the position point with the maximum sample response value; An occlusion occurrence judgment module is used to determine the sum of distances between each valid position point and the position point with the maximum sample response value by using the coordinates of the valid position points and the coordinates of the position point with the maximum sample response value; The occlusion occurrence judgment module is used to determine that occlusion has occurred according to the sum of the distances being greater than or equal to a first threshold; Or the occlusion occurrence judgment module is used to determine that no occlusion occurs according to the sum of the distances being less than a first threshold; An occlusion target recognition module, used for obtaining target position information of a first set number of image frames before a currently received image frame according to the occlusion that has occurred; The occluded target recognition module is used to determine target offset information based on a preset offset vector and target position information; The occluded target recognition module is used to determine the target position information of the currently received image frame according to the target offset information and the target position information of the previous image frame of the currently received image frame.
6. A computer device, characterized in that: It comprises a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the target tracking optimization method according to any one of claims 1 to 4.
7. A storage medium storing computer-readable instructions, characterized in that: When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the target tracking optimization method according to any one of claims 1 to 4.
8. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor, the steps of the target tracking optimization method according to any one of claims 1 to 4 are performed.
