Target tracking method based on task-oriented target tracking algorithm matching strategy

Abstract

The invention provides a target tracking method based on a task-oriented target tracking algorithm matching strategy, which can be applied to the technical field of artificial intelligence. The target tracking method comprises the steps that an image sequence for a target object is acquired, the image sequence comprises I images, I is an integer larger than 1, and the image sequence comprises a visible light image and an infrared image; performing image detection on each image to generate attribute information of each image; in response to the fact that the difference between the attribute information of the ith image and the attribute information of the (i-1) th image is larger than or equal to a preset threshold value, the attribute information of the ith image is input into a target model, and information of a first target algorithm used for processing the ith image is output; and processing each image based on each target algorithm matched with the attribute information of each image to obtain a tracking result for the target object.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a target tracking method based on a task-oriented target tracking algorithm matching strategy. Background Art

[0002] Single-modal target tracking usually relies on one data source, such as visual images. It is simple to implement and computationally efficient, and is suitable for scenes with relatively stable environmental conditions. However, it may not perform well in situations where the ambient lighting changes, there are occlusions, and there are complex backgrounds. In contrast, multimodal target tracking can significantly improve the accuracy and robustness of tracking by integrating different types of data, especially in complex or dynamic environments. However, it is more complex to implement and requires relatively large computing resources. While improving tracking accuracy, it will increase the computational burden of target tracking, resulting in low tracking performance and efficiency.

[0003] In addition, there are a large number of existing multimodal target tracking algorithms, each with its own advantages and disadvantages. Different algorithms have different advantageous scenarios. In practical applications, there is a lack of algorithm matching strategies for different tracking scenarios and task requirements. Summary of the invention

[0004] In view of the above problems, the present invention provides a target tracking method based on a task-oriented target tracking algorithm matching strategy.

[0005] According to a first aspect of the present invention, there is provided a target tracking method based on a task-oriented target tracking algorithm matching strategy, comprising: acquiring an image sequence for a target object, the image sequence comprising I images, I being an integer greater than 1, and the image sequence comprising visible light images and infrared images; performing image detection on each image respectively to generate attribute information of each image; in response to a difference between the attribute information of the ith image and the attribute information of the i-1th image being greater than or equal to a predetermined threshold, inputting the attribute information of the ith image into a target model, and outputting information of a first target algorithm for processing the ith image; wherein the target model is obtained by training an initial model based on the attribute information of a sample image and algorithm information matching the attribute information of the sample image, 1<i≤I; and processing each image respectively based on each target algorithm matching the attribute information of each image to obtain a tracking result for the target object.

[0006] A second aspect of the present invention provides a target tracking device based on a task-oriented target tracking algorithm matching strategy, comprising: an acquisition module, a detection module, an output module and a processing module.

[0007] An acquisition module is used to acquire an image sequence for a target object, the image sequence includes I images, I is an integer greater than 1, and the image sequence includes visible light images and infrared images. A detection module is used to perform image detection on each image respectively and generate attribute information of each image. An output module is used to input the attribute information of the i-th image into a target model in response to the difference between the attribute information of the i-th image and the attribute information of the i-1-th image being greater than or equal to a predetermined threshold, and output information of a first target algorithm for processing the i-th image; wherein the target model is obtained by training an initial model based on the attribute information of a sample image and algorithm information matching the attribute information of the sample image, 1<i≤I. A processing module is used to process each image respectively based on each target algorithm matching the attribute information of each image to obtain a tracking result for the target object.

[0008] A third aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0009] The fourth aspect of the present invention further provides a computer-readable storage medium on which a computer program or instruction is stored, and the steps of the above method are implemented when the above computer program or instruction is executed by a processor.

[0010] The fifth aspect of the present invention also provides a computer program product, including a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0011] According to the embodiments of the present disclosure, image detection is performed on the images in the acquired image sequence, and attribute information corresponding to each image is generated, and a large amount of image information is converted into attribute information of the image instead of directly inputting the image, thereby reducing the amount of data when processing information and improving the efficiency of calculation. When the difference between the attribute information of the ith image and the attribute information of the i-1th image is greater than or equal to a predetermined threshold, the attribute information of the ith image is input into the target model, and the information of the first target algorithm for processing the ith image is output, and the image is processed according to the matched algorithm. In this process, only the attribute information of the image with large differences is input into the model, and then the target tracking algorithm suitable for the image scene is matched, which can reduce the amount of calculation, reduce the frequency of algorithm replacement, and improve the calculation efficiency. At the same time, by training the target model matching algorithm, the algorithm can be quickly and accurately scheduled according to the scene task requirements of the image, while ensuring the target tracking effect, the calculation efficiency of target tracking is improved and the performance is optimized. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0013] Figure 1 An application scenario diagram of a target tracking method based on a task-oriented target tracking algorithm matching strategy according to an embodiment of the present invention is shown;

[0014] Figure 2 A flow chart of a target tracking method based on a task-oriented target tracking algorithm matching strategy according to an embodiment of the present invention is shown;

[0015] Figure 3 A flow chart of a target tracking method based on a task-oriented target tracking algorithm matching strategy according to another embodiment of the present invention is shown;

[0016] Figure 4 A flow chart of a target model training method according to an embodiment of the present invention is shown;

[0017] Figure 5 A block diagram of an evaluation system for the effects of various target tracking algorithms according to an embodiment of the present invention is shown;

[0018] Figure 6 A schematic diagram showing a target algorithm matching process according to an embodiment of the present invention is shown;

[0019] Figure 7 A schematic diagram showing generation of image attribute information according to an embodiment of the present invention is shown;

[0020] Figure 8 A specific example flow chart of a target tracking method according to an embodiment of the present invention is shown;

[0021] Fig. 9 It shows a structural block diagram of a target tracking device based on a task-oriented target tracking algorithm matching strategy according to an embodiment of the present invention;

[0022] Fig.10 A block diagram of an electronic device suitable for implementing a target tracking method based on a task-oriented target tracking algorithm matching strategy according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0023] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0024] Object tracking is a common computer vision task, which aims to continuously monitor the position of objects in video frames through different approaches. It is widely used in autonomous driving, intelligent monitoring, robotics and other fields.

[0025] Unimodal object tracking usually relies on one data source, such as visual images, which has the advantages of simple implementation and high computational efficiency, and is suitable for scenes with relatively stable environmental conditions. However, it may perform poorly in situations such as lighting changes, occlusions, and complex backgrounds.

[0026] In recent years, in order to cope with many challenges in complex environments, the idea of ​​multimodal fusion has been rapidly developed in the field of target tracking. Multimodal target tracking technology is an important research direction in the field of computer vision. It aims to improve the accuracy and robustness of target tracking by combining data from different sensors or information sources. For example, multimodal tracking integrates multiple information including visual, infrared, lidar and acoustic data. Through the information complementation strategy of multiple modal data, more effective target tracking effects can be achieved in complex scenes. For example, the RGBT (Red-Green-Blue Thermal) target tracking algorithm using visible light images and infrared images takes into account the texture information and thermal information of the tracked target at the same time, and can achieve high-precision target tracking in complex or dynamic environments, improving the accuracy and robustness of tracking. However, when applying the multimodal information fusion strategy, the implementation complexity is high and the computing resource requirements are relatively large. While improving the tracking accuracy, it will increase the computing burden of target tracking, and the tracking performance and efficiency are low.

[0027] In addition, there are a large number of existing multimodal target tracking algorithms, each with its own advantages and disadvantages. Different algorithms have different advantageous scenarios. In practical applications, facing different tracking scenarios and task requirements, there is a lack of algorithm matching strategies for different tracking scenarios to adaptively perform tracking tasks.

[0028] In view of this, an embodiment of the present invention provides a target tracking method based on a task-oriented target tracking algorithm matching strategy, performs image detection on images in an acquired image sequence, generates attribute information corresponding to each image, converts a large amount of image information into attribute information of the image, rather than directly inputting the image, thereby reducing the amount of data when processing information and improving the efficiency of calculation. When the difference between the attribute information of the i-th image and the attribute information of the i-1-th image is greater than or equal to a predetermined threshold, the attribute information of the i-th image is input into the target model, the information of the first target algorithm for processing the i-th image is output, and the image is processed according to the matched algorithm. In this process, only the attribute information of the image with a large difference is input into the model, and then the target tracking algorithm suitable for the image scene is matched, which can reduce the amount of calculation, reduce the frequency of changing the algorithm, and improve the calculation efficiency. At the same time, by training the target model matching algorithm, the algorithm can be quickly and accurately scheduled according to the scene task requirements of the image, while ensuring the target tracking effect, the calculation efficiency of the target tracking is improved and the performance is optimized.

[0029] Figure 1 An application scenario diagram of a target tracking method based on a task-oriented target tracking algorithm matching strategy according to an embodiment of the present invention is shown.

[0030] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0031] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0032] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0033] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0034] It should be noted that the target tracking method based on the task-oriented target tracking algorithm matching strategy provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the target tracking device based on the task-oriented target tracking algorithm matching strategy provided in the embodiment of the present invention can generally be set in the server 105. The target tracking method based on the task-oriented target tracking algorithm matching strategy provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the target tracking device based on the task-oriented target tracking algorithm matching strategy provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0035] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0036] The following will be based on Figure 1 The scene described by Figure 2~Figure 8 The target tracking method based on the task-oriented target tracking algorithm matching strategy of the embodiment of the invention is described in detail.

[0037] Figure 2 A flow chart of a target tracking method based on a task-oriented target tracking algorithm matching strategy according to an embodiment of the present invention is shown.

[0038] like Figure 2 As shown, the target tracking method 200 of this embodiment includes operations S210 to S240.

[0039] In operation S210 , an image sequence for a target object is acquired.

[0040] In operation S220, image detection is performed on each image to generate attribute information of each image.

[0041] In operation S230, in response to the difference between the attribute information of the i-th image and the attribute information of the i-1-th image being greater than or equal to a predetermined threshold, the attribute information of the i-th image is input into the target model, and information of the first target algorithm for processing the i-th image is output.

[0042] In operation S240, each image is processed based on each target algorithm matched with the attribute information of each image to obtain a tracking result for the target object.

[0043] According to an embodiment of the present invention, the target object is an object with tracking requirements, for example, the target object is an aerial tracking target in a complex environment. The image sequence includes I images, where I is an integer greater than 1, and the image sequence includes visible light images and infrared images, which can be obtained by a binocular camera and are used to simultaneously obtain texture information and thermal information of the same target object.

[0044] According to an embodiment of the present invention, for each image in the above image sequence, before image detection, the image may be preprocessed, specifically, including operations such as image alignment, time synchronization, and image denoising.

[0045] Image detection includes the detection of image brightness, contrast, clarity, radiance, thermal noise, thermal contrast, etc., that is, the evaluation of the state or quality of the image, and then the generation of attribute information of each image. Attribute information includes a set of brightness standard deviation, contrast standard deviation, and clarity variance obtained after the image is processed by calculation. Brightness, contrast and clarity can be used as indicators for evaluating the state or quality of the image, which form a corresponding indicator set. The above process can simplify the image information containing a large amount of data into attribute information (that is, simplified into an indicator set) and then input it into the target model, rather than directly inputting the image, which can reduce the pressure of network calculation.

[0046] For example, the tth image is an image of a target object entering a bush captured in the dark. The image is detected and the brightness standard deviation, contrast standard deviation and clarity variance corresponding to the image are generated, which respectively reflect the brightness, contrast and clarity of the image. The set is generated. .

[0047] According to an embodiment of the present invention, when the difference between the attribute information of the i-th image and the attribute information of the i-1-th image is greater than or equal to a predetermined threshold, the attribute information of the i-th image is input into the target model, and information of the first target algorithm for processing the i-th image is output, 1<i≤I.

[0048] The predetermined threshold value may be 10%, 20%, 30%, etc., and is not limited here. The first target algorithm may be an RGB (Red-Green-Blue, visible light) image tracking algorithm, a TIR (Thermal Infrared, thermal infrared) image tracking algorithm, or an RGBT image tracking algorithm, which is used to perform the target tracking task. The information of the first target algorithm includes the name of the algorithm.

[0049] The difference between the attribute information of the i-th image and the attribute information of the i-1-th image can be obtained by calculating the difference of the brightness standard deviation, contrast standard deviation, and clarity variance in the attribute information of the two images, and averaging all the differences. It can also be obtained by weighted summing all the differences, which is not limited here.

[0050] The above difference is compared with the predetermined threshold. When the difference is greater than or equal to the predetermined threshold, it means that the change difference between the two images is generally large and the scene change is obvious. It is necessary to input the model to match the target tracking algorithm corresponding to the image attribute information to select an algorithm that is more suitable for the current image task scenario to process the image and obtain the tracking result for the image.

[0051] For example, the preset threshold is 20%, and the attribute information of the 5th image is , the attribute information of the 6th image is , the difference between the calculated attribute information is The average of the above difference values ​​is 68%, which is greater than the predetermined threshold. Therefore, the attribute information of the sixth image is The target model is input, and information of the first target algorithm output for processing the sixth image is the RGBT image tracking algorithm. The sixth image is processed using the RGBT image tracking algorithm to obtain a tracking result for the image.

[0052] The target model is obtained by training the initial model based on the attribute information of the sample image and the algorithm information matching the attribute information of the sample image. The target model can be a neural network model, in which the input is the attribute information of the sample image and the output is the name of the optimal target tracking algorithm.

[0053] According to an embodiment of the present invention, image detection is performed on the images in the acquired image sequence, and attribute information corresponding to each image is generated, and a large amount of image information is converted into attribute information of the image instead of directly inputting the image, thereby reducing the amount of data when processing information and improving the efficiency of calculation. When the difference between the attribute information of the i-th image and the attribute information of the i-1-th image is greater than or equal to a predetermined threshold, the attribute information of the i-th image is input into the target model, and the information of the first target algorithm for processing the i-th image is output, and the image is processed according to the matched algorithm. In this process, only the attribute information of the image with a large difference is input into the model, and then the target tracking algorithm suitable for the image scene is matched, which can reduce the amount of calculation, reduce the frequency of changing the algorithm, and improve the calculation efficiency. At the same time, by training the target model matching algorithm, the algorithm can be quickly and accurately scheduled according to the scene task requirements of the image, while ensuring the target tracking effect, the calculation efficiency of the target tracking is improved and the performance is optimized.

[0054] Figure 3 A flow chart of a target tracking method based on a task-oriented target tracking algorithm matching strategy according to another embodiment of the present invention is shown.

[0055] like Figure 3 As shown, the target tracking method 300 of this embodiment includes operations S310 to S360.

[0056] In operation S310 , an image sequence for a target object is acquired.

[0057] In operation S320, image detection is performed on each image to generate attribute information of each image.

[0058] In operation S330, it is determined whether the difference between the attribute information of the i-th image and the attribute information of the i-1-th image is less than a predetermined threshold. If the determination result is yes, operation S340 is performed, and if the determination result is no, operation S350 is performed.

[0059] In operation S340, algorithm information matching the attribute information of the (i-1)th image is determined as second target algorithm information of the i-th image.

[0060] In operation S350, attribute information of the i-th image is input into the target model, and information of a first target algorithm for processing the i-th image is output.

[0061] In operation S360, each image is processed based on each target algorithm matched with the attribute information of each image to obtain a tracking result for the target object.

[0062] According to an embodiment of the present invention, the second target algorithm may be an RGB image tracking algorithm, a TIR image tracking algorithm, or an RGBT image tracking algorithm. The information of the second target algorithm includes the name of the algorithm.

[0063] In the target model application stage, it is unreasonable to input the image attribute information of each image into the target model after image detection and attribute information generation. On the one hand, the neural network needs to calculate each image, which will have a high computational cost and affect the real-time performance; on the other hand, changing the algorithm too frequently in the same image sequence will also have a high computational burden.

[0064] For similar task scenes, the difference between adjacent images in the image sequence is small. If each image is input into the target model for algorithm matching, it will cause too much calculation, and frequent algorithm changes will also affect the calculation efficiency of the target model. Therefore, the present invention compares the difference in attribute information between two adjacent images, and only when the attribute information of adjacent images changes greatly, the target model is re-input to match the target tracking algorithm.

[0065] For example, the preset threshold is 20%, and the attribute information of the 5th image is , the attribute information of the 6th image is , the difference between the calculated attributes is , the average of the above difference values ​​is 13%, which is less than the predetermined threshold value, so the algorithm "TIR image tracking algorithm" matching the attribute information of the 5th image is determined as the second target algorithm for the 6th image. The 6th image is processed based on the TIR image tracking algorithm matching the attribute information of the 6th image to obtain the tracking result for the 6th image.

[0066] According to an embodiment of the present invention, by calculating the difference in attribute information of adjacent images and comparing the difference with a predetermined threshold, it is determined whether the image needs to be input into a target model for algorithm matching. When the difference is less than the predetermined threshold, it indicates that the scene change is small at this time, and the algorithm is not switched. The algorithm of the previous image is used as the target algorithm, thereby quickly obtaining algorithm information that matches the attribute information of the image. There is no need to input the attribute information of each image into the target model, which can effectively reduce the computing pressure and the consumption of computing resources, and improve the efficiency of target algorithm matching.

[0067] Figure 4 A flow chart of a target model training method according to an embodiment of the present invention is shown.

[0068] like Figure 4 As shown, the method 400 of this embodiment includes operations S431 to S436.

[0069] In operation S431, a plurality of sample images and attribute information corresponding to the plurality of sample images are acquired.

[0070] In operation S432, for each sample image, multiple algorithms are called to process each sample image to obtain multiple tracking results for each sample image.

[0071] In operation S433, a plurality of effect information of each algorithm processing each sample image is determined according to the plurality of tracking results and the operation information of the plurality of algorithms.

[0072] In operation S434, a plurality of algorithm evaluation information of each sample image is determined based on the plurality of effect information.

[0073] In operation S435, algorithm information matching the attribute information of each sample image is determined from the plurality of algorithm evaluation information.

[0074] In operation S436, the initial model is trained using the sample data set to obtain a target model.

[0075] According to an embodiment of the present invention, the plurality of algorithms include a plurality of target tracking algorithms in a target tracking algorithm library. The tracking result includes the coincidence between the tracked target and the real target after executing the target tracking algorithm, the tracking success rate and other results. The operation information of the algorithm includes the tracking frame rate, the model parameter quantity, the video memory occupancy and other operation information.

[0076] There are many types and numbers of existing target tracking algorithms. How to scientifically set up the tracking effect evaluation method, effectively measure the effectiveness and reliability of the tracking system, and then select the most suitable target tracking algorithm according to the environment and target state is of great significance for the realization of tracking tasks. Usually, the evaluation of algorithm effect is divided into multiple dimensions, including accuracy, robustness, and timeliness. Accuracy is usually evaluated by calculating the error between the tracking result and the true target position, including intersection over union and average precision. Robustness refers to the stability of the algorithm under different environmental conditions (such as lighting changes, occlusion, and motion blur), usually evaluated on multiple scenes or data sets. Timeliness refers to the speed at which the algorithm processes frames, usually measured in frames per second (FPS).

[0077] Figure 5 A block diagram of an evaluation system for the effects of various target tracking algorithms according to an embodiment of the present invention is shown.

[0078] In the design of each target tracking algorithm effect evaluation system, it is considered that in some complex scenarios, not only the accuracy of the target tracking algorithm is required to meet the requirements, but also the algorithm must be able to adapt to the rapid changes of the scene, the rapid movement of the target, different degrees of occlusion and other problems. At the same time, there are certain requirements for the execution speed of the algorithm. Therefore, how to reasonably combine these evaluation indicators to design a comprehensive target tracking algorithm effect evaluation system that is suitable for different application scenarios and task requirements is an urgent need in actual application scenarios. Based on the above problems, the present invention proposes an evaluation system for the tracking effects of multiple target tracking algorithms.

[0079] like Figure 5 As shown, in the embodiment of the present invention, the first-level evaluation indicators include accuracy, timeliness and robustness, and the second-level evaluation indicators include intersection-over-union ratio and average precision, tracking frame rate, model parameter quantity and video memory occupancy, time robustness, space robustness and long-term tracking capability. The above-mentioned first-level evaluation indicators and second-level evaluation indicators together constitute the evaluation system of the effects of each target tracking algorithm.

[0080] According to an embodiment of the present invention, the effect information includes at least one of the following: accuracy information, timeliness information and robustness information, wherein the accuracy information, timeliness information and robustness information include the calculated index values ​​of the three indicators of accuracy, timeliness and robustness, which are used to evaluate the tracking effect of the algorithm. The above effect information is specifically described below:

[0081] Accurate information includes intersection of union (IOU) and average precision (AP). It is the key to evaluate whether the algorithm can accurately track the target and whether the algorithm works. Among them, IOU measures the overlap of the tracked target, which can be determined by the overlap between the tracked target and the real target after executing the target tracking algorithm. Average precision measures the Euclidean distance between the target center points. The index value corresponding to the accuracy can be determined by the weighted sum of IOU and AP, that is, the accurate information can be determined, as shown in the following formula (1):

[0082] (1)

[0083] in, The index value representing the accuracy of the algorithm, represents the intersection-over-union ratio of the algorithm, It represents the average precision of the algorithm, a represents the weight of the intersection-over-union ratio, and b represents the weight of the average precision.

[0084] For example, for image a, call algorithms x and y. According to the tracking results and running information of algorithms x and y, the IOU of algorithm x is determined to be 0.8, the AP is 0.9, the IOU weight is 1.2, and the AP weight is 0.9. Therefore, the accuracy index value of algorithm x is 1.77. Similarly, the IOU of algorithm y is determined to be 0.9, the AP is 0.95, the IOU weight is 1.2, and the AP weight is 0.9. Therefore, the accuracy index value of algorithm y is 1.94.

[0085] The timeliness information includes the tracking frame rate (Frame Per Second, FPS), model parameters (Model Parameters, MP) and memory usage (Memory Usage, MU) during algorithm operation, which evaluates the algorithm's task execution speed and the use of computing resources, and reflects the algorithm's real-time performance. The timeliness index value can be determined by tracking the frame rate, model parameters and memory usage, that is, the timeliness information can be determined, as shown in the following formula (2):

[0086] (2)

[0087] in, The index value representing the timeliness of the algorithm, represents the tracking frame rate of the algorithm, represents the model parameter quantity of the algorithm, represents the memory usage of the algorithm, c represents the weight of the tracking frame rate, d represents the weight of the model parameters, and e represents the weight of the memory usage.

[0088] For example, for image a, call algorithms x and y. According to the tracking results and running information of algorithms x and y, determine that the FPS of algorithm x is 30, the MP is 100, the MU is 1024, the weight of FPS is 0.7, the weight of MP is 0.8, and the weight of MU is 0.75. Thus, the timeliness index value of algorithm x is 869. Similarly, determine that the FPS of algorithm y is 25, the MP is 200, the MU is 2048, the weight of FPS is 0.7, the weight of MP is 0.8, and the weight of MU is 0.75. Thus, the timeliness index value of algorithm y is 1713.

[0089] Robustness information includes the algorithm's temporal robustness evaluation (TRE), spatial robustness evaluation (SRE), and long-term tracking capability (LTC), which evaluate the algorithm's stability in abnormal situations and whether it can continue to normally complete the target tracking task in disturbances. Among them, temporal robustness can be determined by calculating the target tracking success rate in continuous time, spatial robustness can be determined by calculating the target tracking success rate at different spatial positions (such as the center and edge of the image), and long-term tracking capability can be determined by calculating the target loss rate in long-term tracking. The robustness index value can be determined by weighted summation of temporal robustness evaluation, spatial robustness evaluation, and long-term tracking capability, that is, the robustness information is determined, as shown in the following formula (3):

[0090] (3)

[0091] in, The index value representing the robustness of the algorithm, represents the time robustness of the algorithm, represents the spatial robustness of the algorithm, represents the long-term tracking ability of the algorithm, f represents the weight of temporal robustness, g represents the weight of spatial robustness, and h represents the weight of long-term tracking ability.

[0092] For example, for image a, call algorithms x and y. According to the tracking results and running information of algorithms x and y, determine that the TRE of algorithm x is 0.8, the SRE is 0.85, the LTC is 0.1, the weight of TRE is 0.6, the weight of SRE is 0.7, and the weight of LTC is 0.75. Thus, the robustness index value of algorithm x is 1.15. Similarly, determine that the TRE of algorithm y is 0.9, the SRE is 0.75, and the LTC is 0.2. The weight of TRE is 0.6, the weight of SRE is 0.7, and the weight of LTC is 0.75. Thus, the robustness index value of algorithm y is 1.21.

[0093] According to an embodiment of the present invention, the algorithm evaluation information includes a comprehensive score of the algorithm. Based on the multiple effect information, the multiple algorithm evaluation information of each sample image is determined by weighted summing the calculated index values ​​of algorithm accuracy, timeliness and robustness, as shown in the following formula (4):

[0094] (4)

[0095] in, represents the comprehensive score of the algorithm, The index value representing the accuracy of the algorithm, The index value representing the timeliness of the algorithm, It represents the index value of the robustness of the algorithm. A represents the weight of accuracy, B represents the weight of timeliness, and C represents the weight of robustness.

[0096] For example, for image a, algorithms x and y are called, and the accuracy, timeliness, and robustness index values ​​for algorithm x are 1.77, 869, and 1.15, respectively. The accuracy, timeliness, and robustness index values ​​for algorithm y are 1.94, 1713, and 1.21, respectively. The weights of accuracy, timeliness, and robustness are 0.9, 0.8, and 0.7, respectively. The comprehensive scores of algorithms x and y are 697 and 1372, respectively.

[0097] According to an embodiment of the present invention, algorithm information matching the attribute information of each sample image can be determined from multiple algorithm evaluation information by sorting the comprehensive scores of the multiple algorithms and determining the algorithm with the smallest sort value as the sample target algorithm.

[0098] For example: for sample image a, the comprehensive score ranking of algorithms x and y is obtained, the ranking value of algorithm x is 1, and the ranking value of algorithm y is 2, then algorithm x is determined as the sample target algorithm that matches the attribute information of sample image a, and the algorithm information of algorithm x is the algorithm information that matches the attribute information of sample image a.

[0099] According to an embodiment of the present invention, a sample data set is used to train an initial model to obtain a target model, wherein the sample data set includes multiple groups of sample data, and each group of sample data includes attribute information of a sample image and algorithm information matching the sample image.

[0100] Figure 6 A schematic diagram of performing target algorithm matching using a sample data set according to an embodiment of the present invention is shown.

[0101] like Figure 6 As shown, a sample image sequence is obtained, including visible light images and infrared images, and data preprocessing is performed on the images, such as image alignment, time synchronization, contrast adjustment, image enhancement, etc. Image detection is performed on the preprocessed images, such as brightness analysis, contrast analysis, and clarity analysis on visible light images, to obtain the attribute information of the image, that is, the target state evaluation index set of the image. It is input into the decision system and algorithm switching module in the target model to obtain the comprehensive scores of multiple target tracking algorithms (target tracking algorithms 1~q) in the target tracking algorithm library, and then the best target tracking algorithm is determined to perform the tracking task. Figure 6The decision system and algorithm switching module shown in are schematic structures of the neural network model.

[0102] Regarding the target model and training, it is worth noting that the target model does not affect the various existing target tracking algorithms. The purpose is to make a comprehensive judgment on the various existing single-modal / multi-modal target tracking algorithms, and select the most suitable algorithm to perform the target tracking task in different scenarios and under different task requirements, ensuring that only one algorithm is selected to perform the target tracking task each time.

[0103] Specifically, the input and output of the target model are designed as follows: first, obtain the state evaluation index set of each image, that is, the attribute information of each image. At this time, each image will be converted into a small index set, and then form a large index set containing all images in the image sequence; secondly, according to the various existing target tracking algorithms, call each algorithm to calculate the tracking effect in the current scene, and based on these effect information and algorithm comprehensive evaluation methods, calculate the comprehensive score of each algorithm as the module output. The state evaluation index of each image corresponds to the comprehensive score of each algorithm under the image, forming an input-output pair. Repeat the above operation for each image sequence to obtain the input and output of the entire data set as the input and output training set of the target model. The sample data sets are shown below:

[0104]

[0105]

[0106]

[0107]

[0108]

[0109] in, Represents the attribute information of the input image in a certain scene, including the attribute information of each image. There are m images in the scene; Represents the attribute information of the t-th input image, including the attribute information of the visible light image and infrared image of the image. Indicates the first attribute information, and so on, each image has a total of n attribute information. Indicates the selected target tracking algorithm library, Represents the first algorithm, and so on, there are a total of q target tracking algorithms to be selected. Represents the scores of each algorithm in the tracking algorithm library, including the scores of each algorithm under each image; represents the scores of each algorithm for the t-th input image, It represents the tracking effect score of the first algorithm for the tth input image. There are q target tracking algorithms to be selected, so each image has a total of q scores.

[0110] Essentially, the above method calculates the tracking results of each tracking algorithm in advance, and a large number of calculation processes are used to generate training data sets, and the target model is obtained after the neural network is trained. When using this model, it is not necessary to calculate each tracking algorithm, and the best algorithm can be selected to perform the tracking task, which saves calculation time and can effectively improve the accuracy of target tracking.

[0111] According to an embodiment of the present invention, for each sample image, multiple algorithms are called to process each sample image, and multiple tracking results for each sample image are obtained, and then the effect information of multiple algorithms is determined. Through the effect information, the evaluation information of the algorithm is obtained, and then the algorithm information matching each sample image is determined. The above information constitutes a sample data set, and a matching relationship between the attribute information of the sample image and the sample target algorithm is established for the training of the target model. In the application of the target model, by inputting the attribute information of the image into the trained target model, the algorithm information matching the attribute information of the image can be output, so as to realize the rapid matching of the best algorithm for different image task scenarios and improve the efficiency of algorithm matching.

[0112] According to an embodiment of the present invention, based on multiple effect information, multiple algorithm evaluation information of each sample image is determined, including: using hierarchical analysis method to process multiple effect information to obtain first weights of each effect information for each sample image; using objective weighting method to process multiple effect information to obtain second weights of each effect information for each sample image; based on the first weights and second weights of multiple effect information, determining target weights of each effect information; and obtaining multiple algorithm evaluation information according to the target weights of each effect information.

[0113] According to an embodiment of the present invention, a method combining the subjective weighting method and the objective weighting method of the hierarchical analysis method is used to reasonably weight the indicators in the above effect information. The specific process is as follows:

[0114] Use the analytic hierarchy process for subjective empowerment:

[0115] First, the importance of indicators of each algorithm is compared to obtain the judgment matrix.

[0116] Table 1 is a table of relative scales of importance of algorithm effect information.

[0117] Table 1

[0118]

[0119] For example, according to expert experience, accuracy is slightly more important than timeliness, accuracy is significantly more important than robustness, and timeliness is slightly more important than robustness. According to the relative scale in Table 1, the judgment matrix A is constructed as shown in the following formula (5):

[0120] (5)

[0121] Calculate the maximum eigenvalue of the judgment matrix , and the corresponding maximum eigenvector , for example, the calculated maximum eigenvalue is m, and the maximum eigenvector is .

[0122] Through consistency testing, subjective random errors are reduced. Specifically, the consistency index CI is first calculated, and the calculation formula is shown in the following formula (6):

[0123] (6)

[0124] in, represents the maximum eigenvalue of the judgment matrix, N represents the dimension of the judgment matrix, and CI represents the consistency index.

[0125] Determine the relationship between the random index RI and the consistency index CI, if , then the subjectively constructed feature vector is reasonable, otherwise the judgment matrix needs to be reconstructed and the consistency check needs to be performed again until the calculated weight vector is reasonable. The value of the random index RI is related to the dimension of the matrix, as shown in Table 2.

[0126] Table 2 shows the random index RI of the n-order matrix.

[0127] Table 2

[0128]

[0129] For example, for the three dimensions of accuracy, robustness and timeliness, N=3, CI=0.03, query Table 2 to get RI=0.58, and get , it means that the subjectively constructed feature vector is reasonable, and the weights of accuracy, robustness and timeliness can be determined based on the feature vector.

[0130] Objective weighting is performed using the CRITIC (Criteria Importance Through Intercrieria Correlation) objective weighting method: Comparative strength refers to the size of the difference between the values ​​of the various evaluation schemes for the same indicator, expressed in the form of standard deviation. The larger the standard deviation, the greater the fluctuation, that is, the greater the difference in values ​​between the schemes, the higher the weight. The conflict between indicators is expressed by the correlation coefficient. If there is a strong positive correlation between two indicators, it means that the conflict is smaller and the weight is lower.

[0131] First, in order to eliminate the influence of different dimensions on the evaluation results, each indicator needs to be dimensionless. The present invention adopts positive indicators, and the larger the value of the indicator, the better. The dimensionless processing formula is shown in the following formula (7):

[0132] (7)

[0133] in, represents the test data value of the i-th indicator under the j-th algorithm, represents the dimensionless processed data, and It represents the maximum and minimum values ​​of the i-th indicator in the test data under p algorithms.

[0134] Calculate the variability of indicators: By using the standard deviation to measure the difference fluctuation of the test data of the algorithm under different indicators, it can reflect the performance difference information of different algorithms. The calculation formula is shown in the following formula (8) and formula (9):

[0135] (8)

[0136] (9)

[0137] in, represents the mean of the test data in p algorithms under the i-th indicator, Represents standard deviation.

[0138] Calculation of index conflict: The correlation coefficient is used to measure the correlation of statistical data under different evaluation indicators of the same algorithm model, reflecting the overlap of algorithm performance information. The calculation formula is shown in the following formula (10) and formula (11):

[0139] (10)

[0140] (11)

[0141] in, represents the test data value of the jth algorithm under the x index, represents the test data value of the jth algorithm under the y index, and They represent the mean of the test data of the algorithm under two indicators, represents the correlation coefficient between the evaluation indicators. Indicates the index conflict of evaluation index x, represents the correlation coefficient between indicator x and indicator i, and n represents the number of other indicators.

[0142] Calculate the objective weight: Combined with the conflict and variability of the indicator evaluation, the objective weight value of the evaluation indicator is calculated. The calculation formula is shown in the following formula (12):

[0143] (12)

[0144] in, Indicates the amount of information. The larger the value, the greater its role in the entire evaluation index system, and the more weight should be assigned to it. i Represents objective weight.

[0145] The subjective and objective weighting methods are complemented to minimize the difference between the final weight vector and various weighting methods, and to maximize the advantages of each method. Specifically, the final target weight is calculated by the following formula (13) and formula (14):

[0146] (13)

[0147] (14)

[0148] in, and Represent the weight vectors of subjective weighting method and objective weighting method respectively, and Indicates its weight ratio, represents the final weight vector. Solving formula (13) yields and The value of , and finally the comprehensive weight vector is obtained.

[0149] It should be noted that in actual applications, if the task scenario does not consider a certain indicator, the weight of the indicator can be set to 0; if the task scenario is special and the number of evaluation indicators or subjective and objective evaluation methods needs to be increased, additional evaluation indicators can also be added.

[0150] According to an embodiment of the present invention, by combining subjective weighting with objective weighting, considering the inherent statistical laws and authoritative values ​​between indicator data, multiple effect information are weighted to obtain the target weight of each effect information. This can not only reflect the objective laws of each effect information based on task requirements and environmental characteristics and reduce subjective influence, but also comply with certain value concepts of decision makers, thereby evaluating the algorithm more scientifically and making up for the shortcomings of single weighting.

[0151] According to an embodiment of the present invention, performing image detection on each image respectively to generate attribute information of each image includes: determining a detection strategy according to the type of each image; and detecting the image based on the detection strategy to obtain attribute information of each image.

[0152] According to an embodiment of the present invention, a detection strategy is determined according to the type of each image, including: in response to the type of the image being a visible light image, determining the detection strategy to perform visible light image detection on the image; in response to the type of the image being an infrared image, determining the detection strategy to perform infrared image detection on the image.

[0153] According to an embodiment of the present invention, a corresponding detection strategy is determined according to different types of images, and based on the detection strategy, a targeted detection is performed on the image, which can reduce the detection error caused by different image types, more accurately obtain the attribute information of the image, and improve the accuracy of image detection.

[0154] According to an embodiment of the present invention, the attribute information of each image includes at least one of the following: brightness information, contrast information and clarity information; based on the detection strategy, the image is detected to obtain the attribute information of each image including: grayscale detection is performed on the image to obtain the first grayscale information of each image; by processing the first grayscale information, the brightness information of each image is obtained; contrast enhancement processing is performed on each image; grayscale detection is performed on the enhanced image to obtain the second grayscale information of each image; by processing the second grayscale information, the contrast information of each image is obtained; Laplace transform processing is performed on each image; pixel detection is performed on the transformed image to obtain the pixel information of each image; and the clarity information of each image is determined by processing the pixel information.

[0155] Figure 7 A schematic diagram of generating image attribute information according to an embodiment of the present invention is shown.

[0156] The mean brightness refers to the average brightness value of all pixels in the image. For visible light images, the mean brightness usually depends on the exposure settings and lighting conditions of the image. In general, the mean brightness should be within a reasonable range to ensure that the image is not too dark or too bright. The standard deviation indicates the distribution of image brightness, that is, the range of brightness variation. A higher standard deviation means a larger brightness variation in the image, and a lower standard deviation means a smaller brightness variation. For target detection tasks, the standard deviation usually needs to be large enough to ensure that the details and target features in the image can be effectively distinguished. In general, the standard deviation should be kept in a moderate range, usually between 40 and 100.

[0157] like Figure 7 As shown, grayscale detection is performed on image 720 to obtain first grayscale information 720-1 of image 720, and brightness information 720-2 of image 720 is obtained by processing the first grayscale information 720-1. The first grayscale information includes the grayscale values ​​of pixels in the image, and the brightness information includes the mean value of the image brightness and the standard deviation of the image brightness.

[0158] The image brightness mean calculation formula is shown in the following formula (15):

[0159] (15)

[0160] in, represents the mean value of image brightness, It is The gray value of a pixel, is the total number of pixels in the image.

[0161] The formula for the standard deviation of image brightness is shown in the following formula (16):

[0162] (16)

[0163] in, represents the standard deviation of image brightness, represents the mean value of image brightness, It is The gray value of a pixel, is the total number of pixels in the image.

[0164] Contrast refers to the difference between the lightest and darkest areas of an image. High contrast helps to clearly distinguish the object from the background, while low contrast images may cause the object to be blurred.

[0165] like Figure 7As shown, contrast enhancement processing is performed on image 720; grayscale detection is performed on enhanced image 721 to obtain second grayscale information 721-1 of image 720; by processing second grayscale information 721-1, contrast information 721-2 of image 720 is obtained. The second grayscale information includes the grayscale value and grayscale mean of each pixel of the image after contrast enhancement processing. Contrast information includes contrast standard deviation.

[0166] In order to better preserve image details and avoid the increase in noise contrast that may be caused by global histogram equalization, CLAHE (Contrast Limited Adaptive Histogram Equalization) was selected as the contrast enhancement algorithm, and the enhanced contrast value was compared with the predefined threshold to evaluate whether the contrast was sufficient.

[0167] Divide the input image into small blocks, and for each block, calculate its grayscale histogram .

[0168] The cumulative distribution function is calculated using the following formula (17):

[0169] (17)

[0170] The equalization mapping normalizes the CDF (Cumulative Distribution Function) to the target range, usually 0 to L-1. The calculation formula of the pixel value is shown in the following formula (18):

[0171] (18)

[0172] in, Represents the grayscale histogram function of each block of the image, N represents the number of small blocks after the image is divided, represents the cumulative distribution function, L represents the gray level, is the minimum CDF value in the local area, Represents the pixel value function.

[0173] To avoid over-enhancing the contrast, CLAHE limits the histogram by clipping it and redistributing the clipping values. After applying histogram equalization to each block, the block boundaries are smoothed by bilinear interpolation to reduce the visual boundaries between blocks. This method enhances image details through local processing while limiting excessive contrast enhancement to avoid amplifying image noise and solve the problem of images with low contrast or non-uniform illumination.

[0174] After adaptive histogram equalization, the pixel value matrix of the image , the global contrast can be expressed as the contrast standard deviation The calculation formula of contrast standard deviation is shown in the following formula (19):

[0175] (19)

[0176] in It is the first The gray value of a pixel, is the mean grayscale value of the processed pixels, represents the contrast standard deviation, and N represents the number of small blocks after the image is divided.

[0177] like Figure 7 As shown, Laplace transform processing is performed on image 720; pixel detection is performed on processed image 722 to obtain pixel information 722-1 of image 720; by processing pixel information 722-1, clarity information 722-2 of image 720 is determined. The pixel information includes the pixel value and pixel mean after Laplace transform processing, and the clarity information includes the clarity variance.

[0178] The Laplace operator is used to calculate the image clarity based on the edge and detail features of the image, describing the high-frequency information of the image. First, the image is converted into a grayscale image to simplify the calculation. Then the grayscale image is convolved with the selected Laplace convolution kernel to obtain the Laplace transformed image. The Laplace operator can be approximated by the convolution kernel shown in the following equation (20):

[0179] (20)

[0180] These kernels will be applied to each pixel of the image, and the Laplace transform of the image will be calculated through convolution operation. The Laplace transformed image represents the high-frequency components of the image, and its variance can reflect the edge and detail features of the image. The image clarity is judged based on the calculated variance value. A higher variance value indicates a clear image, and a lower variance value indicates a blurred image. The calculation formula for the clarity variance is as follows:

[0181] (twenty one)

[0182] in represents the Laplace transformed The pixel value of pixels, represents the mean value of these pixels, is the total number of pixels, Represents the clarity variance.

[0183] According to an embodiment of the present invention, by detecting an image, brightness information, contrast information and clarity information of the image are obtained, wherein the brightness information can be obtained through the first grayscale information, the contrast information can be obtained through the second grayscale information after the image is subjected to contrast enhancement processing, and the clarity information can be obtained through the pixel information after the image is subjected to Laplace transform processing, thereby realizing the conversion of a large amount of image information into attribute information of the image instead of directly inputting the image, reducing the amount of data when processing information and improving the efficiency of calculation.

[0184] The technical solution of the present invention is further explained below in conjunction with specific embodiments. It should be noted that the specific embodiments are only for the purpose of facilitating those skilled in the art to better understand the technical solution of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0185] Figure 8 A specific example flow chart of a target tracking method according to an embodiment of the present invention is shown.

[0186] like Figure 8 As shown, for image sequence 1, data preprocessing is performed on the visible light and thermal infrared images of the sequence, and then the quality detection of the visible light image and the infrared image is performed respectively, and the environment and target state evaluation indicators of the visible light image and the infrared image are obtained respectively, and they are input into the target model. The target model will output the name of the corresponding target tracking algorithm, such as RGB image tracking algorithm, TIR image tracking algorithm and RGBT image tracking algorithm. Then, the tracking task of the image in image sequence 1 is performed according to the algorithm. Secondly, in subsequent images, the above operations are repeated until the tracking task of the tracking scene is completed.

[0187] According to an embodiment of the present invention, a target model for target tracking algorithms of different modalities is first proposed, and the input and output, training process and application strategy of the model are designed to realize the target tracking algorithm based on the model to quickly select the scene characteristics and task requirements, which can play a role in different environments and task conditions and has strong adaptability. In addition, the method can only support the expansion and replacement of algorithms in the algorithm library. After the algorithm is replaced, the target model can be retrained after the input index set of the image is appropriately adjusted, and the extensibility is strong. Secondly, an extensible input image target state evaluation index set is proposed, which is oriented to the field of target tracking of different modalities, contains the basic features of the input image data, and is used as the input of the target model, which greatly reduces the model calculation pressure and supports the customized design requirements of special task scenarios. In addition, based on the three dimensions of accuracy, timeliness and robustness of the algorithm, the present invention proposes a comprehensive target tracking algorithm performance index evaluation system, which can be applied in different application scenarios and task requirements, reflects the comprehensive scores of different algorithms, and combines the target model to quickly select the optimal algorithm to perform the task, with strong balance.

[0188] Based on the target tracking method based on the task-oriented target tracking algorithm matching strategy, the present invention also provides a target tracking device based on the task-oriented target tracking algorithm matching strategy. Fig. 9 The device is described in detail.

[0189] Fig. 9 A structural block diagram of a target tracking device based on a task-oriented target tracking algorithm matching strategy according to an embodiment of the present invention is shown.

[0190] like Fig. 9 As shown, the target tracking device 900 based on the task-oriented target tracking algorithm matching strategy of this embodiment includes an acquisition module 910 , a detection module 920 , an output module 930 and a processing module 940 .

[0191] The acquisition module 910 is used to acquire an image sequence for the target object, the image sequence includes I images, I is an integer greater than or equal to 1, and the image sequence includes a visible light image and an infrared image. In one embodiment, the acquisition module 910 can be used to perform the operation S210 described above, which will not be repeated here.

[0192] The detection module 920 is used to perform image detection on each image and generate attribute information of each image. In one embodiment, the detection module 920 can be used to perform the operation S220 described above, which will not be described in detail here.

[0193] The output module 930 is used to input the attribute information of the i-th image into the target model in response to the difference between the attribute information of the i-th image and the attribute information of the i-1-th image being greater than a predetermined threshold, and output information of a first target algorithm for processing the i-th image; wherein the target model is obtained by training the initial model based on the attribute information of the sample image and the algorithm information matching the attribute information of the sample image, 1<i≤I. In one embodiment, the output module 930 can be used to perform the operation S230 described above, which will not be repeated here.

[0194] The processing module 940 is used to process each image based on each target algorithm matching the attribute information of each image to obtain a tracking result for the target object. In one embodiment, the processing module 940 can be used to perform the operation S240 described above, which will not be repeated here.

[0195] According to an embodiment of the present invention, the output module includes a second target algorithm determination submodule, which is used to determine the algorithm information matching the attribute information of the i-1th image as the second target algorithm information of the i-th image in response to the difference between the attribute information of the i-th image and the attribute information of the i-1th image being less than a predetermined threshold.

[0196] According to an embodiment of the present invention, the output module further includes an acquisition submodule, a calling submodule, an effect information determination submodule, an algorithm evaluation information determination submodule, an algorithm information determination submodule and a training submodule.

[0197] The acquisition submodule is used to acquire multiple sample images and attribute information corresponding to the multiple sample images. The calling submodule is used to call multiple algorithms for each sample image to process each sample image and obtain multiple tracking results for each sample image. The effect information determination submodule is used to determine multiple effect information of each algorithm processing each sample image based on multiple tracking results and operation information of multiple algorithms, wherein the multiple effect information includes at least one of the following: accurate information, timeliness information and robust information. The algorithm evaluation information determination submodule is used to determine multiple algorithm evaluation information of each sample image based on multiple effect information. The algorithm information determination submodule is used to determine algorithm information matching the attribute information of each sample image from multiple algorithm evaluation information. The training submodule uses the sample data set to train the initial model to obtain the target model, wherein the sample data set includes multiple groups of sample data, and each group of sample data includes a sample target algorithm whose attribute information of the sample image matches the sample image.

[0198] According to an embodiment of the present invention, the algorithm evaluation information determination submodule includes a first weight determination unit, a second weight determination unit, a target weight determination unit and an algorithm evaluation information determination unit. The first weight determination unit is used to process multiple effect information using the hierarchical analysis method to obtain a first weight of each effect information for each sample image. The second weight determination unit is used to process multiple effect information using the objective weighting method to obtain a second weight of each effect information for each sample image. The target weight determination unit is used to determine the target weight of each effect information based on the first weight and the second weight of the multiple effect information. The algorithm evaluation information determination unit is used to obtain multiple algorithm evaluation information according to the target weight of each effect information.

[0199] According to an embodiment of the present invention, the detection module includes a detection strategy determination submodule and a detection submodule. The detection strategy determination submodule is used to determine the detection strategy according to the type of each image. The detection submodule is used to detect the image based on the detection strategy to obtain the attribute information of each image.

[0200] According to an embodiment of the present invention, the detection strategy determination submodule includes a visible light image detection unit and an infrared image detection unit. The visible light image detection unit is used to determine that the detection strategy is to perform visible light image detection on the image in response to the type of the image being a visible light image. The infrared image detection unit is used to determine that the detection strategy is to perform infrared image detection on the image in response to the type of the image being an infrared image.

[0201] According to an embodiment of the present invention, the attribute information of each image includes at least one of the following: brightness information, contrast information and clarity information. The detection submodule includes a first grayscale detection unit, a first grayscale information processing unit, a contrast enhancement processing unit, a second grayscale detection unit, a second grayscale information processing unit, a Laplace transform processing unit, a pixel detection unit and a pixel information processing unit.

[0202] The first grayscale detection unit is used to perform grayscale detection on the image to obtain the first grayscale information of each image. The first grayscale information processing unit is used to obtain the brightness information of each image by processing the first grayscale information. The contrast enhancement processing unit is used to perform contrast enhancement processing on each image. The second grayscale detection unit is used to perform grayscale detection on the enhanced image to obtain the second grayscale information of each image. The second grayscale information processing unit is used to obtain the contrast information of each image by processing the second grayscale information. The Laplace transform processing unit is used to perform Laplace transform processing on each image. The pixel detection unit is used to perform pixel detection on the transformed image to obtain the pixel information of each image. The pixel information processing unit is used to determine the clarity information of each image by processing the pixel information.

[0203] According to an embodiment of the present invention, any multiple modules among the acquisition module 910, the detection module 920, the output module 930 and the processing module 940 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the acquisition module 910, the detection module 920, the output module 930 and the processing module 940 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the acquisition module 910, the detection module 920, the output module 930 and the processing module 940 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0204] Fig.10 A block diagram of an electronic device suitable for implementing a target tracking method based on a task-oriented target tracking algorithm matching strategy according to an embodiment of the present invention is shown.

[0205] like Fig.10 As shown, the electronic device 1000 according to an embodiment of the present invention includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 into a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include an onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0206] In RAM 1003, various programs and data required for the operation of electronic device 1000 are stored. Processor 1001, ROM 1002 and RAM 1003 are connected to each other via bus 1004. Processor 1001 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 1002 and / or RAM 1003. It should be noted that the program can also be stored in one or more memories other than ROM 1002 and RAM 1003. Processor 1001 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0207] According to an embodiment of the present invention, the electronic device 1000 may further include an input / output (I / O) interface 1005, which is also connected to the bus 1004. The electronic device 1000 may further include one or more of the following components connected to the input / output (I / O) interface 1005: an input portion 1006 including a keyboard, a mouse, etc.; an output portion 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1008 including a hard disk, etc.; and a communication portion 1009 including a network interface card such as a LAN card, a modem, etc. The communication portion 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the input / output (I / O) interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage portion 1008 as needed.

[0208] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0209] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 1002 and / or RAM 1003 described above and / or one or more memories other than ROM 1002 and RAM 1003.

[0210] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiment of the present invention.

[0211] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 1001. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0212] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1009, and / or installed from the removable medium 1011. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0213] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the processor 1001, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0214] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0215] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0216] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0217] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A target tracking method based on a task-oriented target tracking algorithm matching strategy, characterized in that: The method comprises: Acquire an image sequence for the target object, the image sequence comprising I images, where I is an integer greater than 1, and the image sequence comprises a visible light image and an infrared image; Performing image detection on each image respectively to generate attribute information of each image; In response to a difference between the attribute information of the ith image and the attribute information of the i-1th image being greater than or equal to a predetermined threshold, inputting the attribute information of the ith image into a target model, and outputting information of a first target algorithm for processing the ith image; wherein the target model is obtained by training an initial model based on the attribute information of a sample image and algorithm information matching the attribute information of the sample image, 1<i≤I; and Each image is processed separately based on each target algorithm matched with the attribute information of each image to obtain a tracking result for the target object.

2. The method according to claim 1, characterized in that The method further comprises: In response to the difference between the attribute information of the i-th image and the attribute information of the i-1-th image being less than a predetermined threshold, the algorithm information matching the attribute information of the i-1-th image is determined as the information of the second target algorithm of the i-th image.

3. The method according to claim 2, characterized in that The target model is obtained by training the initial model based on the attribute information of the sample image and the algorithm information matching the attribute information of the sample image, including: Acquire a plurality of sample images and attribute information corresponding to the plurality of sample images; For each sample image, calling multiple algorithms to process each sample image respectively to obtain multiple tracking results for each sample image; Determine, according to the multiple tracking results and the operation information of the multiple algorithms, multiple effect information of each algorithm processing each sample image, wherein the multiple effect information includes at least one of the following: accuracy information, timeliness information and robustness information; Based on the multiple effect information, determining multiple algorithm evaluation information of each sample image; Determining algorithm information matching the attribute information of each sample image from the plurality of algorithm evaluation information; The initial model is trained using a sample data set to obtain a target model, wherein the sample data set includes multiple groups of sample data, and each group of sample data includes attribute information of a sample image and algorithm information matching the sample image.

4. The method according to claim 3, characterized in that The step of determining a plurality of algorithm evaluation information of each sample image based on the plurality of effect information comprises: Processing the plurality of effect information by using a hierarchical analysis method to obtain a first weight of each piece of effect information for each sample image; Processing the plurality of effect information by using an objective weighting method to obtain a second weight of each piece of effect information for each sample image; Determining a target weight for each piece of effect information based on the first weight and the second weight of the plurality of pieces of effect information; According to the target weights of the various effect information, multiple algorithm evaluation information is obtained.

5. The method according to claim 1, characterized in that The performing image detection on each image respectively to generate attribute information of each image includes: Determine the detection strategy based on the type of each image; Based on the detection strategy, the images are detected to obtain attribute information of each image.

6. The method according to claim 5, characterized in that Determining a detection strategy according to the type of each image includes: In response to the type of the image being a visible light image, determining that the detection strategy is to perform visible light image detection on the image; In response to the type of the image being an infrared image, the detection strategy is determined to perform infrared image detection on the image.

7. The method according to claim 5, characterized in that The attribute information of each image includes at least one of the following: brightness information, contrast information and clarity information; the image is detected based on the detection strategy to obtain the attribute information of each image, including: Performing grayscale detection on the image to obtain first grayscale information of each image; Obtaining brightness information of each image by processing the first grayscale information; Perform contrast enhancement on each image; Performing grayscale detection on the enhanced image to obtain second grayscale information of each image; Obtaining contrast information of each image by processing the second grayscale information; Perform Laplace transform on each image; Performing pixel detection on the transformed image to obtain pixel information of each image; By processing the pixel information, the definition information of each image is determined.

8. A target tracking device based on a task-oriented target tracking algorithm matching strategy, characterized in that: The device comprises: An acquisition module, used to acquire an image sequence for a target object, wherein the image sequence includes I images, where I is an integer greater than 1, and the image sequence includes a visible light image and an infrared image; A detection module, used to perform image detection on each image and generate attribute information of each image; an output module, configured to input the attribute information of the i-th image into a target model in response to the difference between the attribute information of the i-th image and the attribute information of the i-1-th image being greater than or equal to a predetermined threshold, and output information of a first target algorithm for processing the i-th image; wherein the target model is obtained by training an initial model based on the attribute information of the sample image and the algorithm information matching the attribute information of the sample image, 1<i≤I; and The processing module is used to process each image based on each target algorithm matched with the attribute information of each image to obtain the tracking result for the target object.

9. An electronic device, comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images