Single target tracking method and device

Abstract

The embodiment of the invention provides a single target tracking method and device, relates to the technical field of target tracking, and can improve the accuracy and robustness of single target tracking. The method provided by the embodiment of the invention comprises the following steps: determining a tracking target in a previous frame of image, and extracting semantic interference information of a surrounding background area by utilizing semantic features of the tracking target in the surrounding background area of the tracking target so as to create a semantic interference feature set; extracting appearance interference information of a surrounding background area in the surrounding background area of the previous frame image by using the appearance features of the tracking target so as to create an appearance interference feature set; positioning a possible area of the tracking target by using the semantic feature and the appearance feature of the tracking target in the rear frame image, and positioning the position of an interferent by using the semantic interference feature set and the appearance interference feature set in the rear frame image; and removing the position of the interferent in the possible existence area of the tracking target to obtain the position of the tracking target in the rear frame image.

Description

Technical Field

[0001] This invention relates to the field of target tracking technology, and in particular to a single target tracking method and apparatus. Background Technology

[0002] Single-object tracking is a research area in computer vision, aiming to locate the position and size of a single object in subsequent frames, given its position and size in the first or early frames. Currently, a major challenge in single-object tracking is tracking failure caused by similar interfering objects. Accurately extracting and utilizing this interference information to improve the accuracy and robustness of single-object tracking is a problem that urgently needs to be solved. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a method that can improve the accuracy and robustness of single-target tracking.

[0004] To achieve the above objectives, according to one aspect of the present invention, a single-target tracking method is provided.

[0005] The single-target tracking method of this invention includes: determining a tracking target in a previous frame image; extracting semantic interference information of the surrounding background region using pre-determined semantic features of the tracking target; and creating a semantic interference feature set based on the extracted semantic interference information; wherein the surrounding background region does not contain the tracking target; extracting appearance interference information of the surrounding background region using pre-determined appearance features of the tracking target; and creating an appearance interference feature set based on the extracted appearance interference information; locating the possible existence region of the tracking target in a subsequent frame image using the semantic and appearance features of the tracking target; locating at least one interference object position in the subsequent frame image using the semantic interference feature set and the appearance interference feature set; and removing the interference object positions in the possible existence region of the tracking target to obtain the position of the tracking target in the subsequent frame image.

[0006] Optionally, the step of extracting semantic interference information of the surrounding background region using predetermined semantic features of the tracking target within the surrounding background region includes: calculating the similarity between the semantic features of the tracking target and multiple unidentified locations in the surrounding background region of the tracking target; determining a first number of target locations with the highest similarity from the unidentified locations; extracting an area of ​​the same size as the tracking target from the target locations and extracting image features from that area; and sorting the extracted image features according to the similarity between the extracted image features and their target locations to obtain the semantic interference information.

[0007] Optionally, the step of creating a semantic interference feature set based on the extracted semantic interference information includes: storing the semantic interference information of each previous frame image in the order of the image frames to form the semantic interference feature set.

[0008] Optionally, the step of extracting appearance interference information of the surrounding background region in the previous frame image using the predetermined appearance features of the tracking target includes: calculating the similarity between the appearance features of the tracking target and multiple unidentified locations in the surrounding background region of the tracking target; determining a second number of target locations with the highest similarity from the unidentified locations; extracting an area of ​​the same size as the tracking target from the target locations and extracting the image features of that area; and sorting the extracted image features according to the similarity of their respective target locations to obtain the appearance interference information.

[0009] Optionally, creating an appearance interference feature set based on the extracted appearance interference information includes: storing the appearance interference information of each previous frame image in the order of the image frames to form the appearance interference feature set.

[0010] Optionally, locating the possible existence region of the tracking target in the subsequent frame image using the semantic and appearance features of the tracking target includes: calculating the similarity between the semantic features of the tracking target and each position to be determined in each subsequent frame image, and determining the third number of semantic candidate positions with the highest similarity from the positions to be determined; calculating the similarity between the appearance features of the tracking target and each position to be determined in each subsequent frame image, and determining the fourth number of appearance candidate positions with the highest similarity from the positions to be determined; and combining the semantic candidate positions and appearance candidate positions in the same subsequent frame image into the possible existence region of the tracking target in that subsequent frame image.

[0011] Optionally, locating at least one interference location in the subsequent frame image using the semantic interference feature set and the appearance interference feature set includes: calculating the similarity between each semantic interference information in the semantic interference feature set and each location to be identified in each subsequent frame image, and determining the location with the highest similarity as the semantic interference location; calculating the similarity between each appearance interference information in the appearance interference feature set and each location to be identified in each subsequent frame image, and determining the location with the highest similarity as the appearance interference location; the semantic interference location and the appearance interference location constitute the at least one interference location.

[0012] Optionally, removing the locations of interfering objects in the possible areas of the tracked target includes: removing the locations of interfering objects in the possible areas of the tracked target in each subsequent frame image to obtain candidate target areas; calculating the similarity between each location in the candidate target areas and the semantic features of the tracked target, and determining the location with the highest similarity as the semantic location; calculating the similarity between each location in the candidate target areas and the appearance features of the tracked target, and determining the location with the highest similarity as the appearance location; and fusing the semantic location and the appearance location to obtain the location of the tracked target in the subsequent frame image.

[0013] Optionally, the method further includes: after determining the position of the tracking target in each subsequent frame image, extracting semantic interference information of the surrounding background region in the subsequent frame image using the semantic features of the tracking target, and updating the semantic interference feature set based on the extracted semantic interference information; wherein the surrounding background region does not contain the tracking target; extracting appearance interference information of the surrounding background region in the subsequent frame image using the appearance features of the tracking target, and updating the appearance interference feature set based on the extracted appearance interference information.

[0014] To achieve the above objectives, according to another aspect of the present invention, a single-target tracking device is provided.

[0015] The single-target tracking device of this invention includes: a first preparation unit, configured to determine a tracking target in a previous frame image, extract semantic interference information of the surrounding background region using predetermined semantic features of the tracking target, and create a semantic interference feature set based on the extracted semantic interference information; wherein the surrounding background region does not contain the tracking target; a second preparation unit, configured to extract appearance interference information of the surrounding background region in the previous frame image using predetermined appearance features of the tracking target, and create an appearance interference feature set based on the extracted appearance interference information; a positioning unit, configured to locate the possible existence area of ​​the tracking target in a subsequent frame image using the semantic features and appearance features of the tracking target, and locate at least one interference object position in the subsequent frame image using the semantic interference feature set and the appearance interference feature set; and a target determination unit, configured to remove interference object positions from the possible existence area of ​​the tracking target to obtain the position of the tracking target in the subsequent frame image.

[0016] To achieve the above objectives, according to another aspect of the present invention, an electronic device is provided.

[0017] An electronic device according to the present invention includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the single-target tracking method provided by the present invention.

[0018] To achieve the above objectives, according to another aspect of the present invention, a non-transitory computer-readable storage medium is provided.

[0019] The present invention provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the single-target tracking method provided by the present invention.

[0020] According to the technical solution of the present invention, one embodiment of the above invention has the following advantages or beneficial effects: By pre-extracting semantic and appearance features of the tracking target, a corresponding interference feature set is constructed based on the previous frame image. Then, these two interference feature sets are used to locate the interference objects in the subsequent frame image from both semantic and appearance perspectives. After locating the possible area of ​​the tracking target using its semantic and appearance features, the locations of each interference object are removed from this area to obtain the tracking target's location. In this way, semantic and appearance interference information is accurately extracted from the previous frame image using the tracking target's semantic and appearance features, and a corresponding feature set is constructed. For the subsequent frame image, the tracking target area is calculated from both semantic and appearance perspectives, and the locations of interference objects in the subsequent frame image are comprehensively extracted using both feature sets. Finally, the interference objects are removed to obtain the tracking target's location in the subsequent frame image. This combination of semantic and appearance-level computational logic improves the comprehensiveness of background interference information extraction, tracking accuracy, and tracking robustness in single-target tracking, and to some extent solves the tracking failure problem caused by similar interference objects.

[0021] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0022] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein: Figure 1 This is a schematic diagram of the main steps of the single-target tracking method in an embodiment of the present invention; Figure 2 This is a schematic diagram of the main parts of the single-target tracking device in an embodiment of the present invention; Figure 3 This is a schematic diagram of the hardware structure of an electronic device used to implement the single-target tracking method of this invention. Detailed Implementation

[0023] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0024] Figure 1 This is a schematic diagram illustrating the main steps of the single-target tracking method in an embodiment of the present invention. See [link / reference] Figure 1 The single-target tracking method of this invention can be executed by a computer, specifically by performing the following steps.

[0025] Step S101: Determine the tracking target in the previous frame image, extract semantic interference information of the surrounding background region using the predetermined semantic features of the tracking target, and create a semantic interference feature set based on the extracted semantic interference information; wherein the tracking target is not contained in the surrounding background region.

[0026] In this specification, the preceding frame image can be one or more frames, and the following frame image can be one or more frames containing multiple elements. The preceding frame image is generated earlier than the following frame image. Semantic features refer to features generated through natural language description. In this specification, semantic features are features formed after alignment operations from semantic modality to image modality. These semantic features can perform related operations (such as similarity calculation) with pixel-based image features. Appearance features refer to features generated based on pixels in the image.

[0027] In this step, the target location has been pre-determined in the previous frame image, which can be done by pre-specifying it. After the target location is determined in the previous frame image, the computer can determine the surrounding background region of the target according to preset rules. The surrounding background region can be centered on the target but does not include the surrounding background area. For example, the preset rules can be: defining the area within a circle with the target as the center and a preset distance as the radius (excluding the target location) as the surrounding background region, or defining a square area with the target as the center and a preset distance as the side length (excluding the target location) as the surrounding background region. It can be understood that the computer can extract relevant information about background interference from the above-mentioned surrounding background region that does not contain the target.

[0028] In this step, the computer can extract semantic interference information from the surrounding background area of ​​the tracked target using pre-determined semantic features of the tracked target. Specifically, the computer can calculate the similarity between the semantic features of the tracked target and multiple unidentified locations in the surrounding background area, and determine a first number of target locations with the highest similarity from the unidentified locations. For example, the first number can be determined as follows: first, determine the number of unidentified locations with a similarity greater than a threshold; if this number is greater than or equal to 5, then the first number is 5, and the target locations are the 5 unidentified locations with the highest similarity; if the number of unidentified locations with a similarity greater than the threshold is less than 5, then this number is determined as the first number, and the number of target locations is less than 5. The multiple unidentified locations in the surrounding background area for which similarity calculation is performed can be determined by using a search box of a preset size and moving in a preset direction (e.g., first from left to right, then from top to bottom).

[0029] Subsequently, the computer extracts a region of the same size as the tracked target from the target location and extracts the image features of that region. The extracted image features are then sorted according to their similarity to the target location (either in ascending or descending order) to obtain the semantic interference information.

[0030] After determining the semantic interference information, the computer can create a semantic interference feature set based on the extracted semantic interference information. Specifically, the computer can store the semantic interference information of each previous frame image in the order of the image frames to form the semantic interference feature set.

[0031] For example, semantic interference feature sets can be used express, ,in, Indicates the number of frames. Indicates the first The first frame exists Semantic interference information, This indicates the total amount of semantic interference information in the image frame.

[0032] Step S102: Extract appearance interference information of the surrounding background region in the previous frame image using the pre-determined appearance features of the tracking target, and create an appearance interference feature set based on the extracted appearance interference information.

[0033] In this step, the computer can extract appearance interference information from the surrounding background region in the previous frame image using the appearance features of the tracked target, and then establish an appearance interference feature set. Specifically, the computer first calculates the similarity between the appearance features of the tracked target and multiple unidentified locations in the surrounding background region of the tracked target, and determines the second number of target locations with the highest similarity from the unidentified locations; for example, the second number can be determined in the following way: first, determine the number of unidentified locations with similarity greater than a threshold; if this number is greater than or equal to 5, then the second number is equal to 5, and the target locations are the 5 unidentified locations with the highest similarity; if the number of unidentified locations with similarity greater than the threshold is less than 5, then this number is determined as the second number, and the number of target locations is less than 5.

[0034] Subsequently, the computer extracts a region of the same size as the tracked target from the target location and extracts image features from this region. The extracted image features are then sorted according to their similarity to the target location to obtain the appearance interference information. Finally, the computer stores the appearance interference information of each previous frame image in frame order to form the appearance interference feature set.

[0035] For example, the appearance interference feature set can be used express, ,in, Indicates the number of frames. Indicates the first The first frame exists One piece of visual interference information, This represents the total amount of appearance interference information in the image frame. It can be understood that the steps of extracting semantic interference information and extracting appearance interference information can be performed in parallel, or in any order.

[0036] Step S103: In the subsequent frame image, locate the possible area where the tracked target may exist using the semantic features and appearance features of the tracked target, and locate at least one interference object location in the subsequent frame image using the semantic interference feature set and the appearance interference feature set.

[0037] In this step, the computer determines the possible regions where the tracked target may exist in the subsequent frame image in the following manner. The computer first calculates the similarity between the semantic features of the tracked target and each undiscriminated location in each subsequent frame image, and then determines the third number of semantic candidate locations with the highest similarity from the undiscriminated locations. For example, the third number can be determined as follows: first, determine the number of undiscriminated locations with a similarity greater than a threshold; if this number is greater than or equal to 5, then the third number is 5, and the semantic candidate locations are the 5 undiscriminated locations with the highest similarity; if the number of undiscriminated locations with a similarity greater than the threshold is less than 5, then this number is determined as the third number, and the number of semantic candidate locations is less than 5. The multiple undiscriminated locations in the subsequent frame image from which the similarity calculation is performed can be determined by using a search box of a preset size and moving in a preset direction (e.g., first from left to right, then from top to bottom).

[0038] Subsequently, the computer calculates the similarity between the appearance features of the tracked target and each undiscriminated location in each subsequent frame image, and determines the fourth number of appearance candidate locations with the highest similarity from the undiscriminated locations. For example, the fourth number can be determined as follows: first, determine the number of undiscriminated locations with a similarity greater than a threshold; if this number is greater than or equal to 5, then the fourth number is 5, and the appearance candidate locations are the 5 undiscriminated locations with the highest similarity; if the number of undiscriminated locations with a similarity greater than the threshold is less than 5, then this number is determined as the fourth number, and the number of appearance candidate locations is less than 5. It is understood that the calculation of semantic candidate locations and appearance candidate locations can be performed in parallel or in any order.

[0039] Finally, the computer combines the semantic candidate locations and appearance candidate locations in the same subsequent frame image into the possible regions where the tracked target may exist in that subsequent frame image.

[0040] After determining the possible location of the tracked target in the subsequent frame image, the computer can locate at least one interference object position in the subsequent frame image using a semantic interference feature set and an appearance interference feature set. Specifically, the computer can perform similarity calculations on each semantic interference information in the semantic interference feature set and each position to be identified in each subsequent frame image, and determine the position to be identified with the highest similarity as the semantic interference object position. The computer can also perform similarity calculations on each appearance interference information in the appearance interference feature set and each position to be identified in each subsequent frame image, and determine the position to be identified with the highest similarity as the appearance interference object position; the semantic interference object position and the appearance interference object position constitute the at least one interference object position. It can be understood that the calculation of the semantic interference object position and the appearance interference object position can be performed in parallel or in any order.

[0041] Step S104: Remove the locations of interfering objects in the possible areas where the tracked target may exist to obtain the location of the tracked target in the subsequent frame image.

[0042] Preferably, in this step, the computer first removes interference locations from the possible areas of the tracked target in each subsequent frame image to obtain candidate target regions. Then, the computer calculates the similarity between each location in the candidate target region and the semantic features of the tracked target, determining the location with the highest similarity as the semantic location; it also calculates the similarity between each location in the candidate target region and the appearance features of the tracked target, determining the location with the highest similarity as the appearance location. These two calculation processes can be performed in parallel or sequentially. Finally, the computer fuses the semantic location and the appearance location to obtain the location of the tracked target in the subsequent frame image. This fusion process can be as follows: the computer combines the pixels of the semantic location and the appearance location into a region, and then extracts the central region of this region, which is the same size as the tracked target, as the tracked target location.

[0043] In practical applications, after determining the position of the tracking target in each subsequent frame, the computer can extract semantic interference information of the surrounding background region using the semantic features of the tracking target, and update the semantic interference feature set based on the extracted semantic interference information. The tracking target is not present in this surrounding background region. In parallel, the computer can extract appearance interference information of the surrounding background region using the appearance features of the tracking target, and update the appearance interference feature set based on the extracted appearance interference information. This allows for continuous updating and iteration of the two appearance interference feature sets, continuously improving the comprehensiveness of background interference information extraction and the accuracy and robustness of single-target tracking.

[0044] Figure 2 This is a schematic diagram of the main parts of a single-target tracking device according to an embodiment of the present invention.

[0045] like Figure 2As shown, the single-target tracking device 20 of this embodiment includes: a first preparation unit 21, configured to determine a tracking target in a previous frame image, extract semantic interference information of the surrounding background region using predetermined semantic features of the tracking target, and create a semantic interference feature set based on the extracted semantic interference information; wherein the surrounding background region does not contain the tracking target; a second preparation unit 22, configured to extract appearance interference information of the surrounding background region in the previous frame image using predetermined appearance features of the tracking target, and create an appearance interference feature set based on the extracted appearance interference information; a positioning unit 23, configured to locate the possible existence area of ​​the tracking target in a subsequent frame image using the semantic features and appearance features of the tracking target, and locate at least one interference object position in the subsequent frame image using the semantic interference feature set and the appearance interference feature set; and a target determination unit 24, configured to remove interference object positions from the possible existence area of ​​the tracking target to obtain the position of the tracking target in the subsequent frame image.

[0046] In this embodiment of the invention, the first preparation unit 21 is further configured to: calculate the similarity between the semantic features of the tracked target and multiple unidentified locations in the surrounding background region of the tracked target; determine the first number of target locations with the highest similarity from the unidentified locations; extract an area of ​​the same size as the tracked target from the target location and extract the image features of the area; sort the extracted image features according to the similarity of the target locations to obtain the semantic interference information.

[0047] As a preferred embodiment, the first preparation unit 21 is further configured to: store the semantic interference information of each previous frame image in the order of the image frames to form the semantic interference feature set.

[0048] Preferably, the second preparation unit 22 is further configured to: calculate the similarity between the appearance features of the tracked target and multiple locations to be identified in the surrounding background area of ​​the tracked target; determine the second number of target locations with the highest similarity from the locations to be identified; extract an area of ​​the same size as the tracked target from the target locations and extract the image features of the area; sort the extracted image features according to the similarity of the target locations to obtain the appearance interference information.

[0049] In one embodiment, the second preparation unit 22 is further configured to: store the appearance interference information of each previous frame image in the order of the image frames to form the appearance interference feature set.

[0050] In an optional technical solution, the positioning unit 23 is further configured to: calculate the similarity between the semantic features of the tracked target and each position to be determined in each subsequent frame image, and determine the third number of semantic candidate positions with the highest similarity from the positions to be determined; calculate the similarity between the appearance features of the tracked target and each position to be determined in each subsequent frame image, and determine the fourth number of appearance candidate positions with the highest similarity from the positions to be determined; and combine the semantic candidate positions and appearance candidate positions in the same subsequent frame image into a possible region where the tracked target may exist in the subsequent frame image.

[0051] In specific applications, the positioning unit 23 is further configured to: perform similarity calculations on each semantic interference information in the semantic interference feature set and each position to be identified in each subsequent frame image, and determine the position to be identified with the highest similarity as the semantic interference object position; perform similarity calculations on each appearance interference information in the appearance interference feature set and each position to be identified in each subsequent frame image, and determine the position to be identified with the highest similarity as the appearance interference object position; the semantic interference object position and the appearance interference object position constitute the at least one interference object position.

[0052] In practical applications, the target determination unit 24 is further configured to: remove interference locations from the possible areas of the tracked target in each subsequent frame image to obtain candidate target areas; calculate the similarity between each location in the candidate target area and the semantic features of the tracked target, and determine the location with the highest similarity as the semantic location; calculate the similarity between each location in the candidate target area and the appearance features of the tracked target, and determine the location with the highest similarity as the appearance location; and fuse the semantic location and the appearance location to obtain the location of the tracked target in the subsequent frame image.

[0053] Furthermore, in this embodiment of the invention, the device 20 further includes a feature set update unit, configured to: after determining the position of the tracking target in each subsequent frame image, extract semantic interference information of the surrounding background region in the subsequent frame image using the semantic features of the tracking target, and update the semantic interference feature set based on the extracted semantic interference information; wherein the surrounding background region does not contain the tracking target; extract appearance interference information of the surrounding background region in the subsequent frame image using the appearance features of the tracking target, and update the appearance interference feature set based on the extracted appearance interference information.

[0054] It should be noted that the single-target tracking device of this invention, as software, can be installed in devices such as computers and mobile terminals.

[0055] In the technical solution of this invention, a corresponding interference feature set is constructed based on the previous frame image by pre-extracting semantic and appearance features of the tracking target. Then, the two interference feature sets are used to locate the interference objects in the subsequent frame image from both semantic and appearance perspectives. After locating the possible area of ​​the tracking target using the semantic and appearance features, the positions of each interference object are removed from the possible area to obtain the tracking target position within that area. Thus, for the previous frame image, semantic and appearance interference information is accurately extracted and corresponding feature sets are constructed by combining the semantic and appearance features of the tracking target. For the subsequent frame image, the tracking target area is calculated from both semantic and appearance perspectives, and the positions of interference objects in the subsequent frame image are comprehensively extracted using the two feature sets. Finally, the interference objects are removed to obtain the tracking target position in the subsequent frame image. This combination of semantic and appearance-level computational logic improves the comprehensiveness of background interference information extraction, tracking accuracy, and tracking robustness during single-target tracking, and to some extent solves the tracking failure problem caused by similar interference objects.

[0056] It should be noted that the collection, analysis, use, transmission, and storage of user personal information involved in the technical solution of this invention all comply with the provisions of relevant laws and regulations, are used for legal and reasonable purposes, and are not shared, disclosed, or sold outside of these legal uses, and are subject to supervision and management by regulatory authorities. Necessary measures should be taken to prevent unauthorized access to such personal information data, ensure that personnel authorized to access personal information data comply with the provisions of relevant laws and regulations, and ensure the security of user personal information. Once this user personal information data is no longer needed, the risk should be minimized by restricting or even prohibiting data collection and / or deleting the data. When used, including in certain related applications, user privacy is protected by de-identifying the data, for example, by removing specific identifiers (e.g., date of birth), controlling the amount or specificity of stored data (e.g., collecting location data at the city level rather than the specific address level), controlling how the data is stored, and / or other de-identification methods.

[0057] According to embodiments of the present invention, the present invention also provides an electronic device and a readable storage medium.

[0058] The electronic device of the present invention includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the at least one processor to perform the single-target tracking method provided by the present invention.

[0059] The non-transitory computer-readable storage medium of the present invention stores computer instructions for causing the computer to execute the single-target tracking method provided by the present invention.

[0060] like Figure 3 The diagram shown is a hardware structure schematic of an electronic device used to implement the method of an embodiment of the present invention. Figure 3 The electronic device includes: one or more processors 31 and memory 32. Figure 3 Taking a processor 31 as an example, the memory 32 is the non-transitory computer-readable storage medium provided by this invention.

[0061] The electronic device of the present invention may further include an input device 33 and an output device 34.

[0062] The processor 31, memory 32, input device 33, and output device 34 can be connected via a bus or other means. Figure 3 Taking the example of a connection between China and Israel via a bus.

[0063] The memory 32, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the method in the embodiments of the present invention. The processor 31 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in the memory 32, thereby implementing the single-target tracking method of the above-described method embodiments.

[0064] The memory 32 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the device. Furthermore, the memory 32 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 32 may optionally include memory remotely located relative to the processor 31, and these remote memories may be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0065] Input device 33 can receive input digital or character information, and generate key signal inputs related to user settings and function control of the device. Output device 34 may include display devices such as a display screen.

[0066] One or more of the above modules are stored in the memory 32, and when executed by one or more processors 31, the single-target tracking method of any of the above method embodiments is executed.

[0067] The above-described product can execute the single-target tracking method provided in the embodiments of the present invention, and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in this embodiment can be found in the single-target tracking method provided in the embodiments of the present invention.

[0068] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A single-target tracking method, characterized in that, include: In the previous frame image, a tracking target is determined. Semantic interference information of the surrounding background region is extracted using pre-determined semantic features of the tracking target. A semantic interference feature set is created based on the extracted semantic interference information. The tracking target is not contained in the surrounding background region. In the surrounding background region of the previous frame image, appearance interference information of the surrounding background region is extracted using the appearance features of the tracked target that are predetermined, and an appearance interference feature set is created based on the extracted appearance interference information; In the subsequent frame image, the semantic and appearance features of the tracked target are used to locate the possible area where the tracked target may exist, and the semantic interference feature set and the appearance interference feature set are used to locate the position of at least one interference object in the subsequent frame image. The positions of interfering objects in the possible areas where the tracked target may exist are removed to obtain the position of the tracked target in the subsequent frame image.

2. The method as described in claim 1, characterized in that, The step of extracting semantic interference information of the surrounding background region using pre-determined semantic features of the tracking target includes: The similarity between the semantic features of the tracked target and multiple unidentified locations in the surrounding background region of the tracked target is calculated, and the first number of target locations with the highest similarity are determined from the unidentified locations; The semantic interference information is obtained by extracting an area of ​​the same size as the tracked target from the target location and extracting the image features of that area, and sorting the extracted image features according to their similarity to the target location.

3. The method as described in claim 2, characterized in that, The creation of a semantic interference feature set based on the extracted semantic interference information includes: The semantic interference information of each previous frame image is stored in the order of the image frames to form the semantic interference feature set.

4. The method as described in claim 1, characterized in that, The step of extracting appearance interference information of the surrounding background region in the previous frame image using pre-determined appearance features of the tracked target includes: The similarity between the appearance features of the tracked target and multiple locations to be identified in the surrounding background area is calculated, and the second number of target locations with the highest similarity are determined from the locations to be identified. The same size region as the tracked target is extracted from the target location, and the image features of the region are extracted. The extracted image features are sorted according to the similarity of their target locations to obtain the appearance interference information.

5. The method as described in claim 4, characterized in that, The creation of an appearance interference feature set based on the extracted appearance interference information includes: The appearance interference information of each previous frame image is stored in the order of the image frames to form the appearance interference feature set.

6. The method as described in claim 1, characterized in that, The step of locating the possible location region of the tracked target in the subsequent frame image using the semantic and appearance features of the tracked target includes: The semantic features of the tracked target are compared with each position to be determined in each subsequent frame image. The third number of semantic candidate positions with the highest similarity are then determined from the positions to be determined. The similarity between the appearance features of the tracked target and each position to be determined in each subsequent frame image is calculated, and the fourth number of appearance candidate positions with the highest similarity are determined from the positions to be determined. The semantic candidate locations and appearance candidate locations in the same subsequent frame image are combined to form the possible regions where the tracked target may exist in that subsequent frame image.

7. The method as described in claim 6, characterized in that, The step of locating at least one interference object in the subsequent frame image using the semantic interference feature set and the appearance interference feature set includes: For each semantic interference information in the semantic interference feature set and for each position to be identified in each subsequent frame image, a similarity calculation is performed, and the position to be identified with the highest similarity is determined as the position of the semantic interference object. For each appearance interference information in the appearance interference feature set and for each position to be identified in each subsequent frame image, a similarity calculation is performed, and the position to be identified with the highest similarity is determined as the appearance interference position; the semantic interference position and the appearance interference position constitute the at least one interference position.

8. The method as described in claim 7, characterized in that, The step of removing the locations of interfering objects in the possible area where the tracked target may exist includes: The locations of interfering objects in the possible areas where the tracked target may exist in each subsequent frame image are removed to obtain the candidate target area; Calculate the similarity between each position in the candidate target region and the semantic features of the tracked target, and determine the position with the highest similarity as the semantic position; calculate the similarity between each position in the candidate target region and the appearance features of the tracked target, and determine the position with the highest similarity as the appearance position; The semantic location and the appearance location are fused to obtain the location of the tracked target in the subsequent frame image.

9. The method as described in claim 8, characterized in that, The method further includes: After determining the position of the tracking target in each subsequent frame image, semantic interference information of the surrounding background region is extracted using the semantic features of the tracking target in the subsequent frame image, and the semantic interference feature set is updated based on the extracted semantic interference information; wherein the tracking target is not contained in the surrounding background region. In the subsequent frame image, appearance interference information of the surrounding background region is extracted using the appearance features of the tracked target, and the appearance interference feature set is updated based on the extracted appearance interference information.

10. A single-target tracking device, characterized in that, include: The first preparation unit is used to determine the tracking target in the previous frame image, extract semantic interference information of the surrounding background region using the predetermined semantic features of the tracking target, and create a semantic interference feature set based on the extracted semantic interference information; wherein the surrounding background region does not contain the tracking target; The second preparation unit is used to extract appearance interference information of the surrounding background region in the previous frame image using the appearance features of the tracked target that are predetermined, and to create an appearance interference feature set based on the extracted appearance interference information. The localization unit is used to locate the possible existence area of ​​the tracked target in the subsequent frame image using the semantic features and appearance features of the tracked target, and to locate the position of at least one interference object in the subsequent frame image using the semantic interference feature set and the appearance interference feature set. The target determination unit is used to remove the positions of interference objects in the possible areas where the tracked target may exist, and obtain the position of the tracked target in the subsequent frame image.

Images