Target object tracking method, device, storage medium and computer equipment

Abstract

The present application relates to a target object tracking method, device, computer-readable storage medium and computer equipment, the method comprising: obtaining the current position information of each of a plurality of candidate objects in a current image frame; obtaining the historical position information of an object to be compared in a previous image frame; the object to be compared includes at least one of a target object and a non-target object; determining the position comparison parameters of each candidate object according to the current position information and the historical position information; excluding the candidate objects whose position comparison parameters meet the interference object determination conditions from the plurality of candidate objects; determining the position information of the target object in the current image frame according to the exclusion result. The scheme provided by the present application can improve the accuracy of target object tracking.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a target tracking method, device, storage medium and computer equipment. Background Art

[0002] With the development of computer technology, more and more scenarios of interaction between users and computers have emerged. For example, users can interact with computers by controlling specified objects. In the scenario of interacting with computers, it is usually necessary to locate and track the target object, and then perform different interactive actions based on the tracked target object.

[0003] The traditional way to locate and track a target object is usually to use a target detection algorithm to detect candidate objects in an image and the confidence level of the candidate objects, and then select the candidate object with the highest confidence level as the current target object. However, when the target object is blurry and the interfering object is clear, the target object identified based on the confidence level is not accurate enough, resulting in target object tracking failure. Summary of the invention

[0004] Based on this, it is necessary to provide a target object method, apparatus, computer-readable storage medium and computer device to address the technical problem of inaccurate target object tracking.

[0005] A target object tracking method, comprising:

[0006] Obtaining current position information of each of multiple candidate objects in the current image frame;

[0007] Acquire historical position information of an object to be compared in a previous image frame; the object to be compared includes at least one of a target object and a non-target object;

[0008] Determine a location comparison parameter for each candidate object according to the current location information and the historical location information;

[0009] Excluding, from among the plurality of candidate objects, candidate objects whose position comparison parameters meet the interference object determination condition;

[0010] The position information of the target object in the current image frame is determined according to the exclusion result.

[0011] A target object tracking device, the device comprising:

[0012] An acquisition module, used to acquire current position information of each of a plurality of candidate objects in a current image frame;

[0013] The acquisition module is also used to acquire historical position information of the object to be compared in the previous image frame; the object to be compared includes at least one of a target object and a non-target object;

[0014] A determination module, used to determine a location comparison parameter of each candidate object according to the current location information and the historical location information;

[0015] An exclusion module, used to exclude, from a plurality of candidate objects, candidate objects whose position comparison parameters meet the interference object determination condition;

[0016] The determination module is further used to determine the position information of the target object in the current image frame according to the exclusion result.

[0017] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the steps of the target object tracking method.

[0018] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the target object tracking method.

[0019] The target object tracking method, device, computer-readable storage medium and computer equipment, when tracking the target object in the current image frame, respectively compare the current position information of multiple candidate objects in the current image frame with the historical position information of the object to be compared in the previous image frame, so as to exclude the candidate objects whose position comparison parameters meet the interference object determination conditions. Among them, the object to be compared includes at least one of the target object and the non-target object. In this way, based on the characteristic that the position change of the candidate object between adjacent image frames of the image sequence is small, the interference object in the candidate object can be efficiently and accurately excluded based on the historical position information. Then, the position information of the target object in the current image frame is determined based on the result of the exclusion. In this way, the position information of the target object is determined based on the remaining candidate objects after the interference object is excluded, which greatly reduces the determination range of the target object, can effectively avoid the interference of other objects in the image frame on the target object, can accurately find the position of the target object in a multi-object scene, and greatly improves the accuracy of target object tracking.

[0020] A target object tracking method, comprising:

[0021] Obtaining respective areas of multiple candidate objects in the current image frame;

[0022] When the ratio of the largest area to the second largest area among the areas is greater than or equal to a threshold, taking the position information of the candidate object with the largest area as the position information of the target object;

[0023] When the ratio of the largest area to the second largest area among the areas is less than the threshold, excluding the candidate objects that meet the interference object determination condition from the multiple candidate objects;

[0024] The position information of the target object in the current image frame is determined according to the exclusion result.

[0025] A target object tracking device, characterized in that the device comprises:

[0026] An acquisition module, used for acquiring the respective areas of a plurality of candidate objects in the current image frame;

[0027] a determination module, configured to use the position information of the candidate object with the largest area as the position information of the target object when the ratio of the largest area to the second largest area among the areas is greater than or equal to a threshold;

[0028] an exclusion module, configured to exclude candidate objects that meet the interference object determination condition from the multiple candidate objects when the ratio of the largest area to the second largest area among the areas is less than the threshold;

[0029] The determination module is further used to determine the position information of the target object in the current image frame according to the exclusion result.

[0030] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the steps of the target object tracking method.

[0031] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the target object tracking method.

[0032] The target object tracking method, device, computer-readable storage medium and computer equipment, when tracking the target object in the current image frame, obtain the area of ​​each of the multiple candidate objects in the current image frame. When the ratio of the maximum area to the second largest area in the area exceeds the threshold, it can be considered that the user expects to specify the object with the largest area as the target object when interacting with the computer. At this time, the candidate object with the largest area can be flexibly switched as the target object. When the ratio of the maximum area to the second largest area in the area is less than the threshold, the candidate object that meets the interference object determination condition is excluded from the multiple candidate objects, and then the position information of the target object in the current image frame is determined according to the result of the exclusion. In this way, the user's intention can be accurately identified by the ratio of the maximum area to the second largest area in the candidate object, and the target object can be flexibly switched. When the user's intention is unclear, the interference of other objects in the image frame to the target object can also be effectively avoided by exclusion, and the position of the target object can be accurately found in a multi-object scene, which greatly improves the accuracy of tracking the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A diagram of an application environment of a target object tracking method in an embodiment;

[0034] Figure 2 A schematic diagram of a target object tracking method in one embodiment;

[0035] Figure 3 A schematic diagram of an interface for determining a target hand in one embodiment;

[0036] Figure 4 A schematic diagram of identifying a gesture type corresponding to a target hand in one embodiment;

[0037] Figure 5 A schematic diagram of a target object tracking method in one embodiment;

[0038] Figure 6 A schematic diagram of a target object tracking method in one embodiment;

[0039] Figure 7 A schematic diagram of a target object tracking method in one embodiment;

[0040] Figure 8 is a structural block diagram of a target object tracking device in one embodiment;

[0041] Fig. 9 is a structural block diagram of a target object tracking device in another embodiment;

[0042] Fig.10 is a structural diagram of a target object tracking device in one embodiment;

[0043] Fig.11 FIG. 4 is a structural block diagram of a computer device in one embodiment. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0045] Figure 1 FIG. 1 is an application environment diagram of a target object tracking method in an embodiment. Figure 1 The target object tracking method is applied to a target object tracking system. The target object tracking includes a terminal 110 and a server 120. The terminal 110 and the server 120 are connected via a network. The terminal 110 can be a desktop terminal or a mobile terminal, and the mobile terminal can be at least one of a mobile phone, a tablet computer, a laptop computer, etc. The server 120 can be implemented by an independent server or a server cluster composed of multiple servers.

[0046] Among them, the terminal 110 and the server 120 can be used to execute the above-mentioned target object tracking method separately, or the terminal 110 and the server 120 can cooperate to execute the above-mentioned target object tracking method. For example, the terminal 110 collects video data through a camera and extracts image frames from the video data locally, thereby executing the target object tracking method to track the target object. Alternatively, the terminal 110 collects video data and sends the collected video data to the server 120, and the server 120 extracts image frames from the video data, thereby executing the target object tracking method and feeding back the position information of the target object to the terminal 110.

[0047] like Figure 2 As shown, in one embodiment, a target object tracking method is provided. This embodiment mainly applies the method to the above Figure 1 The computer device (terminal 110 or server 120) in FIG. Figure 2 , the target object tracking method specifically includes the following steps:

[0048] S202, obtaining current position information of each of a plurality of candidate objects in the current image frame.

[0049] Among them, an image frame is a single frame image in a dynamic picture. A candidate object is an object in an image frame that can be tracked, which can be an object with practical significance or a bounding box of the area where the object is located. For example, when a user interacts with a computer through his hands, the object to be tracked is a hand, then the objects detected as hands in the image frame can be called candidate objects, or the bounding box where the hand is located can be used as a candidate object. For another example, when a user interacts with a computer by controlling a virtual character, the object to be tracked is a virtual character, then the objects detected as virtual characters in the image frame can be called candidate objects, or the bounding box where the virtual character is located can be used as a candidate object.

[0050] The current position information of the candidate object refers to the position information of the candidate object in the current image frame, and the candidate object can be determined by the position information. The position information can specifically be the position coordinates of the candidate object, or relative position information, etc. For example, the current position information of the candidate object can be the center coordinates of the candidate object, the edge coordinates of the candidate object, or the coordinates of the bounding box where the object is located. The current position information of the candidate object can also be the relative distance of a candidate object relative to another candidate object.

[0051] In one embodiment, the computer device can obtain an image frame, perform target detection on the image frame, detect multiple candidate objects in the current image frame through a target detection algorithm, and determine the current position information of each of the multiple candidate objects. Among them, the target detection algorithm can specifically be a target detection algorithm based on a neural network, such as SSD (Single Shot MultiBox Detector), YOLO (You Only Look Once, a real-time target detection algorithm) algorithm, etc. Among them, the SSD algorithm and YOLO algorithm technology are relatively mature, and no further explanation is given here.

[0052] In one embodiment, the computer device performs target detection on the current image frame and can determine the bounding box of the area where the detected object is located (the bounding box can represent the area where the object is located). The shape of the bounding box can be a rectangle, a circle, a triangle, etc., and the shape is not limited. The computer device can use the position coordinates of the bounding box to represent the current position information of the candidate object. For example, when the bounding box is a rectangle, the position coordinates of the upper left corner and the lower right corner of the bounding box can be used as the position information of the candidate object; when the edit box is a circle, the computer device can use the center coordinates and the radius length as the position information of the candidate object.

[0053] In one embodiment, the computer device may acquire the acquired image frames by using a camera under the current field of view of the camera. The camera may be a camera built into the computer device, or an external camera associated with the computer device. The camera may be a monocular camera, a binocular camera, or an RGB-D (Red-Green-Blue-Deep) camera, etc.

[0054] In one embodiment, the computer device may specifically capture image frames at a fixed or dynamic frame rate to obtain the captured image frames. The fixed or dynamic frame rate enables the image frames to form a continuous dynamic picture when played at the fixed or dynamic frame rate, so that the computer device can track a specific target in the continuous dynamic picture.

[0055] In one embodiment, the computer device may obtain recorded video data or obtain real-time recorded video data. The computer device may extract image frames from the video data according to the frame rate of the video data.

[0056] S204, obtaining historical position information of an object to be compared in a previous image frame; the object to be compared includes at least one of a target object and a non-target object.

[0057] Among them, the previous image frame refers to an image frame whose acquisition time is before the acquisition time of the current image frame. The previous image frame can specifically be a previous image frame (such as the previous frame), or multiple previous image frames (such as the previous 5 frames). When the previous image frame is multiple previous image frames, the historical position information of the object to be compared in the previous image frame can be specifically determined based on the historical position information of the object to be compared in the previous multiple image frames, such as selecting the historical position information of the target object in the previous 5 frames with the highest frequency of occurrence as the historical position information of the target object, or determining the historical position information of the target object based on the average historical position of the target object in the previous 5 frames.

[0058] In one embodiment, the computer device may perform the above-mentioned target object tracking method on the image frames in sequence according to the acquisition time of the image frames in the image sequence, so that the computer device can track the target object in the continuous dynamic picture. The position information of the target object and the non-target object in the current image frame determined will be used as the historical position information when processing the next image frame. That is to say, for the previous image frame, the position information of the target object is determined; accordingly, other objects except the target object can be called non-target objects, and their position information is also determined.

[0059] In one embodiment, after determining the position information of the target object in the current image frame, the computer device may record the position information of the target object as the historical position information of the target object, and record the position of the non-target object as the historical position information of the non-target object.

[0060] In one embodiment, the target object tracking method also includes a step of determining the target object in the initial image frame, which specifically includes: when the current image frame is the initial image frame, obtaining initial position information of each of multiple candidate objects in the initial image frame; determining the area of ​​each of the multiple candidate objects in the initial image frame according to the initial position information; and using the initial position information of the candidate object with the largest area as the position information of the target object.

[0061] In one embodiment, when the current image frame is the initial image frame, the computer device may perform target detection on the initial image frame to determine the initial position information of each of the multiple candidate objects in the initial image frame. The computer device may determine the area of ​​the region where each candidate object is located based on the initial position information. The candidate object with the largest area is selected as the target object, that is, the initial position information of the candidate object with the largest area is used as the position information of the target object.

[0062] There are multiple calculation methods for determining the area of ​​each candidate object based on the initial position information. For example, when the candidate object is a bounding box of an object, the area of ​​the candidate object can be calculated based on the position coordinates of the bounding box. When the candidate object is an object with practical significance, the regular figure closest to the area of ​​the object can be located based on the position information of the candidate object, and then the area of ​​the regular figure can be calculated.

[0063] In specific scenarios, refer to Figure 3 , Figure 3 The figure shows a schematic diagram of an interface for determining a target hand in an embodiment. When a user wants to designate a hand as a target hand, the user only needs to extend the hand to be tracked to a close position in front of the camera and stay there for a while. This action can be regarded as an action to attract the camera's attention. The computer device can determine that the area of ​​the hand is the largest area from the currently captured image frame, and thus the hand can be directly used as the target hand.

[0064] In the above embodiment, when the current image frame is the initial image frame, the initial position information of the candidate object with the largest area is used as the position information of the target object. When the user interacts with the computer, the user's intention can be accurately identified and the target object can be quickly and accurately determined from multiple candidate objects.

[0065] In one embodiment, the computer device may identify the object type corresponding to the candidate object in the initial image frame, and use the candidate object corresponding to the preset object type as the target object. For example, when the target object is a target hand, when the current image frame is the initial image frame, the computer device may identify the gesture type corresponding to the candidate hand in the current image frame, and use the candidate hand corresponding to the preset gesture type as the target hand.

[0066] For example, when the preset gesture type is a "thumbs-up" gesture, that is, a thumbs-up gesture with the remaining fingers forming a fist, the computer device can identify the gesture types of all candidate hands in the image frame, and use the candidate hands with the "thumbs-up" gesture type as the target hand.

[0067] S206: Determine a location comparison parameter of each candidate object according to the current location information and the historical location information.

[0068] Among them, the position comparison parameter is a parameter used to compare the position relationship, which can be specifically the distance between the comparison objects, the area difference between the comparison objects, or the overlap between the comparison objects. Specifically, the computer device can determine the position comparison parameters of each candidate object based on the current position information and the historical position information. When the object to be compared is a target object, the computer device can determine the position comparison parameters between the candidate object and the target object; when the object to be compared is a non-target object, the computer device can determine the position comparison parameters between the candidate object and the non-target object; when the object to be compared is a target object and a non-target object, the computer device can determine the position comparison parameters between the candidate object and the target object, and between the non-target object, respectively.

[0069] S208: Exclude, from among the multiple candidate objects, the candidate objects whose position comparison parameters meet the interference object determination condition.

[0070] Among them, the interference object determination condition is a condition for determining that a candidate object is an interference object. When the object to be compared includes a target object, the interference object determination condition may be met specifically when the distance between the candidate object and the target object exceeds a distance threshold, or when the area overlap between the candidate object and the target object is less than a preset threshold, etc. When the object to be compared includes a non-target object, the interference object determination condition may be met specifically when the distance between the candidate object and the non-target object is less than a preset threshold, or when the area overlap between the candidate object and the non-target object is greater than a preset threshold, etc.

[0071] Specifically, after the computer device determines the location comparison parameter of each candidate object according to the current location information and the historical location information, the candidate object whose location comparison parameter meets the interference object determination condition may be excluded from the multiple candidate objects.

[0072] In one embodiment, the object to be compared includes a target object; step S206, i.e., the step of determining the position comparison parameter of each candidate object according to the current position information and the historical position information, includes: determining the distance between each candidate object and the target object according to the current position information and the historical position information of the target object. Step S208, i.e., the step of excluding the candidate objects whose position comparison parameters meet the interference object determination condition from the multiple candidate objects, includes: excluding the candidate objects whose distance exceeds the distance threshold from the multiple candidate objects.

[0073] In one embodiment, the computer device may traverse the current position information of multiple candidate objects in the current image frame, and determine the distance between each candidate object and the target object. There are many ways for the computer device to calculate the distance between the candidate object and the target object, which are not limited here. For example, the computer device may determine the center point coordinates of the candidate object and the target object respectively according to the position information, and calculate the distance between the candidate object and the target object according to the center point coordinates. For another example, the computer device may determine the coordinates of the candidate object and the target object at the same direction and angle, and calculate the distance between the candidate object and the target object according to the coordinates.

[0074] Considering that the time interval for collecting each frame of image is very short (for example, 30 frames of image can be collected in one second), the moving distance of the target object is also very short in a short time. Therefore, the computer device can exclude the candidate objects whose distance from the target object exceeds the distance threshold from the multiple candidate objects. That is, the long-distance candidate objects are excluded. The distance threshold can be a predetermined fixed distance or a dynamic distance, such as a preset N times (for example, 1.5 times) the length of one side of the target object.

[0075] In the above embodiment, the distance between each candidate object and the target object is determined based on the current position information and the historical position information of the target object, and then the candidate objects whose distance exceeds the distance threshold are excluded. This can quickly and accurately exclude the candidate objects at a long distance, greatly reducing the scope of target object determination.

[0076] In one embodiment, the object to be compared is included in the non-target object. Step S206, i.e., the step of determining the position comparison parameters of each candidate object according to the current position information and the historical position information, includes: determining the area overlap of each candidate object with the non-target object according to the current position information and the historical position information of the non-target object. Step S208, i.e., the step of excluding the candidate objects whose position comparison parameters meet the interference object determination condition from the multiple candidate objects, includes: excluding the candidate objects whose area overlap exceeds the overlap threshold from the multiple candidate objects.

[0077] In one embodiment, the computer device may traverse the current position information of multiple candidate objects in the current image frame and match it with the position information of previously recorded non-target objects. If the match is successful, the candidate object is determined to be a non-target object and may be excluded.

[0078] In one embodiment, the computer device may determine the area overlap between each candidate object and the non-target object based on the current position information and the historical position information of the non-target object. Based on the characteristic that the position of the non-target object in the image frames with similar acquisition time does not change much, when the area overlap exceeds the overlap threshold, it can be considered that the candidate object is successfully matched with the non-target object, that is, the possibility that the candidate object is a non-target object is very high. The computer device may exclude the candidate object whose area overlap exceeds the overlap threshold from multiple candidate objects.

[0079] In the above embodiment, the area overlap between each candidate object and the non-target object is determined respectively based on the current position information and the historical position information of the non-target object. From multiple candidate objects, the candidate objects whose area overlap exceeds the overlap threshold are excluded. This can quickly and accurately eliminate interfering objects and greatly reduce the scope of target object determination.

[0080] In one embodiment, when a computer device excludes candidate objects whose position comparison parameters meet the interference object determination conditions, it may exclude only distant objects determined based on the target object in the previous image frame; it may also exclude only interference objects determined based on non-target objects in the previous image frame; it may also exclude both distant objects and interference objects, wherein the order of excluding distant objects and excluding interference objects is not limited.

[0081] S210, determining the position information of the target object in the current image frame according to the elimination result.

[0082] Specifically, the computer device may determine the position information of the target object in the current image frame according to the result after excluding the candidate objects whose position comparison parameters meet the interference object determination condition.

[0083] In one embodiment, after the computer excludes candidate objects whose position comparison parameters meet the interference object determination condition, the target object can be screened out from the remaining candidate objects. Here, the remaining candidate objects can be called suspected target objects.

[0084] In one embodiment, after the computer excludes the candidate objects whose position comparison parameters meet the interference object determination conditions, a suspected candidate object is obtained. When there is only one suspected candidate object, the suspected candidate object can be directly used as the target object, that is, the position information of the suspected candidate object is used as the position information of the target object in the current image frame.

[0085] In one embodiment, after the computer excludes the candidate objects whose position comparison parameters meet the interference object determination conditions, multiple suspected candidate objects are obtained. At this time, the suspected target object with the highest matching degree with the target object in the previous image frame (such as the previous image frame) can be screened out from the multiple suspected candidate objects as the target object in the current image frame. Among them, the matching degree of the suspected object with the target object in the previous image frame can specifically be the matching degree of the position information of the suspected object and the historical position information of the target object, such as the area overlap, the distance size of the center position, or the matching degree of multiple position coordinates.

[0086] In one embodiment, after the computer excludes the candidate objects whose position comparison parameters meet the interference object determination conditions, there is no suspected target object, that is, all candidate objects are excluded. At this time, it is possible that the current camera has not captured the target object. The computer device can use the historical position information of the previous target object as the position information of the current image frame, or return to the initial state and re-designate a specific object as the target object.

[0087] The target object tracking method mentioned above, when tracking the target object in the current image frame, respectively compares the current position information of multiple candidate objects in the current image frame with the historical position information of the object to be compared in the previous image frame, thereby excluding the candidate objects whose position comparison parameters meet the interference object determination conditions. Among them, the object to be compared includes at least one of the target object and the non-target object. In this way, based on the characteristic that the position change of the candidate object between adjacent image frames of the image sequence is small, the interference object among the candidate objects can be efficiently and accurately excluded based on the historical position information. Then, the position information of the target object in the current image frame is determined based on the result of the exclusion. In this way, the position information of the target object is determined based on the remaining candidate objects after excluding the interference objects, which greatly reduces the determination range of the target object, can effectively avoid the interference of other objects in the image frame on the target object, can accurately find the position of the target object in a multi-object scene, and greatly improves the accuracy of target object tracking.

[0088] In one embodiment, the target object tracking method also includes the step of designating a specific object as the target object, which specifically includes: determining the areas of multiple candidate objects in the current image frame according to the current position information; when the ratio of the largest area to the second largest area is greater than or equal to a threshold, using the position information of the candidate object with the largest area as the position information of the target object; when the ratio of the largest area to the second largest area is less than a threshold, executing the step of obtaining the historical position information of the object to be compared in the previous image frame.

[0089] In one embodiment, the computer device can determine the area where each candidate object is located according to the current position information of multiple candidate objects in the current image frame, and then calculate the area of ​​each candidate object. When the current position information of the candidate object is its own position information, the corresponding area can be directly calculated; when the current position information of the candidate object is relative position information, the area where the candidate object is located can be first determined according to the relative position information, and then the area of ​​the area can be calculated. Among them, a variety of calculation methods can be used to determine the area of ​​each candidate object according to the current position information. For example, when the candidate object is a bounding box of an object, the area of ​​the candidate object can be calculated according to the position coordinates of the bounding box; when the candidate object is an object with practical significance, the regular figure closest to the area of ​​the object can be located according to the position information of the candidate object, and then the area of ​​the regular figure can be calculated.

[0090] Further, after the computer device determines the respective areas of the multiple candidate objects, the candidate objects can be sorted by area size. The candidate objects with the largest area and the second largest area are selected, and when the ratio of the largest area to the second largest area is greater than or equal to the threshold, the position information of the candidate object with the largest area is directly used as the position information of the target object. When the ratio of the largest area to the second largest area in the area is less than the threshold, step S202 and the corresponding subsequent steps are executed again. That is, when the ratio of the largest area to the second largest area is greater than or equal to the threshold, it can be considered that the user intends to switch the target object, and at this time, the candidate object with the largest area is directly used as the target object. When the ratio of the largest area to the second largest area is less than the threshold, the user's intention cannot be accurately judged at this time, so the target object can be determined by comparing the current position information with the historical position information of the object to be compared in the previous image frame.

[0091] In one embodiment, after the computer device determines the target object, it can record the position information of the target object as the historical position information of the target object in the subsequent image frame. The computer device can also record the position information of non-target objects in the current image frame as the historical position information of non-target objects in the subsequent image frame.

[0092] In the above embodiment, the areas of multiple candidate objects in the current image frame are determined respectively according to the current position information. When the ratio of the largest area to the second largest area exceeds a threshold, it can be considered that the user wishes to specify the object with the largest area as the target object when interacting with the computer. At this time, the candidate object with the largest area can be flexibly switched as the target object.

[0093] In one embodiment, step S210, that is, the step of determining the position information of the target object in the current image frame based on the result of exclusion, specifically includes: when the result of exclusion is that there is a suspected target object, determining the area overlap between each suspected target object and the target object based on the current position information of the suspected target object and the historical position information of the target object in the previous image frame; and using the position information of the suspected target object corresponding to the maximum area overlap as the position information of the target object in the current image frame.

[0094] In one embodiment, after the computer excludes the candidate objects whose position comparison parameters meet the interference object determination conditions, if there is still a suspected target object, the computer device can calculate the area overlap between the suspected target object and the previous target object based on the current position information of the suspected target object and the historical position information of the target object in the previous image frame. Sort by the size of the area overlap, and select the suspected target object with the largest area overlap as the target object in the current image. The previous image frame can specifically be the previous image frame or the previous two image frames.

[0095] Among them, considering that the target object does not change much in adjacent image frames, it is most accurate to use the suspected target object with the largest area overlap with the target object in the previous image frame as the target object in the current image frame. Alternatively, in another embodiment, the computer device may also use the suspected target object that is closest to the center position of the target object in the previous image frame as the target object in the current image frame; or comprehensively consider the matching of area overlap and center position.

[0096] In one embodiment, after the computer excludes candidate objects whose position comparison parameters meet the interference object determination conditions, if there is only one suspected target object, the position information of the suspected target object can be directly used as the position information of the target object in the current image frame.

[0097] In the above embodiment, when the result of exclusion is that there is a suspected target object, the suspected target object corresponding to the maximum area overlap between the suspected target object and the target object can be used as the target object in the current image frame, which is accurate and quick.

[0098] In one embodiment, step S210, that is, the step of determining the position information of the target object in the current image frame based on the result of exclusion, specifically includes: when the result of exclusion is that there is no suspected target object, searching for the previous image frame where the target object determined last time is located; when the acquisition time interval between the found previous image frame and the current image frame is less than the preset time interval, using the historical position information of the target object in the previous image frame as the position information of the target object in the current image frame.

[0099] In one embodiment, after the computer excludes the candidate objects whose position comparison parameters meet the interference object determination conditions, there is no suspected target object, that is, all candidate objects are excluded. At this time, it is possible that the current camera has not captured the target object. At this time, the computer device can search for the target object determined last time. When the acquisition time interval of the previous image frame where the target object determined last time is located is less than the preset time interval, the historical position information of the target object in the previous image frame is used as the position information of the target object in the current image frame.

[0100] The acquisition frequency of the image frame can be a fixed frequency or a dynamic frequency. When the acquisition frequency of the image frame is a fixed frequency, the acquisition time interval can be determined according to the acquisition time of the previous image frame and the current image frame, or according to the acquisition sequence of the previous image frame and the current image frame. When the acquisition frequency of the image frame is a dynamic frequency, the acquisition time interval can be determined according to the acquisition time of the previous image frame and the current image frame.

[0101] Furthermore, when the acquisition time interval between the found previous image frame and the current image frame is less than a preset time interval, the computer device may use the historical position information of the target object in the previous image frame as the position information of the target object in the current image frame.

[0102] Among them, the acquisition time interval between the previous image frame and the current image frame is less than the preset time interval. Specifically, the interval between the acquisition time of the previous image frame and the acquisition time of the current image frame is less than the preset time interval; it can also be that the previous image frame is the previous N image frames, where N is a preset value, such as the previous 5 image frames.

[0103] In the above embodiment, when the result of exclusion is that there is no suspected target object, since the position of the target object does not change much in a very short time, the historical position information of the target object in the previous image frame with a collection time interval less than the preset time interval can be used as the position information of the target object in the current image frame.

[0104] In one embodiment, the target object tracking method also includes a step for processing target object tracking failure, which specifically includes: when the acquisition time interval between the previous image frame and the current image frame is greater than or equal to a preset time interval, it is determined that the tracking of the target object in the current image frame has failed; the area of ​​each of multiple candidate objects in the next image frame is obtained; and the position information of the candidate object with the largest area is used as the position information of the target object in the next image frame.

[0105] In one embodiment, when the acquisition time interval between the previous image frame and the current image frame reaches a preset time interval, it is determined that the target object tracking in the current image frame fails. At this time, the computer device cannot determine the target object in the current image frame. In this case, the computer device can read the next image frame and perform target object tracking processing on the next image frame.

[0106] In one embodiment, the computer device may obtain the area of ​​each of the multiple candidate objects in the next image frame, and use the position information of the candidate object with the largest area as the position information of the target object in the next image frame. Alternatively, the computer device may identify the object type corresponding to the candidate object in the next image frame, and use the candidate object corresponding to the preset object type as the target object. For example, when the target object is a target hand, the gesture type corresponding to the candidate hand in the next image frame may be identified, and the candidate hand corresponding to the preset gesture type may be used as the target hand.

[0107] In the above embodiment, when the acquisition time interval between the previous image frame and the current image frame is greater than or equal to the preset time interval, it is determined that the target object tracking in the current image frame fails, and the target object needs to be searched again. When the target object tracking fails, the candidate object with the largest area in the next image frame can be used as the target object in the next image frame, so that when the user interacts with the computer, the user's intention can be accurately identified, further improving the accuracy of the target object tracking.

[0108] In one embodiment, the target object tracking method also includes the step of displaying a virtual object, which specifically includes: determining a designated position corresponding to the target object in the current image frame based on the position information of the target object; and displaying a virtual object corresponding to the target object at the designated position.

[0109] The designated position is a pre-set position corresponding to the target object, which may be a position adjacent to the target object, a position opposite to the target object, or a position at a preset distance from the target object, etc., which is not limited in this application. A virtual object is a computer data object, which may be a virtual object synthesized by a computer device, such as a virtual pendant, a virtual pattern, or a virtual character.

[0110] In one embodiment, for each frame of image, when the computer device determines the position information of the target object, a designated position corresponding to the target object in the current image frame may be determined. Further, for each frame of image, when the computer device determines the designated position in the current image frame, a virtual object corresponding to the target object may be displayed at the designated position. In this way, the virtual object can move along with the movement of the target object.

[0111] Among them, the virtual object can be used to highlight the target object, or the virtual object can be combined with the target object to form an image with complete meaning and interest, so that the interest of the user's interaction with the computer can be greatly improved. For example, when the target object is a target hand, the gesture of the target hand is a "single-hand heart" gesture, at this time, a virtual "heart" pattern can be displayed at the corresponding position of the target hand, so that the user and the computer can interact.

[0112] In the above embodiment, based on the position information of the target object, the designated position corresponding to the target object in the current image frame is determined, and a virtual object corresponding to the target object is displayed at the designated position. The virtual object can be displayed to move with the movement of the target object, which greatly improves the fun and timeliness of interaction with the computer.

[0113] In one embodiment, the target object is a target hand. The target object tracking method further includes a step of triggering a preset operation, which specifically includes: determining a gesture type corresponding to the target hand in the current image frame; and triggering a preset operation configured corresponding to the determined gesture type.

[0114] In one embodiment, the computer device may identify the gesture type corresponding to the target hand in the current image frame, and when the gesture type matches a preset gesture type, a preset operation configured corresponding to the determined gesture type may be triggered.

[0115] For example, see Figure 4 , Figure 4 FIG. 1 is a schematic diagram of identifying the gesture type corresponding to the target hand in one embodiment. Figure 4 , according to the above target object tracking method, it can be identified Figure 4 The hand with the largest area in the middle is the target hand. At the same time, it can be determined that the gesture type of the target hand is a "fist" type, so the computer device can trigger a preset operation configured corresponding to the "fist" type, such as playing boxing music, or displaying a virtual fist near the target hand that hits the target hand.

[0116] In one embodiment, the target object is a target hand. When the computer device recognizes that the gesture category corresponding to the target hand is a preset category, the computer device may display a virtual object corresponding to the target object at a designated position corresponding to the target object in the current image frame.

[0117] In one embodiment, when tracking a target object, the computer device may determine the motion trajectory of the target object based on a plurality of image frames, and when a motion trend is detected to be a preset trajectory, a preset operation of the corresponding configuration is executed.

[0118] For example, when the current scene is to display multiple objects in sequence, when the motion trajectory of the target object is tracked to be sliding from left to right, the computer device can adjust the currently displayed objects accordingly according to the sliding order.

[0119] In the above embodiment, when the target object is a target hand, the preset operation of the corresponding configuration can be triggered according to the gesture type corresponding to the target hand in the current image frame, which greatly improves the flexibility of interaction between the user and the computer.

[0120] like Figure 5 As shown, in a specific embodiment, the target object tracking method specifically includes the following steps:

[0121] S502: When the current image frame is an initial image frame, obtain initial position information of each of a plurality of candidate objects in the initial image frame.

[0122] S504: Determine the areas of the plurality of candidate objects in the initial image frame according to the initial position information.

[0123] S506: Using the initial position information of the candidate object with the largest area as the position information of the target object.

[0124] S508: Obtain current position information of each of the multiple candidate objects in the current image frame.

[0125] S510: Determine the areas of a plurality of candidate objects in the current image frame according to the current position information.

[0126] S512: When the ratio of the largest area to the second largest area is greater than or equal to a threshold, the position information of the candidate object with the largest area is used as the position information of the target object.

[0127] S514, when the ratio of the largest area to the second largest area in the area is less than a threshold, obtaining historical position information of the object to be compared in the previous image frame; the object to be compared includes the target object and the non-target object.

[0128] S516: Determine the area overlap between each candidate object and the non-target object according to the current position information and the historical position information of the non-target object.

[0129] S518: Eliminate, from among the multiple candidate objects, candidate objects whose area overlap exceeds an overlap threshold.

[0130] S520: Determine the distance between each candidate object and the target object according to the current position information and the historical position information of the target object.

[0131] S522: Eliminate, from among the multiple candidate objects, candidate objects whose distances exceed a distance threshold.

[0132] S524: When the result of elimination is that there is a suspected target object, the area overlap between each suspected target object and the target object is determined according to the current position information of the suspected target object and the historical position information of the target object in the previous image frame.

[0133] S526: Use the position information of the suspected target object corresponding to the maximum area overlap as the position information of the target object in the current image frame.

[0134] S528: When the result of elimination is that there is no suspected target object, search for the previous image frame where the target object determined last time is located.

[0135] S530: When the acquisition time interval between the found previous image frame and the current image frame is less than the preset time interval, the historical position information of the target object in the previous image frame is used as the position information of the target object in the current image frame.

[0136] S532: When the acquisition time interval between the found previous image frame and the current image frame is greater than or equal to the preset time interval, it is determined that the tracking of the target object in the current image frame fails.

[0137] S534, obtaining the respective areas of multiple candidate objects in the next image frame.

[0138] S536: Use the position information of the candidate object with the largest area as the position information of the target object in the next image frame.

[0139] The target object tracking method mentioned above, when tracking the target object in the current image frame, respectively compares the current position information of multiple candidate objects in the current image frame with the historical position information of the object to be compared in the previous image frame, thereby excluding the candidate objects whose position comparison parameters meet the interference object determination conditions. Among them, the object to be compared includes at least one of the target object and the non-target object. In this way, based on the characteristic that the position change of the candidate object between adjacent image frames of the image sequence is small, the interference object among the candidate objects can be efficiently and accurately excluded based on the historical position information. Then, the position information of the target object in the current image frame is determined based on the result of the exclusion. In this way, the position information of the target object is determined based on the remaining candidate objects after excluding the interference objects, which greatly reduces the determination range of the target object, can effectively avoid the interference of other objects in the image frame on the target object, can accurately find the position of the target object in a multi-object scene, and greatly improves the accuracy of target object tracking.

[0140] Figure 5 FIG. 1 is a flow chart of a target object tracking method in one embodiment. It should be understood that although Figure 5The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 5 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0141] like Figure 6 As shown, in one embodiment, a target object tracking method is provided. This embodiment mainly applies the method to the above Figure 1 The computer device (terminal 110 or server 120) in FIG. Figure 6 , the target object tracking method specifically includes the following steps:

[0142] S602: Obtain respective areas of a plurality of candidate objects in the current image frame.

[0143] Specifically, the computer device may perform target detection on the current image frame, determine the current position information of the candidate object in the current image frame, and then calculate the respective areas of the plurality of candidate objects according to the position information of the candidate objects.

[0144] In one embodiment, when the current position information of the candidate object is its own position information, the corresponding area can be directly calculated; when the current position information of the candidate object is relative position information, the area where the candidate object is located can be first determined based on the relative position information, and then the area of ​​the area can be calculated. Among them, a variety of calculation methods can be used to determine the area of ​​each candidate object based on the current position information of the candidate object. For example, when the candidate object is a bounding box of an object, the area of ​​the candidate object can be calculated based on the position coordinates of the bounding box. When the candidate object is an object with practical significance, the regular figure closest to the area of ​​the object can be located based on the position information of the candidate object, and then the area of ​​the regular figure can be calculated.

[0145] S604: When the ratio of the largest area to the second largest area is greater than or equal to a threshold, the position information of the candidate object with the largest area is used as the position information of the target object.

[0146] In one embodiment, after the computer device determines the respective areas of multiple candidate objects, the candidate objects may be sorted by area size. The candidate objects with the largest area and the second largest area are selected, and when the ratio of the largest area to the second largest area is greater than or equal to a threshold, the position information of the candidate object with the largest area is directly used as the position information of the target object. When the ratio of the largest area to the second largest area is greater than or equal to a threshold, it can be considered that the user intends to switch the target object, and in this case, the candidate object with the largest area is directly used as the target object.

[0147] In one embodiment, after the computer device determines the target object, it can record the position information of the target object as the historical position information of the target object in the subsequent image frame. The computer device can also record the position information of non-target objects in the current image frame as the historical position information of non-target objects in the subsequent image frame.

[0148] S606: When the ratio of the largest area to the second largest area among the areas is less than a threshold, exclude the candidate objects that meet the interference object determination condition from the multiple candidate objects.

[0149] In one embodiment, when the ratio of the largest area to the second largest area is less than a threshold, the user's intention cannot be accurately determined. The computer device may exclude candidate objects that meet the interference object determination criteria from multiple candidate objects, and then determine the target object.

[0150] In one embodiment, the computer device may obtain historical position information of an object to be compared in a previous image frame; the object to be compared includes at least one of a target object and a non-target object. The position comparison parameter of each candidate object is determined based on the current position information and the historical position information, and the candidate object whose position comparison parameter meets the interference object determination condition is excluded from multiple candidate objects. The relevant steps of excluding the candidate object whose position comparison parameter meets the interference object determination condition from multiple candidate objects can be referred to the description in the aforementioned embodiment, and will not be repeated here.

[0151] S608: Determine the position information of the target object in the current image frame according to the elimination result.

[0152] Specifically, the computer device may screen out the target object from the remaining candidate objects based on the result after excluding the candidate objects whose position comparison parameters meet the interference object determination condition. Here, the remaining candidate objects may be referred to as suspected target objects.

[0153] In one embodiment, after the computer excludes the candidate objects whose position comparison parameters meet the interference object determination conditions, a suspected candidate object is obtained. When there is only one suspected candidate object, the suspected candidate object can be directly used as the target object, that is, the position information of the suspected candidate object is used as the position information of the target object in the current image frame.

[0154] In one embodiment, after the computer excludes the candidate objects whose position comparison parameters meet the interference object determination conditions, multiple suspected candidate objects are obtained. At this time, the suspected target object with the highest matching degree with the target object in the previous image frame (such as the previous image frame) can be screened out from the multiple suspected candidate objects as the target object in the current image frame. Among them, the matching degree of the suspected object with the target object in the previous image frame can specifically be the matching degree of the position information of the suspected object and the historical position information of the target object, such as the area overlap, the distance size of the center position, or the matching degree of multiple position coordinates.

[0155] In one embodiment, after the computer excludes the candidate objects whose position comparison parameters meet the interference object determination conditions, there is no suspected target object, that is, all candidate objects are excluded. At this time, it is possible that the current camera has not captured the target object. The computer device can use the historical position information of the previous target object as the position information of the current image frame, or return to the initial state and re-designate a specific object as the target object.

[0156] The target object tracking method, when tracking the target object in the current image frame, obtains the area of ​​each of the multiple candidate objects in the current image frame. When the ratio of the maximum area to the second largest area in the area exceeds the threshold, it can be considered that the user expects to specify the object with the largest area as the target object when interacting with the computer. At this time, the candidate object with the largest area can be flexibly switched as the target object. When the ratio of the maximum area to the second largest area in the area is less than the threshold, the candidate object that meets the interference object determination condition is excluded from the multiple candidate objects, and then the position information of the target object in the current image frame is determined according to the result of the exclusion. In this way, the user's intention can be accurately identified by the ratio of the maximum area to the second largest area in the candidate object, and the target object can be flexibly switched. When the user's intention is unclear, the interference of other objects in the image frame to the target object can also be effectively avoided by exclusion, and the position of the target object can be accurately found in a multi-object scene, which greatly improves the accuracy of tracking the target object.

[0157] In one embodiment, the target object is a target hand as an example to describe the specific content of the target object tracking method in detail. Figure 7 , Figure 7 FIG. 2 shows a flow chart of a target object tracking method in one embodiment. Figure 7As shown, the specific process of the target object tracking method is as follows:

[0158] Step (1): Read the next image frame, detect the positions of all hands in it, and jump to step (3).

[0159] Step (2): Read the next image frame, detect the positions of all hands in it, and jump to step (4).

[0160] Step (3): In the initial stage, determine the detection result of the target detection on the image frame. If no hand is detected, jump to step (1); if a hand is detected, select the hand with the largest area as the target hand, record the position information of the target hand as the historical position information of the target hand, and record the position information of other hands as the historical position information of the interference hands, and jump to step (2).

[0161] Step (4): In the tracking phase, the areas of all hands in the image frame are first calculated, and the candidate hands are sorted by area. The larger hand is judged, that is, the area ratio of the largest hand to the second largest hand among the candidate hands is compared with the preset threshold. The specific judgment rules are as follows:

[0162] Step (4a): When the area ratio of the hand with the largest area to the hand with the second largest area exceeds a preset threshold, the hand with the largest area is switched to be the target hand, and the position information of the target hand is recorded as the historical position information of the target hand. At the same time, the position information of the other hands is recorded as the position information of the interference hand, and jump to step (2).

[0163] Step (4a): When the area ratio of the hand with the largest area to the hand with the second largest area does not exceed the preset threshold, jump to step (5).

[0164] Step (5): Record the positions of all candidate hands in the image frame as candidate positions, and then perform the following processing:

[0165] Step (5a): Traverse the position information of all hands in the candidate positions and match them with the position information of the interference hand recorded previously. If the area overlap is greater than the preset threshold, the hand is determined to be an interference hand, and the hand is deleted from the candidate hands, and jump to step (5b).

[0166] Step (5b): Traverse the positions of all hands in the candidate positions at this time, and calculate the center distance between the candidate hand and the previous historical position of the target hand. If the center distance exceeds the distance threshold, the hand is judged as a non-target hand, deleted from the candidate hands, and jump to step (5c).

[0167] Step (5c): There are two situations at this time. The first situation is that the candidate position is empty at this time, then check the previous historical position information of the target hand. If the previous historical position information is the position 5 frames ago, that is, the acquisition time interval between the previous image frame where the target hand is located and the current image frame is greater than the preset time interval, it means that the tracking of the target hand has failed, and the target hand needs to be found again, and jump to step (1). If the previous historical position is within 5 frames, that is, the acquisition time interval between the previous image frame where the target hand is located and the current image frame is less than the preset time interval, it may be that the current target detection algorithm has not detected the position of the target hand, so the previous historical position information of the target hand is used as the position information of the target hand in the current image frame, and jump to step (2). The second situation is that the candidate position is not empty at this time, then jump to step (5d).

[0168] Step (5d): Traverse the position information of all hands in the candidate position at this time, and calculate the area overlap with the previous historical position information of the target hand. And sort them by area overlap, and select the hand with the largest area overlap as the target hand in the current image frame. The position information of this hand is used as the historical position information of the target hand, and the position information of other hands is recorded as the historical position information of the interference hand, and jump to step (2).

[0169] like Figure 8 As shown, in one embodiment, a target object tracking device 800 is provided, including an acquisition module 801 , a determination module 802 and an exclusion module 803 .

[0170] The acquisition module 801 is used to acquire the current position information of each of the multiple candidate objects in the current image frame.

[0171] The acquisition module 801 is further used to acquire the historical position information of the object to be compared in the previous image frame; the object to be compared includes at least one of a target object and a non-target object.

[0172] The determination module 802 is used to determine the location comparison parameter of each candidate object according to the current location information and the historical location information.

[0173] The exclusion module 803 is used to exclude, from among the multiple candidate objects, the candidate objects whose position comparison parameters meet the interference object determination condition.

[0174] The determination module 802 is further configured to determine the position information of the target object in the current image frame according to the exclusion result.

[0175] In one embodiment, the acquisition module 801 is further used to acquire the initial position information of each of the multiple candidate objects in the initial image frame when the current image frame is the initial image frame. The determination module 802 is further used to determine the area of ​​each of the multiple candidate objects in the initial image frame according to the initial position information; and use the initial position information of the candidate object with the largest area as the position information of the target object.

[0176] In one embodiment, the determination module 802 is further used to determine the areas of the multiple candidate objects in the current image frame according to the current position information; when the ratio of the largest area to the second largest area in the area is greater than or equal to the threshold, the position information of the candidate object with the largest area is used as the position information of the target object. The acquisition module 801 is further used to acquire the historical position information of the object to be compared in the previous image frame when the ratio of the largest area to the second largest area in the area is less than the threshold.

[0177] In one embodiment, the determination module 802 is also used to determine the area overlap between each suspected target object and the target object according to the current position information of the suspected target object and the historical position information of the target object in the previous image frame when the result of exclusion is that there is a suspected target object; and use the position information of the suspected target object corresponding to the maximum area overlap as the position information of the target object in the current image frame.

[0178] In one embodiment, the target object tracking device 800 further includes a search module 804, which is used to search for the previous image frame where the target object determined last time is located when the result of the elimination is that there is no suspected target object. The determination module 802 is also used to use the historical position information of the target object in the previous image frame as the position information of the target object in the current image frame when the acquisition time interval between the previous image frame found and the current image frame is less than a preset time interval.

[0179] In one embodiment, the determination module 802 is further used to determine that the target object tracking in the current image frame fails when the acquisition time interval between the previous image frame and the current image frame is greater than or equal to a preset time interval. The acquisition module 801 is further used to obtain the area of ​​each of the multiple candidate objects in the next image frame. The determination module 802 is further used to use the position information of the candidate object with the largest area as the position information of the target object in the next image frame.

[0180] In one embodiment, the determination module 802 is further used to determine the distance between each candidate object and the target object according to the current location information and the historical location information of the target object. The exclusion module 803 is further used to exclude candidate objects whose distance exceeds a distance threshold from multiple candidate objects.

[0181] In one embodiment, the determination module 802 is further used to determine the area overlap between each candidate object and the non-target object according to the current position information and the historical position information of the non-target object. The exclusion module 803 is further used to exclude candidate objects whose area overlap exceeds the overlap threshold from multiple candidate objects.

[0182] In one embodiment, the target object tracking device 800 further includes a display module 805. The determination module 802 is further configured to determine a designated position corresponding to the target object in the current image frame according to the position information of the target object. The display module 805 is configured to display a virtual object corresponding to the target object at the designated position.

[0183] like Fig. 9 As shown, in one embodiment, the target object tracking device 800 further includes a trigger module 806. The determination module 802 is further used to determine the gesture type corresponding to the target hand in the current image frame. The trigger module 806 is used to trigger a preset operation configured corresponding to the determined gesture type.

[0184] The target object tracking device, when tracking the target object in the current image frame, compares the current position information of each of the multiple candidate objects in the current image frame with the historical position information of the object to be compared in the previous image frame, thereby excluding the candidate objects whose position comparison parameters meet the interference object determination conditions. Among them, the object to be compared includes at least one of the target object and the non-target object. In this way, based on the characteristic that the position change of the candidate object between adjacent image frames of the image sequence is small, the interference object among the candidate objects can be efficiently and accurately excluded based on the historical position information. Then, the position information of the target object in the current image frame is determined based on the result of the exclusion. In this way, the position information of the target object is determined based on the candidate objects remaining after the interference objects are excluded, which greatly reduces the determination range of the target object, can effectively avoid the interference of other objects in the image frame on the target object, can accurately find the position of the target object in a multi-object scene, and greatly improves the accuracy of target object tracking.

[0185] like Fig.10 As shown, in one embodiment, a target object tracking device 1000 is provided, including an acquisition module 1001 , a determination module 1002 and an exclusion module 1003 .

[0186] The acquisition module 1001 is used to acquire the areas of multiple candidate objects in the current image frame.

[0187] The determination module 1002 is configured to use the position information of the candidate object with the largest area as the position information of the target object when the ratio of the largest area to the second largest area among the areas is greater than or equal to a threshold.

[0188] The exclusion module 1003 is used to exclude candidate objects that meet the interference object determination condition from multiple candidate objects when the ratio of the largest area to the second largest area in the area is less than a threshold.

[0189] The determination module 1002 is further configured to determine the position information of the target object in the current image frame according to the elimination result.

[0190] The target object tracking device, when tracking the target object in the current image frame, obtains the area of ​​each of the multiple candidate objects in the current image frame. When the ratio of the maximum area to the second largest area in the area exceeds the threshold, it can be considered that the user expects to specify the object with the largest area as the target object when interacting with the computer. At this time, the candidate object with the largest area can be flexibly switched as the target object. When the ratio of the maximum area to the second largest area in the area is less than the threshold, the candidate object that meets the interference object determination condition is excluded from the multiple candidate objects, and then the position information of the target object in the current image frame is determined according to the result of the exclusion. In this way, the user's intention can be accurately identified by the ratio of the maximum area to the second largest area in the candidate object, and the target object can be flexibly switched. When the user's intention is unclear, the interference of other objects in the image frame to the target object can also be effectively avoided by exclusion, and the position of the target object can be accurately found in a multi-object scene, which greatly improves the accuracy of tracking the target object.

[0191] Fig.11 The internal structure diagram of a computer device in one embodiment is shown. The computer device may specifically be Figure 1 The terminal 110 in FIG. Fig.11 As shown, the computer device includes a processor, a memory, a network interface, an image acquisition device and a display screen connected through a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program, which, when executed by the processor, enables the processor to implement a target object tracking method. The internal memory may also store a computer program, which, when executed by the processor, enables the processor to execute a target object tracking method. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the image acquisition device of the computer device may be a camera.

[0192] Those skilled in the art will understand that Fig.11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0193] In one embodiment, the target object tracking device provided by the present application can be implemented in the form of a computer program. The computer program can be Fig.11 The computer device is run on the computer device shown. The memory of the computer device can store various program modules constituting the target object tracking device, such as: Figure 8 or Fig.10 The acquisition module, determination module and exclusion module shown in the figure. The computer program composed of various program modules enables the processor to execute the steps of the target object tracking method of each embodiment of the present application described in this specification.

[0194] For example, Fig.11 The computer device shown can be Figure 8 The acquisition module in the target object tracking device shown executes step S202 and step S204. The computer device may execute step S206 and step S210 through the determination module. The computer device may execute step S208 through the exclusion module.

[0195] For example, Fig.11 The computer device shown can be Fig.10 The acquisition module in the target object tracking device shown executes step S602. The computer device may execute steps S604 and S608 through the determination module. The computer device may execute step S606 through the exclusion module.

[0196] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the target object tracking method described above. The steps of the target object tracking method here may be the steps of the target object tracking method in each of the above embodiments.

[0197] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the processor executes the steps of the target object tracking method described above. The steps of the target object tracking method here can be the steps of the target object tracking method in each of the above embodiments.

[0198] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0199] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0200] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A target object tracking method, comprising: Obtaining current position information of each of multiple candidate objects in the current image frame; Determine the respective areas of a plurality of candidate objects in the current image frame according to the current position information; When the ratio of the largest area to the second largest area among the areas is greater than or equal to a threshold, taking the position information of the candidate object with the largest area as the position information of the target object; When the ratio of the largest area to the second largest area in the areas is less than the threshold, acquiring historical position information of the object to be compared in the previous image frame; the object to be compared includes at least one of a target object and a non-target object; Determine a location comparison parameter for each candidate object according to the current location information and the historical location information; Excluding, from among the plurality of candidate objects, candidate objects whose position comparison parameters meet the interference object determination condition; Determine the position information of the target object in the current image frame according to the exclusion result, and record the position information of the target object and non-target objects in the current image frame.

2. The method according to claim 1, characterized in that: The method further comprises: When the current image frame is an initial image frame, obtaining initial position information of each of a plurality of candidate objects in the initial image frame; Determine the respective areas of a plurality of candidate objects in the initial image frame according to the initial position information; The initial position information of the candidate object with the largest area is used as the position information of the target object.

3. The method according to claim 1, characterized in that The step of determining the position information of the target object in the current image frame according to the result of the exclusion includes: When the result of exclusion is that there is a suspected target object, determining the area overlap between each suspected target object and the target object according to the current position information of the suspected target object and the historical position information of the target object in the previous image frame; The position information of the suspected target object corresponding to the maximum area overlap is used as the position information of the target object in the current image frame.

4. The method according to claim 1, characterized in that: The step of determining the position information of the target object in the current image frame according to the result of the exclusion includes: When the result of the exclusion is that there is no suspected target object, searching for the previous image frame where the target object determined last time is located; When the acquisition time interval between the found previous image frame and the current image frame is less than the preset time interval, the historical position information of the target object in the previous image frame is used as the position information of the target object in the current image frame.

5. The method according to claim 4, characterized in that The method further comprises: When the acquisition time interval between the found previous image frame and the current image frame is greater than or equal to the preset time interval, it is determined that the tracking of the target object in the current image frame fails; Obtaining respective areas of a plurality of candidate objects in the next image frame; The position information of the candidate object with the largest area is used as the position information of the target object in the next image frame.

6. The method according to claim 1, characterized in that The object to be compared includes a target object; and determining the location comparison parameters of each candidate object according to the current location information and the historical location information includes: Determine the distance between each candidate object and the target object according to the current position information and the historical position information of the target object; The step of excluding, from among the plurality of candidate objects, candidate objects whose position comparison parameters meet the interference object determination condition comprises: From the plurality of candidate objects, the candidate objects whose distance exceeds a distance threshold are excluded.

7. The method according to claim 1, characterized in that The object to be compared includes a non-target object; and determining the location comparison parameter of each candidate object according to the current location information and the historical location information includes: Determine the area overlap between each candidate object and the non-target object according to the current position information and the historical position information of the non-target object; The step of excluding, from among the plurality of candidate objects, candidate objects whose position comparison parameters meet the interference object determination condition comprises: From the plurality of candidate objects, the candidate objects whose area overlap exceeds an overlap threshold are excluded.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Determining a designated position corresponding to the target object in the current image frame according to the position information of the target object; A virtual object corresponding to the target object is displayed at the designated position.

9. The method according to any one of claims 1 to 7, characterized in that: The target object is a target hand; the method further comprises: Determining a gesture type corresponding to the target hand in the current image frame; A preset operation configured corresponding to the determined gesture type is triggered.

10. A target object tracking device, characterized in that: The device comprises: An acquisition module, used to acquire current position information of each of a plurality of candidate objects in a current image frame; A determination module, configured to determine the areas of a plurality of candidate objects in the current image frame according to the current position information; The determination module is further configured to use the position information of the candidate object with the largest area as the position information of the target object when the ratio of the largest area to the second largest area among the areas is greater than or equal to a threshold; The acquisition module is further configured to acquire historical position information of an object to be compared in a previous image frame when the ratio of the largest area to the second largest area in the areas is less than the threshold; the object to be compared includes at least one of a target object and a non-target object; A determination module, used to determine a location comparison parameter of each candidate object according to the current location information and the historical location information; An exclusion module, used to exclude, from a plurality of candidate objects, candidate objects whose position comparison parameters meet the interference object determination condition; The determination module is further used to determine the position information of the target object in the current image frame according to the exclusion result, and record the position information of the target object and non-target objects in the current image frame.

11. The device according to claim 10, characterized in that The acquisition module is further used to acquire initial position information of each of the multiple candidate objects in the initial image frame when the current image frame is the initial image frame; The determination module is further used to determine the areas of the multiple candidate objects in the initial image frame according to the initial position information; and use the initial position information of the candidate object with the largest area as the position information of the target object.

12. The device according to claim 10, characterized in that The determining module is further used for: When the result of exclusion is that there is a suspected target object, determining the area overlap between each suspected target object and the target object according to the current position information of the suspected target object and the historical position information of the target object in the previous image frame; The position information of the suspected target object corresponding to the maximum area overlap is used as the position information of the target object in the current image frame.

13. The device according to claim 10, characterized in that The device also includes a search module, wherein: The search module is used to search for the previous image frame where the target object determined last time is located when the result of exclusion is that there is no suspected target object; The determination module is further configured to use the historical position information of the target object in the previous image frame as the position information of the target object in the current image frame when the acquisition time interval between the previous image frame and the current image frame is less than a preset time interval.

14. The device according to claim 13, characterized in that The determination module is further configured to determine that the target object in the current image frame fails to be tracked when the acquisition time interval between the found previous image frame and the current image frame is greater than or equal to a preset time interval; The acquisition module is further used to acquire the area of ​​each of the plurality of candidate objects in the next image frame; The determination module is further configured to use the position information of the candidate object with the largest area as the position information of the target object in the next image frame.

15. The device according to claim 10, characterized in that The object to be compared includes a target object; The determination module is further used to determine the distance between each candidate object and the target object according to the current position information and the historical position information of the target object; The exclusion module is further used to exclude, from multiple candidate objects, candidate objects whose distance exceeds a distance threshold.

16. The device according to claim 10, characterized in that The object to be compared includes a non-target object; The determination module is further used to determine the area overlap between each candidate object and the non-target object according to the current position information and the historical position information of the non-target object; The exclusion module is further used to exclude, from multiple candidate objects, candidate objects whose area overlap exceeds an overlap threshold.

17. The device according to any one of claims 10 to 16, characterized in that The device also includes a display module, wherein: The determination module is used to determine a designated position corresponding to the target object in the current image frame according to the position information of the target object; The display module is used to display a virtual object corresponding to the target object at the designated position.

18. The device according to any one of claims 10 to 16, characterized in that The target object is a target hand; the device also includes a trigger module, wherein: The determination module is further used to determine the gesture type corresponding to the target hand in the current image frame; The trigger module is used to trigger a preset operation configured corresponding to the determined gesture type.

19. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 9.

20. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 9.

Images