A target object tracking method, device and storage medium

Abstract

This invention discloses a target object tracking method, apparatus, and storage medium. In this embodiment, when tracking a target object, the target object to be tracked, along with its target character feature information and auxiliary feature information, are first determined. Candidate character region images and candidate auxiliary region images corresponding to each object are determined in the image to identify the corresponding candidate character feature information and candidate auxiliary feature information for each object. Finally, the target object is identified and tracked from each object based on the candidate character feature information and candidate auxiliary feature information. This method offers good real-time performance in video viewing scenarios and improves the accuracy of target tracking by introducing target character feature information and target auxiliary feature information to jointly identify and track the target object.

Description

Technical Field

[0001] This invention relates to the field of video processing technology, and in particular to a target object tracking method, apparatus, and storage medium. Background Technology

[0002] Many applications in smart living scenarios utilize video-based target object tracking, such as pedestrian trajectory tracking and video person localization. Especially in video viewing scenarios, users typically have a target object of interest, and to improve the user's viewing experience, video target object tracking is necessary.

[0003] In related technologies, target tracking is generally based on Person Re-identification (ReID) technology. ReID is a technique that uses computer vision to determine whether a specific pedestrian exists in an image or video sequence. It is widely considered a sub-problem of image retrieval. Given a surveillance image of a pedestrian, the goal is to retrieve images of that pedestrian across different devices. It aims to overcome the visual limitations of current fixed cameras and can be combined with pedestrian detection / tracking technologies, making it widely applicable in fields such as intelligent video surveillance and intelligent security. For example, in an area with video sequences captured by multiple cameras, ReID requires retrieving all images of a pedestrian of interest captured by one camera, as seen on other cameras.

[0004] The problems with ReID technology are that the algorithm is highly complex, its real-time performance is poor in video viewing scenarios, and the accuracy of target object tracking cannot be guaranteed. Summary of the Invention

[0005] This invention provides a target object tracking method, apparatus, and storage medium to address the problems in related technologies where target object tracking schemes have high algorithm complexity, poor real-time performance in video viewing scenarios, and the inability to guarantee the accuracy of target tracking.

[0006] This invention provides a target object tracking method, the method comprising:

[0007] Identify the target object to be tracked, and obtain the target character feature information and target auxiliary feature information of the target object;

[0008] Determine the candidate character region image and the candidate auxiliary region image outside the candidate character region image for each object in the image; and based on the candidate character region image and the candidate auxiliary region image for each object, determine the candidate character feature information and the candidate auxiliary feature information for each object.

[0009] The candidate character feature information and candidate auxiliary feature information corresponding to each object are matched with the target character feature information and target auxiliary feature information. The objects that are successfully matched are taken as target objects and tracked.

[0010] Further, determining the candidate character region image corresponding to each object in the image and the candidate auxiliary region image outside the candidate character region image includes:

[0011] The human detection algorithm is used to determine the candidate tracking region image corresponding to each object in the image.

[0012] For each candidate tracking region image, the candidate tracking region image is input into a pre-trained character segmentation model. Based on the character segmentation model, candidate character region images are determined in the candidate tracking region image. Images in the candidate tracking region image other than the candidate character region images are used as candidate auxiliary region images.

[0013] Furthermore, the step of determining the candidate tracking region image corresponding to each object in the image using a human detection algorithm includes:

[0014] By using human detection algorithms and pose estimation algorithms, candidate tracking regions that meet pose requirements are determined for each object in the image.

[0015] Furthermore, based on the candidate character region image corresponding to each object, the candidate character feature information corresponding to each object is determined, including:

[0016] For each object, the candidate character region image corresponding to the object is input into a pre-trained character recognition model, and the candidate character feature information corresponding to the object is determined based on the character recognition model.

[0017] Further, determining the candidate character feature information corresponding to the object based on the character recognition model includes:

[0018] Based on the alphabetic character submodule in the character recognition model, the first feature vector corresponding to the alphabetic character region image in the candidate character region image is determined.

[0019] Based on the digit sub-module in the character recognition model, the second feature vector corresponding to the digit contour in the candidate character region image is determined.

[0020] The first feature vector and the second feature vector are concatenated dimensionally, and the candidate character feature information corresponding to the concatenated feature vector is determined based on the character recognition submodule in the character recognition model.

[0021] Further, determining the candidate auxiliary feature information corresponding to each object based on the candidate auxiliary region image corresponding to each object includes:

[0022] For each object, according to a preset pixel selection rule, a preset number of pixels are selected from the candidate auxiliary region image corresponding to the object, and the color information or texture information of the preset number of pixels is used as the candidate auxiliary feature information corresponding to the object.

[0023] Furthermore, the training process of the character segmentation model includes:

[0024] For each sample tracking region image in the first training set, the sample tracking region image and the corresponding label information are input into the character segmentation model to train the character segmentation model; wherein, the label information includes the position information of the character region image in the sample tracking region image, and the position information of the character region image includes the position information of letter text regions and the position information of number contours.

[0025] Furthermore, the training process of the character recognition model includes:

[0026] For each sample character region image in the second training set, the sample letter character region image in the sample character region image is input into the letter character submodule in the character recognition model, and the first sample feature vector corresponding to the sample letter character region image is determined based on the letter character submodule.

[0027] The sample digit contour in the sample character region image is input into the digit submodule of the character recognition model, and the second sample feature vector corresponding to the sample digit contour is determined based on the digit submodule.

[0028] The first sample feature vector and the second sample feature vector are concatenated dimensionally, and the sample character feature information corresponding to the dimensionally concatenated sample feature vector is determined based on the character recognition submodule in the character recognition model.

[0029] Based on the sample character feature information and the real character feature information corresponding to the sample character region image, the alphabetic character submodule, the numeric submodule, and the character recognition submodule in the character recognition model are trained.

[0030] On the other hand, embodiments of the present invention provide a target object tracking device, the device comprising:

[0031] The acquisition module is used to determine the target object to be tracked and to acquire the target character feature information and target auxiliary feature information of the target object;

[0032] The determination module is used to determine the candidate character region image and the candidate auxiliary region image outside the candidate character region image for each object in the image; and based on the candidate character region image and the candidate auxiliary region image for each object, determine the candidate character feature information and the candidate auxiliary feature information for each object.

[0033] The tracking module is used to match the candidate character feature information and candidate auxiliary feature information corresponding to each object with the target character feature information and target auxiliary feature information, and to track the successfully matched objects as target objects.

[0034] In another aspect, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described above.

[0035] This invention provides a target object tracking method, apparatus, and storage medium. The method includes: determining a target object to be tracked, and acquiring target character feature information and target auxiliary feature information of the target object; determining a candidate character region image and a candidate auxiliary region image outside the candidate character region image for each object in an image; and determining candidate character feature information and candidate auxiliary feature information for each object based on the candidate character region image and candidate auxiliary region image for each object; matching the candidate character feature information and candidate auxiliary feature information for each object with the target character feature information and target auxiliary feature information, and tracking the successfully matched object as the target object.

[0036] The above technical solution has the following advantages or beneficial effects:

[0037] In this embodiment of the invention, when tracking a target object, the target object to be tracked, along with its target character feature information and target auxiliary feature information, are first determined. Candidate character region images and candidate auxiliary region images corresponding to each object are determined in the image to identify the candidate character feature information and candidate auxiliary feature information corresponding to each object. Finally, the target object is identified and tracked from each object based on the candidate character feature information and candidate auxiliary feature information. Compared to ReID technology, this method has lower algorithm complexity and higher target object recognition efficiency, thus offering better real-time performance in video viewing scenarios. Furthermore, the introduction of target character feature information and target auxiliary feature information to jointly identify and track the target object improves the accuracy of target tracking. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a schematic diagram of the target object tracking process provided in an embodiment of the present invention;

[0040] Figure 2 A flowchart for target object tracking provided in an embodiment of the present invention;

[0041] Figure 3 This is a schematic diagram of the player's jersey and player number area provided in an embodiment of the present invention;

[0042] Figure 4 This is a schematic diagram illustrating the optimized character recognition effect of the player character region image provided in an embodiment of the present invention;

[0043] Figure 5 A schematic diagram of the auxiliary region image provided in an embodiment of the present invention;

[0044] Figure 6 This is a schematic diagram of the character recognition process provided in an embodiment of the present invention;

[0045] Figure 7 This is a schematic diagram of the target object tracking device provided in an embodiment of the present invention;

[0046] Figure 8 This is a schematic diagram of the electronic device structure provided in an embodiment of the present invention. Detailed Implementation

[0047] The present invention will now be described in further detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0048] The target object tracking method provided in this invention can be applied to free-viewpoint viewing scenarios. During the viewing process, users may want the video perspective to follow their favorite player. However, in target object tracking applications, it has been found that once a user selects a target object of interest, the target object is easily lost if it is obscured or moves rapidly, resulting in a loss of the user's viewpoint and impacting the user experience. Based on these considerations, this invention proposes a scheme for quickly determining the target object to be tracked based on matching character feature information and auxiliary feature information, improving the real-time performance and accuracy of target object tracking.

[0049] Figure 1 This is a schematic diagram of a target object tracking process provided in an embodiment of the present invention. The process includes the following steps:

[0050] S101: Determine the target object to be tracked, and obtain the target character feature information and target auxiliary feature information of the target object.

[0051] S102: Determine the candidate character region image and the candidate auxiliary region image outside the candidate character region image for each object in the image; and based on the candidate character region image and the candidate auxiliary region image for each object, determine the candidate character feature information and the candidate auxiliary feature information for each object.

[0052] S103: Match the candidate character feature information and candidate auxiliary feature information corresponding to each object with the target character feature information and target auxiliary feature information, and take the successfully matched object as the target object and track it.

[0053] The target object tracking method provided in this embodiment of the invention is applied to electronic devices, such as PCs and tablets.

[0054] The electronic device first identifies the target object to be tracked. Specifically, in a free-viewpoint game viewing application, the electronic device provides a user interface displaying a list of all objects in the game. Users can select the target object from the list manually or via voice. Manually, the user clicks on a target object in the list, and the electronic device identifies the selected object based on the user's click. Voice-based tracking involves the user inputting voice information containing object identifiers, such as the object's name or jersey number. The electronic device acquires the user's voice input and analyzes it to determine the target object.

[0055] The electronic device stores character feature information and auxiliary feature information for each object. The character feature information includes the object's jersey character feature information, specifically alphanumeric character feature information and numeric character feature information. The auxiliary feature information includes the object's jersey color information or texture information. After identifying the target object to be tracked, the electronic device uses the target object's character feature information and auxiliary feature information as the target character feature information and target auxiliary feature information, respectively.

[0056] An electronic device acquires images from a video and determines candidate character region images and candidate auxiliary region images outside the candidate character region images for each object in the images. Specifically, a human detection algorithm can be used to determine a human detection box corresponding to each object in the image. These human detection boxes are then input into a pre-trained character segmentation model to obtain character region images within each human detection box. The images outside the character region images within the human detection boxes are used as auxiliary region images. In this embodiment of the invention, the obtained character region images within each human detection box and the auxiliary region images are referred to as the candidate character region images and candidate auxiliary region images outside the candidate character region images for each object, respectively.

[0057] To more accurately determine the candidate character region image and candidate auxiliary region image corresponding to each object, in this embodiment of the invention, determining the candidate character region image and the candidate auxiliary region image outside the candidate character region image corresponding to each object in the image includes:

[0058] The human detection algorithm is used to determine the candidate tracking region image corresponding to each object in the image.

[0059] For each candidate tracking region image, the candidate tracking region image is input into a pre-trained character segmentation model. Based on the character segmentation model, candidate character region images are determined in the candidate tracking region image. Images in the candidate tracking region image other than the candidate character region images are used as candidate auxiliary region images.

[0060] First, a human detection algorithm is used to determine the corresponding human detection box for each object in the image. The upper body region of the human body within the detection box is then cropped and used as a candidate tracking region image, thus reducing interference from candidate character region images. For each candidate tracking region image, this image is input into a pre-trained character segmentation model. Based on this model, candidate character region images are determined within the candidate tracking region images. Finally, images outside the candidate character region images within the candidate tracking region images are used as candidate auxiliary region images.

[0061] The candidate character feature information and candidate auxiliary feature information corresponding to each object are matched with the target character feature information and target auxiliary feature information. Objects that successfully match are designated as target objects and tracked. In other words, objects whose candidate character feature information and target character feature information both successfully match, and whose candidate auxiliary feature information also successfully matches target auxiliary feature information, are designated as target objects and tracked.

[0062] In this embodiment of the invention, determining the candidate auxiliary feature information corresponding to each object based on the candidate auxiliary region image corresponding to each object includes:

[0063] For each object, according to a preset pixel selection rule, a preset number of pixels are selected from the candidate auxiliary region image corresponding to the object, and the color information or texture information of the preset number of pixels is used as the candidate auxiliary feature information corresponding to the object.

[0064] When selecting a preset number of pixels from the candidate auxiliary region image corresponding to each object, the preset number of pixels can be randomly selected, for example, 50 or 500 pixels. Alternatively, the candidate auxiliary region image can be divided into regions, for example, divided into 6 equal parts to obtain 6 partitions, and then pixels can be selected from each of the 6 partitions to obtain the preset number of pixels. The color information or texture information of the preset number of pixels is used as the candidate auxiliary feature information corresponding to the object.

[0065] Taking color information as an example, if the target auxiliary feature is white, the percentage of white pixels out of a preset number of pixels is counted. If this percentage is greater than a preset threshold, the auxiliary feature is considered a successful match; otherwise, the match is considered unsuccessful. The preset threshold is, for example, 0.8 or 0.9.

[0066] In this embodiment of the invention, the training process of the character segmentation model includes:

[0067] For each sample tracking region image in the first training set, the sample tracking region image and the corresponding label information are input into the character segmentation model to train the character segmentation model; wherein, the label information includes the position information of the character region image in the sample tracking region image, and the position information of the character region image includes the position information of letter text regions and the position information of number contours.

[0068] The electronic device stores a first training set, which includes a large number of sample tracking region images. Each sample tracking region image has corresponding label information, including the positional information of character region images within the sample tracking region image. The positional information of the character region images includes the positional information of letter characters and the positional information of digit contours. Each sample tracking region image and its corresponding label information are input into the character segmentation model. The character segmentation model outputs the positional information of the training character region image corresponding to each sample tracking region image. A loss function value is calculated based on the positional information of the training character region images and the positional information of the character region images carried by the label information. The parameters of the character segmentation model are adjusted based on the loss function value. When the loss function value meets the requirements, the character segmentation model training is complete.

[0069] To more accurately determine the candidate tracking region image corresponding to each object, in this embodiment of the invention, determining the candidate tracking region image corresponding to each object in the image using a human detection algorithm includes:

[0070] By using human detection algorithms and pose estimation algorithms, candidate tracking regions that meet pose requirements are determined for each object in the image.

[0071] In this embodiment of the invention, a human detection algorithm is used to determine the human bounding box for each object, and a pose estimation algorithm is used to determine the pose of each object, such as a standing pose or a bent-over pose. By combining the human bounding box and the corresponding pose of each object, the upper body region of each object can be determined more accurately, and then the upper body region of each object is cropped as a candidate tracking region image.

[0072] Furthermore, after determining the pose of each object using a pose estimation algorithm, the electronic device can select candidate tracking region images that meet the pose requirements and filter out those that do not. Pose requirements can be specific to facial deflection angles; for example, images with a facial deflection angle within 20 degrees are considered acceptable, while those exceeding 20 degrees are filtered out.

[0073] In this embodiment of the invention, determining the candidate character feature information corresponding to each object based on the candidate character region image corresponding to each object includes:

[0074] For each object, the candidate character region image corresponding to the object is input into a pre-trained character recognition model, and the candidate character feature information corresponding to the object is determined based on the character recognition model.

[0075] The electronic device stores a trained character recognition model. When determining the candidate character feature information corresponding to each object, for each object, the candidate character region image corresponding to the object is input into the character recognition model, and the character recognition model outputs the candidate character feature information corresponding to the object.

[0076] To improve the accuracy of the determined candidate character feature information, in this embodiment of the invention, determining the candidate character feature information corresponding to the object based on the character recognition model includes:

[0077] Based on the alphabetic character submodule in the character recognition model, the first feature vector corresponding to the alphabetic character region image in the candidate character region image is determined.

[0078] Based on the digit sub-module in the character recognition model, the second feature vector corresponding to the digit contour in the candidate character region image is determined.

[0079] The first feature vector and the second feature vector are concatenated dimensionally, and the candidate character feature information corresponding to the concatenated feature vector is determined based on the character recognition submodule in the character recognition model.

[0080] Character feature information generally includes alphanumeric information and numeric information. To improve the recognition accuracy of candidate character feature information corresponding to a given object, the character recognition model includes an alphanumeric submodule, a numeric submodule, and a character recognition submodule. The alphanumeric submodule is used to recognize alphanumeric features in the candidate character region image, and the numeric submodule is used to recognize numeric features in the candidate character region image. Then, the alphanumeric features and numeric features are concatenated and input into the character recognition submodule to obtain the final candidate character feature information.

[0081] Specifically, the character segmentation model is used to determine the alphabetic text regions and digit contours within the candidate character region images. The alphabetic text regions are then input into the alphabetic text submodule of the character recognition model, which determines the first feature vector corresponding to each region. The digit contours are input into the digit submodule of the character recognition model, which determines the second feature vector corresponding to each contour. The first and second feature vectors are then concatenated dimensionally, and the concatenated feature vector is input into the character recognition submodule of the model. The character recognition submodule then determines the candidate character feature information corresponding to this concatenated feature vector.

[0082] This invention improves the recognition accuracy of candidate character feature information by extracting features of letter text regions and number contours from candidate character region images respectively, and then combining the two for feature extraction.

[0083] The training process of the character recognition model includes:

[0084] For each sample character region image in the second training set, the sample letter character region image in the sample character region image is input into the letter character submodule in the character recognition model, and the first sample feature vector corresponding to the sample letter character region image is determined based on the letter character submodule.

[0085] The sample digit contour in the sample character region image is input into the digit submodule of the character recognition model, and the second sample feature vector corresponding to the sample digit contour is determined based on the digit submodule.

[0086] The first sample feature vector and the second sample feature vector are concatenated dimensionally, and the sample character feature information corresponding to the dimensionally concatenated sample feature vector is determined based on the character recognition submodule in the character recognition model.

[0087] Based on the sample character feature information and the real character feature information corresponding to the sample character region image, the alphabetic character submodule, the numeric submodule, and the character recognition submodule in the character recognition model are trained.

[0088] The electronic device stores a second training set, which contains a large number of sample character region images. Each sample character region image contains a sample letter text region image and a sample number contour. Each sample character region image also contains corresponding real character feature information, namely real letter text and real numbers.

[0089] The sample alphabetic text region image from the sample character region image is input into the alphabetic text submodule of the character recognition model to obtain the first sample feature vector corresponding to the sample alphabetic text region image. The sample digit contour from the sample character region image is input into the digit submodule of the character recognition model to obtain the second sample feature vector corresponding to the digit contour. Then, the first and second sample feature vectors are concatenated dimensionally and input into the character recognition submodule of the character recognition model to determine the sample character feature information corresponding to the dimensional concatenated sample feature vector. The loss function value between the sample character feature information and the real character feature information is calculated. The parameters of the alphabetic text submodule, digit submodule, and character recognition submodule in the character recognition model are adjusted according to the loss function value, and training continues. When the loss function value meets the requirements, the training of the alphabetic text submodule, digit submodule, and character recognition submodule in the character recognition model is considered complete.

[0090] The following describes in detail the target object tracking process provided by the embodiments of the present invention, taking watching a ball game as an example and with reference to the accompanying drawings.

[0091] The target object tracking method provided in this invention primarily addresses the problems of target object loss and difficulty in locating in video target object tracking applications. Video target object tracking is used in many smart living applications, such as pedestrian trajectory querying and video character localization. This invention provides a video target object tracking method for the video viewing field, supporting cross-viewpoint tracking, simplifying the target localization method, and solving the problems of complex target tracking algorithms and poor tracking effects. It improves target tracking accuracy and thus enhances the user's viewing experience.

[0092] The main innovations of this invention include: First, it proposes a method for rapidly selecting key features of interest in the field of target object tracking. This method calculates the location of the player's jersey area based on human detection algorithms and player motion posture estimation, thus quickly determining candidate tracking area images. Second, this invention proposes an optimized scheme for feature matching methods in player tracking scenarios during sports events. Based on character segmentation and character recognition technology, it proposes an innovative character recognition scheme. The candidate character area image corresponding to the object is input into the character recognition model, which recognizes alphabetic characters and numbers respectively, improving character recognition accuracy. This mainly solves the problem of difficult character recognition and matching under high-speed motion and deformation conditions.

[0093] This invention primarily relates to a video target object tracking method, applicable to various fields related to target tracking. Taking a free-viewpoint viewing scenario as an example, this invention describes the video target tracking requirements. During a game, users may want the video view to follow their favorite player. However, in target tracking applications, it has been found that once a user selects a target of interest, issues such as occlusion or rapid movement can easily lead to target loss, resulting in a loss of the user's viewpoint and negatively impacting the user experience. Therefore, this invention proposes a target tracking method that rapidly determines the tracking area, employs a character recognition scheme based on character shape and image segmentation, and combines this with auxiliary key feature image matching to address the problems encountered in this application scenario.

[0094] Figure 2 The target object tracking flowchart provided in this embodiment of the invention includes: 1. Selecting a video tracking target object and obtaining the target character feature information and target auxiliary feature information of the target object; 2. Video image detection and target object segmentation; 3. Jersey character recognition and matching; 4. Auxiliary feature information determination and matching; 5. Target localization and result output.

[0095] 1. Select the target object for video tracking and obtain the target character feature information and target auxiliary feature information of the target object.

[0096] The free-viewpoint match viewing application provides a user interface where users can manually or via voice select their favorite players. After the user selects a player, the application system will automatically export the player number data (corresponding to target character feature information) and the player's jersey color feature data for this match (corresponding to target auxiliary feature information). The exported data D is used to locate the target player later.

[0097] 2. Video image detection and target object segmentation.

[0098] Human detection algorithms are used to detect human bodies in the match video images, locating all player positions. A player number region selection method is then used to quickly determine the tracking area, selecting the player's jersey and player number region. Figure 3 As shown. The player number region selection method calculates the player's jersey region location based on human detection algorithms and player motion posture estimation. It then obtains the player's jersey number region image through image cropping, quickly identifying the tracking feature region. This method requires creating a player posture judgment algorithm to determine if the player's posture meets the requirements and quickly filter regions of interest in the image. This method uses a jersey character segmentation algorithm to... Figure 3 The image shown is processed to obtain the player character region image, and the character recognition effect is optimized to obtain the following result: Figure 4 The image shown. The auxiliary region image is determined by the character mask region, as shown. Figure 5 As shown. Figure 5 The part containing the characters is the character region image, and the part outside the characters is the auxiliary region image.

[0099] 3. Jersey character recognition and matching.

[0100] The player jersey character images may be distorted or tilted. First, you need to... Figure 4 The image shown is corrected and denoised. The character recognition algorithm proposed in this invention is used to extract character features and recognize content from the player's jersey character image. The recognized content is matched with the user's selected player information D. Only the player position information that matches the player of interest is retained. This information is further confirmed to be the target of interest through an auxiliary feature step.

[0101] To improve the accuracy of jersey character recognition, this invention, based on the characteristics of jersey character segmentation, divides characters into alphanumeric regions and numeric outlines. To improve model calculation speed and reduce image noise, the alphanumeric regions are input into the alphanumeric submodule of the character recognition model for feature extraction. The numeric outlines are then input into the numeric submodule of the character recognition model for feature extraction. Finally, the features of both are fused and input into the character recognition submodule to obtain the final recognition result, thus improving character recognition accuracy. This solves the problem of difficult character recognition and matching under high-speed motion and deformation conditions. The specific implementation process is as follows: Figure 6 As shown.

[0102] 4. Determination and matching of auxiliary feature information.

[0103] To improve the accuracy of target object recognition, this invention randomly samples the auxiliary region image to obtain a preset number of pixels. The sampling pixel positions and sampling number can be adjusted based on the recognition effect. After obtaining the preset number of pixels, a pixel matching method is used to match the preset number of pixels with the uniform color attribute in data D, and the corresponding matching result is output. The calculation formula is as follows:

[0104]

[0105] Where x represents the color information of a preset number of pixels, and y represents the color information of pixels in data D.

[0106] 5. Target positioning and result output.

[0107] Player selection is achieved by matching player jersey character features with auxiliary feature matching results, enabling target localization and online tracking of video targets. The calculation results of auxiliary features and player character features are simpler than traditional target localization methods in the field of target tracking, thus improving the speed of target localization and enabling rapid real-time tracking of video targets.

[0108] Figure 7 This is a schematic diagram of a target object tracking device provided in an embodiment of the present invention. The device includes:

[0109] The acquisition module 71 is used to determine the target object to be tracked and to acquire the target character feature information and target auxiliary feature information of the target object;

[0110] The determination module 72 is used to determine the candidate character region image and the candidate auxiliary region image outside the candidate character region image for each object in the image; and based on the candidate character region image and the candidate auxiliary region image for each object, determine the candidate character feature information and the candidate auxiliary feature information for each object.

[0111] The tracking module 73 is used to match the candidate character feature information and candidate auxiliary feature information corresponding to each object with the target character feature information and target auxiliary feature information, and to take the successfully matched object as the target object and track it.

[0112] The determining module 72 is specifically used to determine the candidate tracking region image corresponding to each object in the image through a human detection algorithm; for each candidate tracking region image, the candidate tracking region image is input into a pre-trained character segmentation model, and candidate character region images in the candidate tracking region image are determined based on the character segmentation model; and images in the candidate tracking region image other than the candidate character region images are used as candidate auxiliary region images.

[0113] The determining module 72 is specifically used to determine the candidate tracking region image that meets the posture requirements for each object in the image by using human detection algorithm and posture estimation algorithm.

[0114] The determining module 72 is specifically used to input the candidate character region image corresponding to the object into a pre-trained character recognition model for each object, and determine the candidate character feature information corresponding to the object based on the character recognition model.

[0115] The determining module 72 is specifically used to determine, based on the alphabetic text submodule in the character recognition model, a first feature vector corresponding to the alphabetic text region image in the candidate character region image; based on the digit submodule in the character recognition model, a second feature vector corresponding to the digit contour in the candidate character region image; to perform dimensional concatenation on the first feature vector and the second feature vector; and based on the character recognition submodule in the character recognition model, to determine the candidate character feature information corresponding to the dimensional concatenated feature vector.

[0116] The determining module 72 is specifically used to select a preset number of pixels from the candidate auxiliary region image corresponding to each object according to a preset pixel selection rule, and use the color information or texture information of the preset number of pixels as the candidate auxiliary feature information corresponding to the object.

[0117] The device further includes:

[0118] The first training module 74 is used to input the sample tracking region image and the corresponding label information into the character segmentation model for each sample tracking region image in the first training set, and to train the character segmentation model information; wherein, the label information includes the position information of the character region image in the sample tracking region image, and the position information of the character region image includes the position information of letter text region and the position information of number contour.

[0119] The device further includes:

[0120] The second training module 75 is used for each sample character region image in the second training set to input a sample alphabetic character region image into the alphabetic character submodule of the character recognition model, and determine a first sample feature vector corresponding to the sample alphabetic character region image based on the alphabetic character submodule; input a sample digit contour from the sample character region image into the digit submodule of the character recognition model, and determine a second sample feature vector corresponding to the sample digit contour based on the digit submodule; perform dimensional concatenation on the first and second sample feature vectors, and determine sample character feature information corresponding to the dimensional concatenated sample feature vector based on the character recognition submodule of the character recognition model; and train the alphabetic character submodule, digit submodule, and character recognition submodule of the character recognition model according to the sample character feature information and the real character feature information corresponding to the sample character region image.

[0121] This invention also provides an electronic device, such as... Figure 8 As shown, it includes: processor 301, communication interface 302, memory 303 and communication bus 304, wherein processor 301, communication interface 302 and memory 303 communicate with each other through communication bus 304;

[0122] The memory 303 stores a computer program, which, when executed by the processor 301, causes the processor 301 to perform the following steps:

[0123] Identify the target object to be tracked, and obtain the target character feature information and target auxiliary feature information of the target object;

[0124] Determine the candidate character region image and the candidate auxiliary region image outside the candidate character region image for each object in the image; and based on the candidate character region image and the candidate auxiliary region image for each object, determine the candidate character feature information and the candidate auxiliary feature information for each object.

[0125] The candidate character feature information and candidate auxiliary feature information corresponding to each object are matched with the target character feature information and target auxiliary feature information. The objects that are successfully matched are taken as target objects and tracked.

[0126] Based on the same inventive concept, this embodiment of the invention also provides an electronic device. Since the principle of the above electronic device in solving the problem is similar to that of the target object tracking method, the implementation of the above electronic device can refer to the implementation of the method, and the repeated parts will not be described again.

[0127] The electronic devices provided in the embodiments of the present invention can specifically be desktop computers, portable computers, smartphones, tablet computers, personal digital assistants (PDAs), network-side devices, etc.

[0128] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0129] Communication interface 302 is used for communication between the above-mentioned electronic device and other devices.

[0130] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0131] The processors mentioned above can be general-purpose processors, including central processing units, network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0132] This invention also provides a computer-readable storage medium storing a computer program executable by an electronic device. When the program is run on the electronic device, the electronic device performs the following steps:

[0133] Identify the target object to be tracked, and obtain the target character feature information and target auxiliary feature information of the target object;

[0134] Determine the candidate character region image and the candidate auxiliary region image outside the candidate character region image for each object in the image; and based on the candidate character region image and the candidate auxiliary region image for each object, determine the candidate character feature information and the candidate auxiliary feature information for each object.

[0135] The candidate character feature information and candidate auxiliary feature information corresponding to each object are matched with the target character feature information and target auxiliary feature information. The objects that are successfully matched are taken as target objects and tracked.

[0136] Based on the same inventive concept, this embodiment of the invention also provides a computer-readable storage medium. Since the principle of solving the problem when the processor executes the computer program stored on the computer-readable storage medium is similar to that of the target object tracking method, the implementation of the processor executing the computer program stored on the computer-readable storage medium can be referred to the implementation of the method, and the repeated parts will not be described again.

[0137] The aforementioned computer-readable storage medium can be any available medium or data storage device that can be accessed by the processor in an electronic device, including but not limited to magnetic storage such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), optical storage such as CDs, DVDs, BDs, HVDs, etc., and semiconductor storage such as ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs), etc.

[0138] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0139] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0140] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0141] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0142] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A target object tracking method characterized by, The method includes: Identify the target object to be tracked, and obtain the target character feature information and target auxiliary feature information of the target object; Determine the candidate character region image and the candidate auxiliary region image outside the candidate character region image for each object in the image; and based on the candidate character region image and the candidate auxiliary region image for each object, determine the candidate character feature information and the candidate auxiliary feature information for each object. The candidate character feature information and candidate auxiliary feature information corresponding to each object are matched with the target character feature information and target auxiliary feature information, and the successfully matched objects are taken as target objects and tracked. The determination of the candidate character region image corresponding to each object in the image and the candidate auxiliary region image outside the candidate character region image includes: The human detection algorithm determines the candidate tracking region image corresponding to each object in the image; wherein, the human detection algorithm determines the human detection box corresponding to each object in the image, and the upper body region of the human body in the human detection box is obtained by cropping, and the obtained upper body region is used as the candidate tracking region image. For each candidate tracking region image, the candidate tracking region image is input into a pre-trained character segmentation model. Based on the character segmentation model, candidate character region images are determined in the candidate tracking region image. Images in the candidate tracking region image other than the candidate character region images are used as candidate auxiliary region images. The step of determining the candidate tracking region image corresponding to each object in the image through a human detection algorithm includes: By using human detection algorithms and pose estimation algorithms, candidate tracking region images that meet pose requirements are determined for each object in the image. Among them, candidate tracking region images that meet the pose requirements are selected, and candidate tracking region images that do not meet the pose requirements are filtered out; the pose requirements are the face deflection angle requirements.

2. The method of claim 1, wherein, Based on the candidate character region image corresponding to each object, the candidate character feature information corresponding to each object is determined as follows: For each object, the candidate character region image corresponding to the object is input into a pre-trained character recognition model, and the candidate character feature information corresponding to the object is determined based on the character recognition model.

3. The method as described in claim 2, characterized in that, The step of determining the candidate character feature information corresponding to the object based on the character recognition model includes: Based on the alphabetic character submodule in the character recognition model, the first feature vector corresponding to the alphabetic character region image in the candidate character region image is determined. Based on the digit sub-module in the character recognition model, the second feature vector corresponding to the digit contour in the candidate character region image is determined. The first feature vector and the second feature vector are concatenated dimensionally, and the candidate character feature information corresponding to the concatenated feature vector is determined based on the character recognition submodule in the character recognition model.

4. The method as described in claim 1, characterized in that, The step of determining the candidate auxiliary feature information corresponding to each object based on the candidate auxiliary region image corresponding to each object includes: For each object, according to a preset pixel selection rule, a preset number of pixels are selected from the candidate auxiliary region image corresponding to the object, and the color information or texture information of the preset number of pixels is used as the candidate auxiliary feature information corresponding to the object.

5. The method as described in claim 2, characterized in that, The training process of the character segmentation model includes: For each sample tracking region image in the first training set, the sample tracking region image and the corresponding label information are input into the character segmentation model to train the character segmentation model; wherein, the label information includes the position information of the character region image in the sample tracking region image, and the position information of the character region image includes the position information of letter text regions and the position information of number contours.

6. The method as described in claim 3, characterized in that, The training process of the character recognition model includes: For each sample character region image in the second training set, the sample letter character region image in the sample character region image is input into the letter character submodule in the character recognition model, and the first sample feature vector corresponding to the sample letter character region image is determined based on the letter character submodule. The sample digit contour in the sample character region image is input into the digit submodule of the character recognition model, and the second sample feature vector corresponding to the sample digit contour is determined based on the digit submodule. The first sample feature vector and the second sample feature vector are concatenated dimensionally, and the sample character feature information corresponding to the concatenated sample feature vector is determined based on the character recognition submodule in the character recognition model. The alphabetic character submodule, the numeric submodule, and the character recognition submodule in the character recognition model are trained based on the sample character feature information and the real character feature information corresponding to the sample character region image.

7. A target object tracking device, characterized in that, The device includes: The acquisition module is used to determine the target object to be tracked and to acquire the target character feature information and target auxiliary feature information of the target object; The determination module is used to determine the candidate character region image and the candidate auxiliary region image outside the candidate character region image for each object in the image; and based on the candidate character region image and the candidate auxiliary region image for each object, determine the candidate character feature information and the candidate auxiliary feature information for each object. The tracking module is used to match the candidate character feature information and candidate auxiliary feature information corresponding to each object with the target character feature information and target auxiliary feature information, and to track the successfully matched objects as target objects; The determining module is specifically used to determine the candidate tracking region image corresponding to each object in the image through a human detection algorithm; wherein, the human detection algorithm determines the human detection box corresponding to each object in the image, and the upper body region of the human body in the human detection box is obtained by cropping, and the obtained upper body region is used as the candidate tracking region image; for each candidate tracking region image, the candidate tracking region image is input into a pre-trained character segmentation model, and the candidate character region image in the candidate tracking region image is determined based on the character segmentation model, and the image in the candidate tracking region image other than the candidate character region image is used as the candidate auxiliary region image; The determining module is specifically used to determine the candidate tracking region image that meets the pose requirements for each object in the image by using a human detection algorithm and a pose estimation algorithm; wherein, the candidate tracking region image that meets the pose requirements is selected, and the candidate tracking region image that does not meet the pose requirements is filtered out; the pose requirements are the face deflection angle requirements.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-6.

Images