Training method and device of tracking model, electronic equipment and storage medium
By determining and matching the target image pairs in the candidate data group, the tracking model is trained to solve the tracking stability problem in multi-objective scenarios, and the stability and robustness of the tracking model are improved under real-time guarantee.
Patent Information
- Application Number
- CN202411413986.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-11
- Publication Date
- 2025-01-10
AI Technical Summary
The existing single-target tracking method lacks real-time performance on the embedded end and cannot guarantee the tracking stability of the target morphology changes, especially in multi-target scenarios, it is difficult to stably track the same type of target.
By determining at least two candidate data groups, selecting the target templates and search images in the target data group to match based on the preset proportion and the number of candidate image pairs, training the tracking model to ensure that the number of target image pairs is close to actual requirements and improving the model's Robustness and anti-interference ability.
On the premise of ensuring real-time tracking, the stability and robustness of the tracking model are significantly improved, and the same type of target can be tracked stably, saving data processing time and limiting the type of tracking targets.
Smart Images
Figure CN119295776A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a training method, device, electronic device and storage medium for a tracking model. Background Art
[0002] In some tracking scenarios, the tracked objects of the same type are often the same object. For example, when tracking a person, there is only the same person in the tracking picture, and there may be other non-human objects or animals at the same time. When tracking a cat, there is only the same cat in the tracking picture, and there may be other non-cat objects or animals at the same time. Alternatively, in some tracking scenarios, it is not necessary to distinguish whether the tracked objects of the same type are the same object, and only specific objects of the same type need to be stably tracked. For example, although there are two cats in the picture, there are also other non-cat objects such as dogs and people. It is sufficient to stably track any one cat without tracking other non-cat targets such as dogs and people.
[0003] If the existing single target tracking method is used to solve the tracking problem in the above scenario, the online template update solution is better in terms of effect. However, the real-time performance of this solution on the embedded side cannot meet the demand. If the offline training solution is used, the stability of tracking cannot be guaranteed when the target changes its shape. Therefore, it is very important to improve the stability of the tracking model while ensuring the real-time performance of the tracking. Summary of the invention
[0004] The present invention provides a training method, device, electronic device and storage medium for a tracking model, so as to improve the stability of the tracking model while ensuring the real-time tracking.
[0005] According to one aspect of the present invention, a method for training a tracking model is provided, the method comprising:
[0006] Determine at least two candidate data groups; wherein the same candidate data group includes data formed by annotating the same type of tracking objects;
[0007] Determine a first number of types required for training the tracking model, and determine the number of candidate image pairs; determine a second number of target image pairs corresponding to each target tracking object type based on a preset ratio and the number of candidate image pairs;
[0008] Selecting a candidate data group corresponding to the target tracking object type from the candidate data group as a target data group, and determining a second number of target template images and target search images corresponding to the target tracking object type based on the target data in the target data group and the second number;
[0009] The target template image and the target search image are matched according to a preset matching method to determine a second number of target image pairs corresponding to the target tracking object type, and the target image pairs are used to train the tracking model; wherein all objects of the same target tracking object type contained in the same target search image are tracking objects.
[0010] According to another aspect of the present invention, there is provided a training device for a tracking model, the device comprising:
[0011] A data group determination module, used to determine at least two candidate data groups; wherein the same candidate data group includes data formed by annotating the same type of tracked objects;
[0012] A quantity determination module, used to determine a first number of types required for training the tracking model and to determine the number of candidate image pairs; and to determine a second number of target image pairs corresponding to each target tracking object type based on a preset ratio and the number of candidate image pairs;
[0013] an image determination module, configured to select a candidate data group corresponding to the target tracking object type from the candidate data group as a target data group, and determine a second number of target template images and target search images corresponding to the target tracking object type based on the target data in the target data group and the second number;
[0014] An image pair training module is used to match the target template image and the target search image according to a preset matching method to determine a second number of target image pairs corresponding to the target tracking object type, and use the target image pairs to train the tracking model; wherein all objects of the same target tracking object type contained in the same target search image are tracking objects.
[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:
[0016] at least one processor; and
[0017] a memory communicatively connected to the at least one processor; wherein,
[0018] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the training method of the tracking model described in any embodiment of the present invention.
[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the training method of the tracking model described in any embodiment of the present invention when executed.
[0020] The technical solution of the embodiment of the present invention determines at least two candidate data groups, determines a first number of types required for training a tracking model and determines the number of candidate image pairs, determines a second number of target image pairs corresponding to each target tracking object type based on a preset ratio and the number of candidate image pairs, and the sum of the second numbers in the present application is the same as the number of candidate image pairs. The second number of target image pairs is accurately determined based on a preset ratio for describing the ratio of extracting data from the target data group, which can ensure that the number of target image pairs is selected to be closer to the number required for actual training, and further selects a candidate data group corresponding to the target tracking object type from the candidate data group as the target data group, and determines the target image pairs based on the target data in the target data group and the second number. A second number of target template images and target search images corresponding to the target tracking object type, and then matching the target template image and the target search image according to a preset matching method to determine a second number of target image pairs corresponding to the target tracking object type. It can be seen that the image pairs of the present application can come from different training data in the same data group, and are not image pairs obtained by processing the same training data in the same data group, which greatly saves data processing time and limits the types of tracking targets, so that the tracking model obtained by training the tracking model with the image relative to the present application can stably track the same type of tracking targets, has strong robustness and anti-interference ability, and achieves the improvement of the stability of the tracking model while ensuring real-time tracking.
[0021] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0023] Figure 1 is a flow chart of a tracking model training method provided according to an embodiment of the present invention;
[0024] Figure 2is a flowchart of another tracking model training method provided according to an embodiment of the present invention;
[0025] Figure 3 is a structural schematic diagram of a tracking model training device provided according to an embodiment of the present invention;
[0026] Figure 4 It is a structural schematic diagram of an electronic device for implementing the training method of the tracking model of the embodiment of the present invention provided according to the embodiment of the present invention. DETAILED DESCRIPTION
[0027] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0029] Embodiment 1
[0030] Figure 1 A flowchart of a tracking model training method provided in an embodiment of the present invention. This embodiment is applicable to the case of obtaining training data used when training a single-target tracking model. The method can be executed by a tracking model training device, which can be implemented in the form of hardware and / or software, and can be configured in any electronic device with network communication function.
[0031] like Figure 1 As shown, the training method of the tracking model of the present application includes the following process:
[0032] S110, determining at least two candidate data groups; wherein the same candidate data group includes data formed by annotating the same type of tracking objects.
[0033] The data in the candidate data group is used to train the tracking model.
[0034] Specifically, data information related to the tracking training task is obtained, the data information including but not limited to video data or image data, and the data information is preprocessed and enhanced to improve the generalization ability of the data, and the data information is grouped according to different types of tracking objects indicated in the data information, so that the tracking objects in each data group are of the same type, and these data groups are used as candidate data groups.
[0035] As an optional but non-limiting implementation method, the process of determining at least two candidate data groups may include: obtaining reference data for tracking model training, the reference data including video data or image data, annotating the tracking objects in the reference data to determine the type of tracking objects corresponding to the reference data, wherein the annotation process may annotate according to information such as the position, shape, size, and type of the tracking objects, and further grouping reference data sets of the same type of tracking objects into one data group as a candidate data group.
[0036] The technical solution of this embodiment adopts the above method to determine the candidate data group, ensuring that the data in each data group are the same type of tracking objects, so that when the data in the candidate data group is subsequently used to train the tracking model, it is guaranteed that the specified type of tracking target is trained.
[0037] S120, determining a first number of types required for training the tracking model, and determining a number of candidate image pairs; determining a second number of target image pairs corresponding to each target tracking object type based on a preset ratio and the number of candidate image pairs.
[0038] Among them, the candidate image pair includes a candidate template image and a candidate search image, the candidate template image is used to perform feature comparison with the candidate search image to determine the tracking object, the target image pair includes a target template image and a target search image, the target template image is used to perform feature comparison with the target search image to determine the tracking object, and the first number of types corresponds to the type of the candidate data group. The target tracking object type is one of the first number of types, the sum of the second number is the same as the number of candidate image pairs, and the preset ratio is used to describe the ratio of data extracted from the target data group. The preset ratio can be set according to actual needs; for example, assuming that the training process requires 2 types of tracking objects, then the preset ratio can be set to 1:1 or 1:2 or other ratios that meet the training requirements. The present application can use a 1:1 ratio to allocate the number of candidate image pairs.
[0039] In this embodiment, the number of candidate image pairs is determined in order to determine how much data is needed to train the current tracking model.
[0040] For example, if there are A candidate data groups, it means there are A types of tracking objects, and it is determined that the current training process requires B types of tracking objects, that is, the first number is B, then candidate data groups of B types of tracking objects are selected from the A candidate data groups for model training. At the same time, it is determined that the training process requires N candidate image pairs for tracking training, that is, the number of candidate image pairs is N, so that the amount of data required for each type can be determined later according to the N candidate image pairs, and the sum of the data used for each type is N.
[0041] In this embodiment, a preset ratio is obtained, and the number of candidate image pairs is allocated according to the preset ratio as the second number of target image pairs corresponding to each target tracking object type.
[0042] As an optional but non-limiting manner, determining a second number of target image pairs corresponding to each target tracking object type based on a preset ratio and the number of candidate image pairs includes: taking a ratio of the number of candidate image pairs to the first number as the second number of target image pairs corresponding to each target tracking object type.
[0043] Specifically, the ratio of the number of candidate image pairs to the first number is used as the second number of target image pairs corresponding to each target tracking object type. This method can be considered as a preset ratio used to represent the ratio for equally distributing data. For example, the first number B and the number N of candidate image pairs are determined, and the second number D is N / B. This also ensures that whether it is single-card or multi-card training, the data on all cards only contains the selected B types of data.
[0044] S130. Select a candidate data group corresponding to the target tracking object type from the candidate data groups as the target data group, and determine a second number of target template images and target search images corresponding to the target tracking object type based on the target data in the target data group and the second number.
[0045] Specifically, determining a second number of target template images and a target search image corresponding to the target tracking object type based on the target data and the second number in the target data group includes: obtaining the target number D of the target image pair, then randomly selecting D target data from the target data group as reference training data, or randomly selecting D target data from the target data group as reference training data according to a preset selection method, and then performing augmentation, cropping, scaling and other operations on the reference training data to obtain the target search image, and at the same time performing augmentation, cropping, scaling and other operations on the reference training data to determine to obtain the target template image.
[0046] S140. Match the target template image and the target search image according to a preset matching method to determine a second number of target image pairs corresponding to the target tracking object type, so as to train the tracking model using the target image pairs.
[0047] Among them, all objects of the same target tracking object type contained in the same target search image are tracking objects. For example, if the target tracking object type is type A, and the target search image contains 3 objects corresponding to type A, then these 3 objects are all tracking objects of this target search image. The target image pair includes a target template image and a target search image. The target template image is used to perform feature comparison with the target search image to determine the tracking object during model training. The preset matching method can be used to describe the method of pairing the target template image and the target search image. For example, the preset matching method can be a method of randomly matching the target template image and the target search image. For example, if 10 target template images and 10 target search images are obtained, they can be randomly combined in pairs to form a target image pair.
[0048] Specifically, a second number of target template images and target search images corresponding to the target tracking object type are obtained, and the target template image and the target search image are matched according to a preset matching method to determine a second number of target image pairs corresponding to the target tracking object type. The specific process may be:
[0049] The target search image and the target template image are matched according to a preset matching method to determine a second number of target image pairs corresponding to the target tracking object type. It can be seen that the image pairs of the present application can come from different training data in the same data group, and are not image pairs obtained by processing the same training data in the same data group. This greatly saves data processing time and limits the category of tracking targets, so that the tracking model obtained by training the tracking model using the image relative to the present application can stably track the same type of tracking objects and has strong robustness and anti-interference capabilities.
[0050] As an optional but non-limiting manner, the target template image and the target search image are matched according to a preset matching method to determine a second number of target image pairs corresponding to the target tracking object type, including: determining a second number of arrangement labels, the arrangement labels are different from each other; marking the second number of target template images according to the arrangement labels to obtain marked target template images; marking the second number of target search images according to the arrangement labels to obtain marked target search images; and treating the target template image and the target search image with the same arrangement label as a target image pair.
[0051] The technical solution of this embodiment marks the target template image and the target search image according to the arrangement labels, and then regards the target template image and the target search image with the same arrangement labels as a target image pair, which can more accurately determine the target image pair, so that the tracking model obtained by training the tracking model using the target graph of the present invention can stably track the same type of tracking targets and has strong robustness and anti-interference capabilities.
[0052] Optionally, after determining the second number of target image pairs corresponding to the target tracking object type, the present application uses the target image pairs to train the single target tracking model in an offline training manner, so that parameters do not need to be updated during the use phase, thereby ensuring the real-time tracking.
[0053] The technical solution of the embodiment of the present invention determines at least two candidate data groups, determines a first number of types required for training a tracking model and determines the number of candidate image pairs, determines a second number of target image pairs corresponding to each target tracking object type based on a preset ratio and the number of candidate image pairs, and the sum of the second numbers in the present application is the same as the number of candidate image pairs. The second number of target image pairs is accurately determined based on a preset ratio for describing the ratio of extracting data from the target data group, which can ensure that the number of target image pairs is selected to be closer to the number required for actual training, and further selects a candidate data group corresponding to the target tracking object type from the candidate data group as the target data group, and determines the target image pairs based on the target data in the target data group and the second number. A second number of target template images and target search images corresponding to the target tracking object type, and then matching the target template image and the target search image according to a preset matching method to determine a second number of target image pairs corresponding to the target tracking object type. It can be seen that the image pairs of the present application can come from different training data in the same data group, and are not image pairs obtained by processing the same training data in the same data group, which greatly saves data processing time and limits the types of tracking targets, so that the tracking model obtained by training the tracking model with the image relative to the present application can stably track the same type of tracking targets, has strong robustness and anti-interference ability, and achieves the improvement of the stability of the tracking model while ensuring real-time tracking.
[0054] Embodiment 2
[0055] Figure 2 A flowchart of another method for training a tracking model provided in an embodiment of the present invention. The technical solution of this embodiment further optimizes the process of determining a second number of target template images and target search images corresponding to the target tracking object type based on the target data in the target data group and the second number in S130 in the aforementioned embodiment on the basis of the above-mentioned embodiment. This embodiment can be combined with various optional schemes in one or more of the above-mentioned embodiments.
[0056] like Figure 2 As shown, the training method of the tracking model of the present invention may include:
[0057] S210, determining at least two candidate data groups, determining a first number of types required for training the tracking model and determining the number of candidate image pairs, determining a second number of target image pairs corresponding to each target tracking object type based on a preset ratio and the number of candidate image pairs, and selecting a candidate data group corresponding to the target tracking object type from the candidate data groups as the target data group.
[0058] The data in the candidate data group is used to train the tracking model, and the same candidate data group includes data formed by annotating the same type of tracking objects. The image pair includes a template image and a search image, and the template image is used to perform feature comparison with the search image to determine the tracking object. The first number of types corresponds to the type of the candidate data group. The target tracking object type is one of the first number of types. The sum of the second number is the same as the number of candidate image pairs, and the preset ratio is used to describe the ratio of data extracted from the target data group.
[0059] S220: Select a second number of target data from the target data group as reference training data, where the reference training data includes video data and / or image data.
[0060] S230: Determine a second number of target template images corresponding to the target tracking object type based on the reference training data, and mark a tracking object in the same target template image.
[0061] Specifically, after augmentation processing is performed on each reference training data, processing is performed according to a preset processing method to determine a second number of target template images, and the preset processing method at least includes cropping and scaling.
[0062] As an optional but non-limiting manner, determining a second number of target template images corresponding to the target tracking object type based on the reference training data may include the following steps A1-A2:
[0063] Step A1: If the reference training data is image data, each image in the reference training data is used as a first original image, where the first original image is an image after augmentation processing; a tracking object belonging to the target tracking object type in the first original image is used as a first reference object, and the position of the first reference object is used as a first reference position; further based on the first reference object and the first reference position, the first original image is processed according to a preset processing method to determine a target template image; the preset processing method at least includes cropping and scaling.
[0064] Step A2: If the reference training data is video data, an image is selected from each video in the reference training data as the second original image, where the second original image is an image after augmentation processing; then a tracking object belonging to the target tracking object type in the second original image is used as the second reference object, and the position of the second reference object is used as the second reference position; further based on the second reference object and the second reference position, the second original image is processed according to a preset processing method to determine the target template image; the preset processing method at least includes cropping and scaling.
[0065] The technical solution of this embodiment ensures that no matter the reference training data is image data or video data, it can be processed to obtain an accurate target template image.
[0066] S240: Determine a second number of target search images corresponding to the target tracking object type based on the reference training data, and objects of the same target tracking object type contained in the same target search image are all tracking objects.
[0067] Specifically, after augmentation processing is performed on each reference training data, processing is performed according to a preset processing method to determine a second number of target search images, and the preset processing method at least includes cropping and scaling.
[0068] As an optional but non-limiting manner, determining a second number of target search images corresponding to the target tracking object type based on the reference training data may include the following steps B1-B2:
[0069] Step B1: If the reference training data is image data, each image in the reference training data is used as a third original image, and the third original image is an image after augmentation processing; then the third original image is processed according to a preset processing method to determine a target search image, and all objects of the same target tracking object type contained in the target search image are used as target tracking objects.
[0070] Step B2: If the reference training data is video data, an image is selected from each video in the reference training data as the fourth original image, and the fourth original image is an image after augmentation processing; then the fourth original image is processed according to a preset processing method to determine a target search image, and all objects of the same target tracking object type contained in the target search image are taken as target tracking objects.
[0071] The technical solution of this embodiment ensures that no matter whether the reference training data is image data or video data, it can be processed to obtain an accurate target search image.
[0072] S250: Match the target template image and the target search image according to a preset matching method to determine a second number of target image pairs corresponding to the target tracking object type, so as to train the tracking model using the target image pairs.
[0073] The technical solution of the embodiment of the present invention determines at least two candidate data groups, determines a first number of types required for training a tracking model and determines the number of candidate image pairs, determines a second number of target image pairs corresponding to each target tracking object type based on a preset ratio and the number of candidate image pairs, and the sum of the second numbers in the present application is the same as the number of candidate image pairs. The second number of target image pairs is accurately determined according to a preset ratio used to describe the ratio of extracting data from the target data group, which can ensure that the number of target image pairs is closer to the number required for actual training, and then selects a candidate data group corresponding to the target tracking object type from the candidate data group as the target data group, and then selects a second number of target data from the target data group as reference training data, ensuring that the number of reference training data meets the training requirements, and further determines a second number of target template images corresponding to the target tracking object type based on the reference training data, and determines based on the reference training data A second number of target search images corresponding to the target tracking object type. Since one tracking object is marked in the same target template image, all objects of the same target tracking object type contained in the same target search image are tracking objects, thereby ensuring tracking training for tracking objects of the same type. Then, the target template image and the target search image are matched according to a preset matching method to determine the second number of target image pairs corresponding to the target tracking object type. It can be seen that the image pairs of the present application can come from different training data in the same data group, and are not image pairs obtained by processing the same training data in the same data group. This greatly saves data processing time and limits the types of tracking targets, so that the tracking model obtained by training the tracking model using the image pair of the present application can stably track the same type of tracking targets, has strong robustness and anti-interference ability, and achieves the improvement of the stability of the tracking model while ensuring real-time tracking.
[0074] Embodiment 3
[0075] Figure 3 A structural schematic diagram of a tracking model training device provided in an embodiment of the present invention. This embodiment can be applied to the case of obtaining training data used when training a single target tracking model. The tracking model training device can be implemented in the form of hardware and / or software. The tracking model training device can be configured in any electronic device with network communication function.
[0076] like Figure 3 As shown, the training device of the tracking model of the present application includes:
[0077] The data group determination module 310 is used to determine at least two candidate data groups; wherein the data in the candidate data groups are used to train the tracking model, and the same candidate data group includes data formed by annotating the same type of tracking objects.
[0078] The quantity determination module 320 is used to determine the first number of types required for training the tracking model and the number of candidate image pairs; based on a preset ratio and the number of candidate image pairs, a second number of target image pairs corresponding to each target tracking object type is determined.
[0079] The image determination module 330 is used to select a candidate data group corresponding to the target tracking object type from the candidate data group as the target data group, and determine a second number of target template images and target search images corresponding to the target tracking object type based on the target data in the target data group and the second number.
[0080] The image pair training module 340 is used to match the target template image and the target search image according to a preset matching method to determine a second number of target image pairs corresponding to the target tracking object type, and use the target image pairs to train the tracking model; wherein all objects of the same target tracking object type contained in the same target search image are tracking objects.
[0081] The image pair includes a template image and a search image, the template image is used to perform feature comparison with the search image to determine the tracking object, the first number of types corresponds to the type of the candidate data group. The target tracking object type is one of the first number of types, the sum of the second number is the same as the number of candidate image pairs, and the preset ratio is used to describe the ratio of data extracted from the target data group.
[0082] Based on the above embodiment, optionally, a data group determination module is used to: obtain reference data for tracking model training, the reference data including video data or image data; label the tracking objects in the reference data to determine the type of tracking objects corresponding to the reference data; and group reference data sets of the same type of tracking objects into a data group as a candidate data group.
[0083] Based on the above embodiment, optionally, the quantity determination module is used to: use the ratio of the number of the candidate image pairs to the first number as the second number of target image pairs corresponding to each target tracking object type.
[0084] Based on the above embodiment, optionally, the image determination module includes:
[0085] A training data determination unit, configured to select the second number of target data from the target data group as reference training data, where the reference training data includes video data and / or image data;
[0086] A template image determining unit, configured to determine a second number of target template images corresponding to the target tracking object type based on the reference training data, wherein one tracking object is marked in the same target template image;
[0087] A search image determination unit is used to determine a second number of target search images corresponding to the target tracking object type based on the reference training data, and all objects of the same target tracking object type contained in the same target search image are tracking objects.
[0088] On the basis of the above embodiment, optionally, if the reference training data is image data, the template image determination unit is used to: take each image in the reference training data as a first original image, where the first original image is an image after augmentation processing; take a tracking object belonging to the target tracking object type in the first original image as a first reference object, and take the position of the first reference object as a first reference position; based on the first reference object and the first reference position, process the first original image in a preset processing manner to determine a target template image;
[0089] On the basis of the above embodiment, optionally, if the reference training data is video data, the template image determination unit is used to: select an image from each video in the reference training data as a second original image, where the second original image is an image after augmentation processing; take a tracking object belonging to the target tracking object type in the second original image as a second reference object, and take the position of the second reference object as a second reference position; based on the second reference object and the second reference position, process the second original image according to a preset processing method to determine the target template image.
[0090] On the basis of the above embodiment, optionally, if the reference training data is image data, the search image determination unit is used to: use each image in the reference training data as a third original image, where the third original image is an image after augmentation processing; process the third original image according to a preset processing method to determine a target search image, and use all objects of the same target tracking object type contained in the target search image as target tracking objects;
[0091] On the basis of the above embodiment, optionally, if the reference training data is video data, the search image determination unit is used to: select an image from each video in the reference training data as a fourth original image, and the fourth original image is an image after augmentation processing; process the fourth original image according to a preset processing method to determine a target search image, and use all objects of the same target tracking object type contained in the target search image as target tracking objects.
[0092] Based on the above embodiment, optionally, the image pair training module includes a target image pair determination unit, which is used to: determine a second number of arrangement labels, wherein the arrangement labels are different from each other; mark the second number of target template images according to the arrangement labels to obtain marked target template images; mark the second number of target search images according to the arrangement labels to obtain marked target search images; and treat the target template image and the target search image with the same arrangement label as a target image pair.
[0093] The training device for the tracking model provided in the embodiment of the present invention can execute the training method for the tracking model provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0094] The acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations and do not violate public order and good morals.
[0095] Embodiment 3
[0096] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
[0097] Figure 4 A schematic diagram of the structure of an electronic device that can be used to implement the training method of the tracking model of an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.
[0098] like Figure 4As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0099] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0100] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the training method of the tracking model.
[0101] In some embodiments, the training method of the tracking model may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the training method of the tracking model described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the training method of the tracking model in any other appropriate manner (e.g., by means of firmware).
[0102] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0103] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0104] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0105] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0106] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0107] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
[0108] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.
[0109] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for training a tracking model, characterized in that: The method comprises: Determine at least two candidate data groups; wherein the same candidate data group includes data formed by annotating the same type of tracking objects; Determine a first number of types required for training the tracking model, and determine the number of candidate image pairs; determine a second number of target image pairs corresponding to each target tracking object type based on a preset ratio and the number of candidate image pairs; Selecting a candidate data group corresponding to the target tracking object type from the candidate data group as a target data group, and determining a second number of target template images and target search images corresponding to the target tracking object type based on the target data in the target data group and the second number; The target template image and the target search image are matched according to a preset matching method to determine a second number of target image pairs corresponding to the target tracking object type, and the target image pairs are used to train the tracking model; wherein all objects of the same target tracking object type contained in the same target search image are tracking objects.
2. The method according to claim 1, characterized in that Identify at least two candidate data sets, including: Acquire reference data for tracking model training, wherein the reference data includes video data or image data; Marking the tracking object in the reference data to determine the type of the tracking object corresponding to the reference data; The reference data sets that belong to the same type of tracked objects are grouped together as candidate data sets.
3. The method according to claim 1, characterized in that Determining a second number of target image pairs corresponding to each target tracking object type based on a preset ratio and the number of the candidate image pairs includes: The ratio of the number of the candidate image pairs to the first number is used as the second number of target image pairs corresponding to each target tracking object type.
4. The method according to claim 1, characterized in that Determining a second number of target template images and target search images corresponding to the target tracking object type based on the target data in the target data group and the second number, comprising: Selecting the second number of target data from the target data group as reference training data, wherein the reference training data includes video data and / or image data; Determine a second number of target template images corresponding to the target tracking object type based on the reference training data, and mark a tracking object in the same target template image; A second number of target search images corresponding to the target tracking object type is determined based on the reference training data, and all objects of the same target tracking object type contained in the same target search image are tracking objects.
5. The method according to claim 4, characterized in that If the reference training data is image data, determining a second number of target template images corresponding to the target tracking object type based on the reference training data includes: Taking each image in the reference training data as a first original image, where the first original image is an image after augmentation processing; Taking a tracking object belonging to the target tracking object type in the first original image as a first reference object, and taking a position where the first reference object is located as a first reference position; Based on the first reference object and the first reference position, processing the first original image in a preset processing manner to determine a target template image; Correspondingly, if the reference training data is video data, determining a second number of target template images corresponding to the target tracking object type based on the reference training data includes: Selecting an image from each video in the reference training data as a second original image, where the second original image is an image after augmentation processing; taking a tracking object belonging to the target tracking object type in the second original image as a second reference object, and taking a position where the second reference object is located as a second reference position; Based on the second reference object and the second reference position, the second original image is processed in a preset processing manner to determine a target template image.
6. The method according to claim 4, characterized in that If the reference training data is image data, determining a second number of target search images corresponding to the target tracking object type based on the reference training data includes: Using each image in the reference training data as a third original image, wherein the third original image is an image after augmentation processing; Processing the third original image according to a preset processing method to determine a target search image, and taking all objects of the same target tracking object type contained in the target search image as target tracking objects; Correspondingly, if the reference training data is video data, determining a second number of target search images corresponding to the target tracking object type based on the reference training data includes: Selecting an image from each video in the reference training data as a fourth original image, wherein the fourth original image is an image after augmentation processing; The fourth original image is processed according to a preset processing method to determine a target search image, and all objects of the same target tracking object type contained in the target search image are taken as target tracking objects.
7. The method according to claim 1, characterized in that Matching the target template image and the target search image according to a preset matching method to determine a second number of target image pairs corresponding to the target tracking object type, including: Determining a second number of arrangement labels, wherein the arrangement labels are different from each other; Marking the second number of target template images respectively according to the arrangement numbers to obtain marked target template images; Marking a second number of target search images according to the arrangement labels to obtain marked target search images; The target template image and the target search image with the same arrangement number are regarded as a target image pair.
8. A training device for a tracking model, characterized in that: The device comprises: A data group determination module, used to determine at least two candidate data groups; wherein the same candidate data group includes data formed by annotating the same type of tracked objects; A quantity determination module, used to determine a first number of types required for training the tracking model and determine the number of candidate image pairs; determine a second number of target image pairs corresponding to each target tracking object type based on a preset ratio and the number of candidate image pairs; an image determination module, configured to select a candidate data group corresponding to the target tracking object type from the candidate data group as a target data group, and determine a second number of target template images and target search images corresponding to the target tracking object type based on the target data in the target data group and the second number; An image pair training module is used to match the target template image and the target search image according to a preset matching method to determine a second number of target image pairs corresponding to the target tracking object type, and use the target image pairs to train the tracking model; wherein all objects of the same target tracking object type contained in the same target search image are tracking objects.
9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the tracking model training method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the training method of the tracking model described in any one of claims 1 to 7 when executed.