Feature set extension method and medium

Abstract

The application discloses a feature set expansion method and a medium; wherein, the method comprises: acquiring the i-th feature data; wherein, the i-th feature data and the feature data in the feature set are associated with the same type of object; the feature data in the feature set comprises a verification feature for identifying the same type of object; i is an integer greater than or equal to 1; at least part of the data in the i-th feature data is subjected to tracking fusion processing to obtain the j-th feature data; wherein, the association degree between the j-th feature data and the feature data in the feature set is greater than an association threshold; j is a positive integer different from i; the j-th feature data is added to the feature set to expand the feature set; wherein, the expanded feature set is used for identifying the same type of object.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of object recognition, and in particular to a feature set expansion method and a medium. BACKGROUND

[0002] In the field of person re-identification (ReID), a tracking template is used to determine whether a detected pedestrian target is a new pedestrian, and therefore, the number of tracking templates and the feature dimensions represented thereby are particularly important for the recognition and tracking accuracy of ReID. In related technologies, new feature data that is more than a matching threshold in matching degree with existing feature data in a tracking template is added to the tracking template to expand the tracking template. However, such a tracking template expansion method cannot meet actual target tracking requirements. SUMMARY

[0003] To address the above problems, embodiments of the present application provide a feature set expansion method and a medium.

[0004] The technical solutions provided by the embodiments of the present application are as follows:

[0005] The embodiments of the present application provide a feature set expansion method, which comprises the following steps:

[0006] obtaining i-th feature data; wherein the i-th feature data and feature data in a feature set are associated with the same type of object; the feature data in the feature set comprises verification features for identifying the same type of object; i is an integer greater than or equal to 1;

[0007] performing tracking fusion processing on at least part of the data in the i-th feature data to obtain j-th feature data; wherein the association degree between the j-th feature data and the feature data in the feature set is greater than an association threshold; j is a positive integer different from i;

[0008] adding the j-th feature data to the feature set to expand the feature set; wherein the expanded feature set is used for identifying the same type of object.

[0009] In some embodiments, the performing tracking fusion processing on at least part of the data in the i-th feature data to obtain j-th feature data comprises:

[0010] performing tracking fusion processing on at least part of the data in the i-th feature data based on the feature data in the feature set to obtain the j-th feature data.

[0011] In some embodiments, the ith feature data at least includes mth feature data and nth feature data, where m and n are different positive integers; the tracking fusion processing on at least part of the data in the ith feature data to obtain jth feature data includes:

[0012] tracking fusion processing on at least part of the data in the mth feature data to obtain a first feature component;

[0013] tracking fusion processing on at least part of the data in the nth feature data to obtain a second feature component;

[0014] processing the first feature component and the second feature component to obtain the jth feature data.

[0015] In some embodiments, the processing the first feature component and the second feature component to obtain the jth feature data includes:

[0016] determining a weight parameter;

[0017] based on the weight parameter, performing weighted fusion on the first feature component and the second feature component to obtain the jth feature data.

[0018] In some embodiments, the tracking fusion processing on at least part of the data in the ith feature data to obtain jth feature data includes:

[0019] tracking identification and fusion processing on at least part of the data in the ith feature data by a mapping network to obtain the jth feature data;

[0020] The mapping network is obtained by training in the following way:

[0021] based on a mapping network of an intermediate state on feature samples, the mapping network is trained by a target re-identification FAST-ReID model and a text-image matching pre-training (Contrastive Language-Image Pretraining, CLIP) model to obtain the mapping network.

[0022] In some embodiments, the mapping network is obtained by training based on a mapping network of an intermediate state on feature samples by a FAST-ReID model and a CLIP model, including:

[0023] obtaining first data and second data from the feature samples;

[0024] performing feature extraction on the first data and the second data respectively by the FAST-ReID model to obtain first extraction results and second extraction results;

[0025] The first extraction result and the second extraction result are processed by the mapping network of the intermediate state respectively to obtain a first result and a second result.

[0026] The first data and the second data are subjected to feature extraction by the CLIP model to obtain a third result.

[0027] Based on the first result, the second result and the third result, the parameters of the mapping network of the intermediate state are adjusted to obtain the mapping network.

[0028] In some embodiments, the adjusting of the parameters of the mapping network of the intermediate state based on the first result, the second result and the third result to obtain the mapping network comprises:

[0029] The first result and the second result are subjected to weighted fusion processing to obtain a first fusion result.

[0030] The first fusion result is processed by the mapping network of the intermediate state to obtain a mapping result.

[0031] Based on the matching degree between the third result and the mapping result, the parameters of the mapping network of the intermediate state are adjusted to obtain the mapping network.

[0032] In some embodiments, the feature extraction of the first data and the second data by the CLIP model to obtain a third result comprises:

[0033] The first data and the second data are subjected to weighted fusion processing to obtain a second fusion result.

[0034] The second fusion result is subjected to feature extraction by the CLIP model to obtain the third result.

[0035] In some embodiments, the mapping network of the intermediate state is obtained by training in the following manner:

[0036] Third data is obtained from the feature sample;

[0037] The third data is subjected to feature extraction by the FAST-ReID model to obtain a third extraction result.

[0038] The third extraction result is processed by the mapping network of the initial state to obtain a fourth result.

[0039] The third data is subjected to feature extraction by the CLIP model to obtain a fifth result.

[0040] based on a matching degree between the fourth result and the fifth result, adjusting parameters of the mapping network of the initial state to obtain the mapping network of the intermediate state.

[0041] The embodiment of the application further provides a computer readable storage medium, wherein the storage medium stores a computer program; when the computer program is executed by a processor of an electronic device, the computer program can realize the feature set extension method.

[0042] The feature set extension method provided by the embodiment of the application can reduce redundant features in the ith feature data by tracking and fusing at least part of the feature data in the ith feature data, improve the association degree between the jth feature data and the feature data in the feature set, and since the ith feature data and the feature data in the feature set are associated with the same type of object, after adding the jth feature data to the feature set, the feature set after extension can improve the feature representation accuracy of the verification feature in the feature set, reduce the redundancy of the feature data, and enhance the association degree between the feature data, so that the negative influence of determining whether to add new feature data to the feature set by using a fixed threshold and the feature redundancy and feature interference generated by the feature set can be weakened. When the feature set is a template feature library, the template feature library is applied to the ReID field, and the same type of object is a pedestrian, the feature set extension method provided by the embodiment of the application can reduce the number of redundant features and interference features in the feature set extension process, improve the pertinence and flexibility of the feature set extension, and thus can greatly improve the pedestrian recognition accuracy and reduce the target loss probability. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 A flowchart of template feature updating provided by the related art is shown;

[0044] Figure 2 A flowchart of the feature set extension method provided by the embodiment of the application is shown;

[0045] Figure 3 A structural diagram of the bounding box detection diagram provided by the embodiment of the application is shown;

[0046] Figure 4 A flowchart of the parameter adjustment of the mapping network of the intermediate state provided by the embodiment of the application is shown;

[0047] Figure 5A schematic diagram of the mapping network for the initial training state provided in an embodiment of this application;

[0048] Figure 6 This is a schematic diagram of the structure of the feature template provided in the embodiments of this application;

[0049] Figure 7 This is a schematic diagram of the structure for obtaining a new feature template through a mapping network, provided in an embodiment of this application.

[0050] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0051] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0052] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0053] In practical ReID applications, factors such as pedestrian pose, image acquisition angle, and viewpoint occlusion can all affect target tracking accuracy. Even for the same pedestrian, changes in appearance will alter their corresponding feature data. If the feature data extracted from the pedestrian image is matched using pre-defined template features, the probability of a match failure is high, leading to target loss. To reduce the probability of target loss, related technologies have provided template feature update techniques.

[0054] Figure 1 A flowchart illustrating the template feature update process for related technologies. For example... Figure 1 As shown, after image sequence 101 is input to target detection module 102, target detection module 102 can detect targets in each frame of image sequence 101. Motion camera 103 is used to estimate the target position in the next frame of image sequence 101. After the estimation result of motion camera 103 is input to Kalman filter module 104, this module estimates the position of the target in the tracking template in the next frame of image with the help of tracking template 105, that is, obtains the predicted bounding box. At the same time, appearance feature extraction module 106 can extract the appearance features contained in the output result of target detection module 102 through neural network. Thus, matching calculation module 107 performs matching calculation on predicted bounding box and appearance features, and then sends the matching calculation result to template update module 108, and template update module 108 determines whether to update the above appearance features to the tracking template, thus obtaining the updated tracking template 109.

[0055] In actual applications, the template updating module 108 usually directly adds the appearance feature extracted by the appearance feature extraction module into the tracking template when the matching degree between the appearance feature extracted by the appearance feature extraction module and the feature data in the tracking template is greater than a matching threshold.

[0056] In Figure 1 In the flow shown, a neural network with better feature extraction performance is used in the appearance feature extraction module, so that the appearance feature of the pedestrian can be stably extracted even when the appearance of the pedestrian changes greatly; and through the above flow, the number of appearance features contained in the tracking template can be expanded to a certain extent, so that the probability of target loss can be reduced to a certain extent even when the appearance feature of the pedestrian changes.

[0057] However, in actual ReID, the optimal matching threshold corresponding to different recognition tasks and pedestrian objects is not unique, so in the above scheme, the selection of the fixed threshold is one-sided in the process of determining whether to add a new appearance feature to the feature template, and the fixed threshold cannot realize comprehensive expansion and tracking of the tracking template, so the recognition accuracy of ReID cannot be improved.

[0058] Based on the above problems, an embodiment of the present application provides a feature set expansion method and medium.

[0059] It should be noted that the feature set expansion method provided by the embodiment of the present application can be realized by a processor of an electronic device. The processor can be at least one of an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, and a microprocessor.

[0060] Exemplarily, the electronic device can be a physical machine device or a virtual machine device.

[0061] Exemplarily, the electronic device can be a computer device.

[0062] Figure 2 The flowchart of the feature set expansion method provided by the embodiment of the present application is shown in FIG. 1.Figure 2 As shown, the flow can include the following steps:

[0063] Step 201, obtaining the i-th feature data.

[0064] The i-th feature data is associated with feature data in the feature set to the same type of object; the feature data in the feature set includes a verification feature for identifying the same type of object; i is an integer greater than or equal to 1.

[0065] In an embodiment, the i-th feature data can include feature data obtained by identifying pedestrian images or videos; for example, a neural network can be used to extract features from pedestrian images or videos, thereby obtaining the i-th feature data.

[0066] In an embodiment, the i-th feature data can include posture features and / or shape features of pedestrians; for example, the posture features can include walking posture and / or hand gestures of pedestrians; the shape features can include at least one of the height, fatness, and clothing of pedestrians.

[0067] In an embodiment, the same type of object can include stationary objects such as buildings, mountains, etc.

[0068] In an embodiment, the same type of object can include objects with the ability to move or move, such as pedestrians and flowing water, etc.

[0069] In an embodiment, the verification feature can include feature data used for classifying and / or attribute judgment of newly extracted feature data.

[0070] In an embodiment, the feature set can include verification features of multiple objects, which can represent the features of multiple objects from multiple dimensions in multiple scenarios; for example, the multiple objects can include pedestrians.

[0071] Step 202, performing tracking fusion processing on at least part of the i-th feature data to obtain the j-th feature data.

[0072] The degree of association between the j-th feature data and the feature data in the feature set is greater than the association threshold; j is a positive integer different from 1.

[0073] In an embodiment, the data amount of the j-th feature data can be less than or equal to the data amount of the i-th feature data, and the dimension of the j-th feature data can be less than or equal to the dimension of the i-th feature data.

[0074] In an embodiment, the tracking fusion processing of the at least part of the i-th feature data to obtain the j-th feature data can be implemented by any of the following ways:

[0075] The selective extraction of the data in the i-th feature data to obtain the at least part of the feature data, and the fusion enhancement processing of the at least part of the feature data to obtain the j-th feature data. Accordingly, the association degree between the j-th feature data and the feature data in the feature set can be greater than the association threshold, which can include that the j-th feature data and the feature data in the feature set are respectively used to represent the specified part of the object of the same type or the feature in the specified scene.

[0076] The tracking search of the data in the i-th feature data to obtain the at least part of the feature data, and the fusion enhancement processing of the at least part of the feature data based on the preset feature enhancement strategy to obtain the j-th feature data. Accordingly, the feature represented by the j-th feature data can be the associated feature of the feature represented by the feature data in the feature set, such as the arm feature represented by the feature data in the feature set and the hand feature represented by the j-th feature data. At this time, the j-th feature data and the feature data in the feature set are associated as the verification feature of the arm and the hand recognition.

[0077] In an embodiment, the association degree can be determined by matching and comparing the j-th feature data and at least part of the verification feature in the feature set.

[0078] In an embodiment, the association degree can be determined before the tracking fusion processing of the at least part of the i-th feature data.

[0079] In an embodiment, the association threshold can be determined in advance, or can be adjusted or determined according to at least one of the type of the object corresponding to the feature set, the scene associated with the verification feature, and the verification accuracy.

[0080] Step 203, adding the j-th feature data to the feature set to expand the feature set.

[0081] The expanded feature set is used to identify the same type of object.

[0082] In an embodiment, the adding of the j-th feature data to the feature set can be implemented by any of the following ways:

[0083] The jth feature data is added into a feature data subset corresponding to the feature category in the feature set according to the feature category. The feature category can include a posture category, a part category, a scene category, and the like. The posture category can include a walking posture and / or a gesture posture. The part category can include a category of a body part of a pedestrian, such as a head, a leg, and the like. The scene category can include a public transportation scene and a dining scene.

[0084] The jth feature data is added into a feature data subset corresponding to the object identifier in the feature set according to the object identifier. The object identifier can include a pedestrian identifier, and the pedestrian identifier can include a number and / or a name of the pedestrian.

[0085] In an embodiment, the extended feature set includes a verification feature that has a higher recognition accuracy for the same type of object under at least one of a scene migration, a posture migration, and a part migration, compared with the verification feature included in the feature set. The scene migration includes a migration of the same type of object from a first type of scene to a second type of scene. The posture migration includes a migration of the same type of object from a first type of posture to a second type of posture. The part migration includes a migration of the same type of object from a first part to a second part.

[0086] In an embodiment, the extended feature set is extended in at least one dimension or has an improved degree of refinement in at least one dimension, compared with the feature set. In this case, when the same type of object is identified based on the extended feature set, the recognition efficiency for the same type of object can be improved in terms of a variety of scenes, a variety of feature categories, a degree of refinement of feature recognition, and an accuracy of feature recognition.

[0087] From the above, the feature set extension method provided in the embodiments of the present application can track and fuse at least part of the feature data in the ith feature data after obtaining the ith feature data associated with the feature data in the feature set to the same object, and the association degree between the jth feature data obtained and the feature data in the feature set is greater than the association threshold. In this way, by tracking and fusing at least part of the feature data in the ith feature data, the redundant features in the ith feature data can be reduced, and the association degree between the jth feature data and the feature data in the feature set can be improved. Moreover, since the ith feature data and the feature data in the feature set are associated with the same type of object, after adding the jth feature data to the feature set, the feature representation accuracy of the verification features in the extended feature set can be improved, the redundancy of the feature data can be reduced, and the association degree between the feature data can be enhanced, thereby weakening the negative effects of determining whether to add new feature data to the feature set by using a fixed threshold, and the feature redundancy and feature interference caused by the feature set. When the feature set is a template feature library, the template feature library is applied to the ReID field, and the same type of object is a pedestrian, the feature set extension method provided in the embodiments of the present application can reduce the number of redundant features and interference features during the feature set extension process, improve the pertinence and flexibility of the feature set extension, and thus greatly improve the accuracy of pedestrian recognition and reduce the probability of target loss.

[0088] Based on the foregoing embodiments, in the feature set extension method provided in the embodiments of the present application, the tracking and fusion processing is performed on at least part of the data in the ith feature data to obtain the jth feature data, which can be implemented in the following manner:

[0089] Based on the feature data in the feature set, the tracking and fusion processing is performed on at least part of the data in the ith feature data to obtain the jth feature data.

[0090] In an implementation manner, the tracking and fusion strategy can be determined based on the distribution state of the feature data in the feature set, and the tracking and fusion processing is performed on at least part of the data in the ith feature data according to the tracking and fusion strategy. Exemplarily, the distribution state can include the distribution number of the feature data in the feature set in terms of scene, part, and posture.

[0091] For example, if the number of feature data in the feature set in a specified scene is small, the tracking and fusion strategy can be determined as follows: the feature data in the ith feature data is screened and tracked according to the feature information of the specified scene, and the result of the screening and tracking is fused.

[0092] For example, if the feature set does not contain feature data of the specified part, feature information of the specified part can be determined, and the tracking fusion strategy is determined as: screening and tracking the feature data in the ith feature data according to the feature information of the specified part, and then performing fusion processing on the result of the screening and tracking.

[0093] As can be seen from the above, in the feature set extension method provided by the embodiments of the present application, the jth feature data is obtained by performing tracking fusion processing on at least part of the features in the ith feature data based on the feature data in the feature set. In this way, based on the feature data in the feature set, the direction, target and degree of the tracking fusion processing can be determined, thereby improving the pertinence of the tracking fusion processing on at least part of the feature data in the ith feature data. In this way, not only the efficiency of the tracking fusion processing can be improved, but also the correlation degree between the jth feature data and the feature data in the feature set can be improved.

[0094] Based on the foregoing embodiments, in the feature set extension method provided by the embodiments of the present application, the ith feature data at least includes the mth feature data and the nth feature data.

[0095] Wherein, m and n are different positive integers.

[0096] In an implementation manner, the mth feature data and the nth feature data can be feature data of the same object, such as the same target pedestrian. For example, the mth feature data and the nth feature data can correspond to different dimensions of the same target pedestrian, such as the mth feature data corresponding to the head feature of the target pedestrian and the nth feature data corresponding to the leg feature of the target pedestrian. For another example, the mth feature data corresponds to the feature of the target pedestrian in a first scene, and the nth feature data corresponds to the feature of the target pedestrian in a second scene.

[0097] For example, the tracking fusion processing on at least part of the feature data in the ith feature data to obtain the jth feature data can be implemented by the following steps:

[0098] Step A1, performing tracking fusion processing on at least part of the feature data in the mth feature data to obtain a first feature component, and performing tracking fusion processing on at least part of the feature data in the nth feature data to obtain a second feature component.

[0099] In an implementation, the first feature component and the second feature component can be in the same or different feature dimensions, and the degree of association between the first feature component and the second feature component and the feature data in the feature set is greater than the association threshold, such as the first feature component and the second feature component are both in the dimension of walking posture, and the degree of association between the first feature component and the second feature component and the feature data in the feature set is greater than the association threshold; for example, the first feature component can be in the height dimension, and the second feature component can be in the walking posture dimension, and the degree of association between the first feature component and the second feature component and the feature data in the feature set is greater than the association threshold.

[0100] In an implementation, the obtaining process of the first feature component and the second feature component is the same as the obtaining process of the jth feature data in the foregoing embodiments, which will not be repeated here.

[0101] Step A2, processing the first feature component and the second feature component to obtain the jth feature data.

[0102] In an implementation, the jth feature data can be obtained by any of the following ways:

[0103] Based on the feature parts or dimensions represented by the first feature component and the second feature component respectively, the first feature component and the second feature component are associated and fused in parts or dimensions to obtain the jth feature data; for example, the first feature component represents the arm feature of the pedestrian, and the second feature component represents the hand feature of the pedestrian, then the first feature component and the second feature component can be spliced and fused according to the association relationship between the arm and the hand to obtain the jth feature data.

[0104] If the feature parts or dimensions represented by the first feature component and the second feature component are the same, the first feature component and the second feature component are superimposed and fused to obtain the jth feature data.

[0105] As can be seen from the above, in the feature set expansion method provided by the embodiments of the present application, the ith feature data at least includes the mth feature data and the nth feature data, and after at least part of the data in the mth feature data and at least part of the data in the nth feature data are tracked and fused, the first feature component and the second feature component are obtained, thereby realizing the targeted screening, tracking and fusion processing of the mth feature data and the nth feature data, and further reducing the redundancy of the features in the first feature component and the second feature component; and the first feature component and the second feature component are fused to obtain the jth feature data, so that the jth feature data can comprehensively reflect the effective features contained in the mth feature data and the nth feature data, thereby further improving the recognition accuracy of the jth feature data for the same type of object.

[0106] Based on the foregoing embodiments, the feature set extension method provided in the embodiments of the present application can achieve the following by processing the first feature component and the second feature component to obtain the jth feature data:

[0107] determining a weight parameter, and performing weighted fusion on the first feature component and the second feature component based on the weight parameter to obtain the jth feature data.

[0108] In an embodiment, the weight parameter can include a plurality of weight values. For example, the number of weight values can be the same as the number of feature components to be fused.

[0109] In an embodiment, the sum of the weight values in the weight parameter can be 1.

[0110] In an embodiment, the weight values in the weight parameter can be predetermined. For example, the weight values can also change with changes in feature categories. For example, the weight value corresponding to the head feature can be a first numerical value, and the weight value corresponding to the leg feature can be a second numerical value. For example, the weight values corresponding to different object features can be different. For example, the weight value corresponding to the feature data of a child can be different from the weight value corresponding to the feature data of an elderly person.

[0111] In an embodiment, the weight parameter can be determined according to feature fusion requirements. For example, the feature fusion requirements can include requirements for filtering, filtering, weakening, or strengthening of feature data.

[0112] In an embodiment, the jth feature data can be obtained by:

[0113] performing weighted summation on the first feature component and the second feature component based on the weight values in the weight parameter, and determining the result of the weighted summation as the jth feature data.

[0114] As can be seen from the above, in the feature set extension method provided in the embodiments of the present application, after the weight parameter is determined, the jth feature data is obtained by performing weighted fusion on the first feature component and the second feature component based on the weight parameter. Therefore, by adjusting the weight parameter, targeted and flexible filtering and processing of the feature data in the first feature component and the second feature component can be achieved. Furthermore, through the above weighted fusion operation, feature association and feature aggregation of the feature data in the first feature component and the second feature component can be achieved, thereby improving the consistency between the jth feature data and the actual features of the same type of object, such as a pedestrian, and further improving the accuracy of the jth feature data.

[0115] Based on the foregoing embodiments, the feature set extension method provided by the embodiments of the present application can realize the tracking fusion processing of at least part of the feature data in the i-th feature data to obtain the j-th feature data in the following manner:

[0116] The mapping network is configured to track, filter, extract, and fuse the target feature in the i-th feature data based on the tracking fusion strategy contained in the mapping network.

[0117] In an embodiment, the mapping network is configured to track, filter, extract, and fuse the target feature in the i-th feature data based on the tracking fusion strategy contained in the mapping network.

[0118] In an embodiment, the mapping network can analyze whether the i-th feature data and the feature data in the feature set have correlation or the strength of the correlation degree according to the tracking fusion strategy contained in the mapping network.

[0119] For example, whether the i-th feature data and the feature data in the feature set have correlation can include whether the i-th feature data and the feature data in the feature set are associated with the same object, such as the same pedestrian.

[0120] For example, the strength of the correlation degree between the i-th feature data and the feature data in the feature set can be determined by whether the i-th feature data and the feature data in the feature set have part association, such as the trunk part of a pedestrian being adjacent to the arm part, so that it can be determined that the feature data of the trunk part and the feature data of the arm part have a correlation relationship with a strength greater than a correlation threshold.

[0121] In an embodiment, the j-th feature data can be obtained in any of the following manners:

[0122] The mapping network determines at least one target feature contained in the feature set. The mapping network tracks and filters the feature data in the i-th feature data to determine whether the i-th feature data contains the at least one target feature. If the i-th feature data contains the at least one target feature, the mapping network fuses the feature data in the i-th feature data containing the at least one target feature, and determines the result of the fusion processing as the j-th feature data.

[0123] The mapping network determines the associated feature of the at least one target feature, and tracks the feature data in the i-th feature data based on the associated feature to determine whether the i-th feature data contains the associated feature. If the i-th feature data contains the associated feature, the mapping network fuses the feature data in the i-th feature data containing the associated feature, and determines the result of the fusion processing as the j-th feature data.

[0124] Correspondingly, the mapping network can be trained in the following manner:

[0125] The mapping network of the intermediate state is trained based on the feature sample pair by the FAST-ReID model and the CLIP model to obtain the mapping network.

[0126] In an embodiment, the mapping network of the intermediate state can include a network model obtained after preliminary training of the tracking network of the initial state.

[0127] In an embodiment, the feature sample can include multiple images, which can correspond to at least one of multiple pedestrians, multiple scenes, multiple parts, multiple angles, and multiple postures.

[0128] The CLIP model is an open-source network model used to extract feature codes of texts or images. The model is powerful and can efficiently and accurately extract feature codes of images and map the feature codes in a hidden space. However, the model cannot be applied in the actual ReID field due to its complex training process, large amount of calculation, and slow running.

[0129] The FAST-ReID model is a collection tool box of ReID methods, which can convert the bounding box diagram shown in FIG. 1 into a feature vector representing the appearance, i.e., appearance features. Figure 3 The Fast-ReID is widely used in target tracking tasks due to its high operation speed. Figure 3 The structure diagram of the bounding box diagram provided by the embodiments of the present application is shown in FIG. 1. Figure 3 As shown in FIG. 1, the image frame diagram 301 is a frame formed by the edge pixels of any image, and the detection frame diagram 302 is a rectangular frame formed by the pixel area of the pedestrian object contained in any image.

[0130] In actual application, although the operation speed of the FAST-ReID model is fast, the feature detection accuracy is low. Therefore, the embodiments of the present application train the mapping network of the intermediate state by combining the FAST-ReID model and the CLIP model. On the one hand, by virtue of the high-precision feature recognition characteristics of the CLIP model, the training effect of the mapping network of the intermediate state can be improved, and the accuracy of the data tracking and fusion processing of the mapping network on the i-th feature data can be improved. On the other hand, by virtue of the fast operation characteristics of the FAST-ReID model, the training speed of the mapping network of the intermediate state can be accelerated. At the same time, by virtue of the mapping network trained by the FAST-ReID model and the CLIP model, effective features in the i-th feature data can be stably and efficiently obtained in the process of tracking and identifying and fusing at least part of the data in the i-th feature data.

[0131] Based on the foregoing embodiments, the feature set extension method provided by the embodiments of the present application includes the following steps:

[0132] Step B1, obtaining first data and second data from the feature samples.

[0133] In an embodiment, the first data and the second data can be any image in the feature samples.

[0134] In an embodiment, the first data and the second data can represent the same or different feature dimensions of the same pedestrian, and can also correspond to the same or different scenes in which the same pedestrian is located.

[0135] Step B2, performing feature extraction on the first data and the second data respectively by using the FAST-ReID model to obtain a first extraction result and a second extraction result.

[0136] In an embodiment, the FAST-ReID model can be used to sequentially perform feature extraction on the first data and the second data to obtain the first extraction result and the second extraction result respectively.

[0137] Step B3, processing the first extraction result and the second extraction result respectively by using the mapping network of the intermediate state to obtain a first result and a second result.

[0138] In an embodiment, the mapping network of the intermediate state can be used to sequentially perform feature tracking on the first extraction result and the second extraction result to obtain the first result and the second result respectively.

[0139] Step B4, performing feature extraction on the first data and the second data by using the CLIP model to obtain a third result.

[0140] In an embodiment, the CLIP model can be used to sequentially perform feature tracking on the first data and the second data to obtain the third result.

[0141] Step B5, adjusting parameters of the mapping network of the intermediate state based on the first result, the second result and the third result to obtain the mapping network.

[0142] In an embodiment, the mapping network can be obtained by the following method:

[0143] Based on the difference between the first result and the third result in at least one aspect, the first part of the parameters of the mapping network of the intermediate state is adjusted, based on the difference between the second result and the third result in at least one aspect, the second part of the parameters of the mapping network of the intermediate state is adjusted, and then the mapping network of the intermediate state with the first part of the parameters and the second part of the parameters adjusted is determined as the mapping network.

[0144] Based on the difference between the first result, the second result and the third result, the parameters of the mapping network of the intermediate state are adjusted, so that the feature extraction result of the feature extraction branch composed of the FAST-ReID model and the mapping network of the intermediate state can approximate the feature extraction result of the CLIP model, thereby realizing the fitting of the CLIP model by the FAST-ReID model and the mapping network of the intermediate state, that is, the mapping network of the intermediate state at this time can realize the mapping between the feature extraction result of the FAST-ReID model and the feature extraction result of the CLIP model.

[0145] From the above, in the feature set expansion method provided by the embodiments of the present application, the FAST-ReID model is used to respectively perform feature extraction on the first data and the second data to obtain the first extraction result and the second extraction result, and then the mapping network of the intermediate state is used to respectively process the first extraction result and the second extraction result to obtain the first result and the second result. Since the FAST-ReID model has a relatively fast operation speed, the operation process of obtaining the first result and the second result by connecting the FAST-ReID model and the mapping network of the intermediate state in series can not only realize the mapping tracking processing of the first extraction result and the second extraction result output by the FAST-ReID model, but also can weaken the negative influence of the time consumed by the above mapping tracking processing on model training. In addition, in the process of performing feature extraction on the first data and the second data by the CLIP model to obtain the third result, the accuracy of the third result can be improved by virtue of the superiority of the CLIP model in feature extraction. At the same time, the parameters of the mapping network of the intermediate state are adjusted based on the first result, the second result and the third result to obtain the mapping network, and the tracking mapping accuracy of the mapping network between the feature extraction results of the FAST-ReID model and the CLIP model can be improved by virtue of the superiority of the FAST-ReID model and the CLIP model in operation speed and feature extraction, respectively.

[0146] Based on the foregoing embodiments, in the feature set expansion method provided by the embodiments of the present application, the parameters of the mapping network of the intermediate state are adjusted based on the first result, the second result and the third result to obtain the mapping network, which can be realized in the following way:

[0147] The first result and the second result are weighted and fused to obtain a first fusion result; the first fusion result is processed by the mapping network of the intermediate state to obtain a mapping result; parameters of the mapping network of the intermediate state are adjusted based on a matching degree between the third result and the tracking result to obtain the mapping network.

[0148] In an implementation, the above-mentioned weighted fusion processing can be implemented based on the weight parameter in the foregoing embodiments, that is, the first result and the second result are weighted and summed by weight values in the weight parameter to obtain the first fusion result.

[0149] In an implementation, the mapping result is obtained in the same way as the first result and the second result in the foregoing embodiments.

[0150] In an implementation, the parameters of the mapping network of the intermediate state can be adjusted based on a difference between the same features between the third result and the mapping result or a feature difference corresponding to the same pixel point, so as to obtain the mapping network.

[0151] In an implementation, a loss function can be defined, and the difference between the mapping result and the third result is tracked and counted by the loss function, and in a case where the difference between the mapping result and the third result is less than or equal to a difference threshold, the adjustment of the parameters of the mapping network of the intermediate state can be stopped, so as to obtain the mapping network.

[0152] As can be seen from the above, in the feature set expansion method provided by the embodiments of the present application, the first fusion result is obtained by weighted and fused processing of the first result and the second result, so that the first fusion result can carry more comprehensive and rich feature information; and the mapping result is obtained by processing the first fusion result by the mapping network of the intermediate state, so that the mapping result can contain effective feature information in the first fusion result, thereby reducing the redundancy of the mapping result; at the same time, based on the matching degree between the third result and the mapping result, the parameters of the mapping network of the intermediate state are adjusted, and by virtue of the richness of the feature information in the mapping result and the accuracy of the features in the third result, the mapping network of the intermediate state can be quickly and accurately and targetedly adjusted, thereby improving the accuracy and stability of the feature tracking and fusion of the mapping network.

[0153] Based on the foregoing embodiments, in the feature set expansion method provided by the embodiments of the present application, the third result is obtained by performing feature extraction on the first data and the second data by the CLIP model, which can be implemented in the following way:

[0154] The first data and the second data are weighted and fused to obtain a second fusion result; the second fusion result is subjected to feature extraction by the CLIP model to obtain the third result.

[0155] In an implementation manner, the second fusion result can be obtained by any of the following manners:

[0156] The first data and the second data are weighted and fused by the weight parameter provided in the foregoing embodiments, so as to obtain the second fusion result.

[0157] The corresponding pixel points in the first data and the second data are weighted and fused by the weight parameter, so as to obtain the second fusion result.

[0158] The associated pixel features in the first data and the second data are weighted and fused based on the weight parameter, so as to obtain the second fusion result.

[0159] As can be seen from the above, in the feature set expansion method provided in the embodiments of the present application, the second fusion result is obtained by weighted and fused processing of the first data and the second data, so that the second fusion result can contain the pixel features contained in the first data and the second data, thereby the richness of the feature information in the second fusion result can be improved; and the third result is obtained by feature extraction of the second fusion result by the CLIP model, so that the third result can contain the pixel features of the first data and the second data, and the efficiency of obtaining the third result can be improved compared with sequentially extracting features of the first data and the second data by the CLIP model.

[0160] Figure 4 A flowchart for adjusting parameters of the intermediate state mapping network provided in the embodiments of the present application is shown. In the embodiments of the present application, the first data and the second data can be a first image and a second image respectively.

[0161] As shown in Figure 4 The first image input module 401 can input the first data and the second data obtained from the feature sample to the FAST-ReID model 402 respectively, so that the first extraction result and the second extraction result output by the FAST-ReID model 402 can be obtained; the first extraction result and the second extraction result are input to the intermediate state mapping network 403, and after processing of the intermediate state mapping network 403, the first result and the second result can be obtained, and the first result and the second result are weighted and fused based on the weight parameter by the feature fusion module 404, so that the first fusion result can be obtained.

[0162] At the same time, the first image input module 401 can simultaneously input the first data and the second data to the image fusion module 405, so that the image fusion module can weight and fuse the first data and the second data based on the weight parameter, so as to obtain the second fusion result; the second fusion result is input to the CLIP model 406, and after feature extraction operation of the CLIP model 406, the third result can be obtained.

[0163] As Figure 4 shown, the mapping network 403 in the intermediate state can process the first fusion result to obtain a mapping result, and then adjust the parameters of the mapping network 403 in the intermediate state according to the mapping result and the third result, so as to obtain the mapping network.

[0164] In the above training process, the mapping result and the third result can be processed by a loss function, so as to determine the stop training condition of the mapping network in the intermediate state. The loss function can be as shown in formula (1):

[0165] Loss=||ξ[(1-t)·I i +t·I i ]-[(1-t)·F(G(I i ))+t·F(G(I j ))]|| 2 (1)

[0166] Wherein, 1-t and t can be the weight value in the weight parameter in the foregoing embodiments; I i and I j can be the first data and the second data in the foregoing embodiments. As can be seen from formula (4), the linear weighting of I i and I j is realized by the weight parameters 1-t and t, so the weight parameter can also be called a linear interpolation parameter.

[0167] In formula (4), (1-t)·I i +t·I j can be the second fusion result in the foregoing embodiments, and ξ[(1-t)·I i +t·I j ] can represent the third result obtained by the CLIP model for feature extraction on the linearly weighted I i and I j ; G(I i ) and G(I j ) respectively represent the first extraction result and the second extraction result obtained by processing I i and I j by the FAST-ReID model; F(G(I i )) represents the first result obtained by processing the first extraction result by the mapping network in the intermediate state, and F(G(I j )) represents the second result obtained by processing the second extraction result by the mapping network in the intermediate state; (1-t)·F(G(I i ))+t·F(G(I j )) corresponds to Figure 4The branch includes the FAST-ReID model, the mapping network of the intermediate state, and the feature fusion module, and the branch implements operations of processing the feature extraction result output by the FAST-ReID model through the mapping network of the intermediate state, transforming the feature extraction result output by the FAST-ReID model to the implicit space, and generating a new feature vector in the implicit space.

[0168] In the physical world, the features of pedestrians are in a continuous distribution or an approximate continuous distribution state, but the pixel features in the actually collected images are in a discrete state. In order to enable the pixel features in the images to approximate the real feature distribution of pedestrians in the physical world, a continuous function is used to represent the images in mathematics, but the form of the continuous function may be unknown. Therefore, a neural network is used to approximate the continuous function in the related art, so as to indirectly realize the continuous representation of the pixel features in the images. This method is referred to as implicit neural representation (INR). At present, this technology is widely applied in the fields of image generation, image super-resolution, 3D reconstruction, and the like.

[0169] The mapping network of the intermediate state also provides a method for linearly weighting feature data in the implicit space, so that the first fusion result obtained through the linear weighting can effectively compensate for the discreteness of the pixel features between the first data and the second data. In addition, the INR in the related art usually only maps the feature data to another dimension, but in the embodiment of the present application, through the mapping network of the intermediate state and the mapping network obtained after training, the feature data can be mapped to the implicit space, and new feature data, that is, the first fusion result, can be generated in the implicit space through linear weighting fusion, that is, the linear interpolation method, so that diversified feature processing can be realized in the implicit space.

[0170] Based on the foregoing embodiment, the mapping network of the intermediate state in the feature set expansion method provided by the embodiment of the present application can be obtained through the following steps:

[0171] The third data is obtained from the feature sample; the third data is subjected to feature extraction through the FAST-ReID model to obtain a third extraction result; the third extraction result is processed through the mapping network of the initial state to obtain a fourth result; the third data is subjected to feature extraction through the CLIP model to obtain a fifth result; and the parameters of the mapping model of the initial state are adjusted based on the matching degree between the fourth result and the fifth result to obtain the mapping network of the intermediate state.

[0172] Figure 5 The flowchart for training the mapping network of the initial state provided by the embodiment of the present application is as follows: Figure 5It is shown that, under the condition that the third data is image data, the second image input module 501 can obtain the third data from the feature sample and input the third data to the FAST-ReID model 402, and the feature extraction operation of the FAST-ReID model 402 can obtain the third extraction result.

[0173] Exemplarily, in order to reduce the feature data storage overhead, the third extraction result can be subjected to normalization processing, that is, the third extraction result is input into the first normalization model 502, and the first normalization result 503 is obtained through the normalization operation of the first normalization model 502.

[0174] At the same time, the second image input module 501 can input the third data to the CLIP model 406, and the fifth result can be obtained through the feature extraction of the CLIP model 406, and then the fifth result is input into the second normalization model 504, and the second normalization result 505 can be obtained.

[0175] The mapping network in the initial state 506 can process the first normalization result to obtain the fourth result, and then adjust the parameters of the mapping network in the initial state 506 according to the matching degree between the fourth result and the fifth result, so as to obtain the mapping network in the intermediate state.

[0176] As can be seen from the above, in the embodiment of the application, the matching result between the fourth result output by the mapping network in the initial state and the fifth result output by the CLIP network is obtained, the mapping network in the initial state is trained to obtain the mapping network in the intermediate state, the parameters of the mapping network in the initial state are adjusted, so that the network can map the third extraction result output by the FAST-ReID model to the fifth result output by the CLIP model, so that the network can effectively compensate for the precision defect of the FAST-ReID model in feature extraction; and by processing the features in the implicit space, it also lays a foundation for subsequent training of the mapping network in the intermediate state.

[0177] The mapping network obtained through the training of the above two links can not only realize the mapping of any feature data to the feature extraction result of the CLIP model, that is, the strengthening of the feature information in any feature data; but also can realize the feature strengthening tracking of the fusion result of any number of feature data, that is, the linear interpolation result, in the implicit space.

[0178] Exemplarily, the feature set expansion method provided by the foregoing embodiments can be realized by the mapping network, so as to realize tracking processing of any feature data, any number of feature data, and determine the result of the tracking processing as a new feature template, and then add the new feature template to the feature template library, so as to realize expansion of the feature template library.

[0179] Figure 6 This is a schematic diagram of the structure of the feature template provided in the embodiments of this application, such as... Figure 6 As shown, the feature templates in feature template library 601 can be divided into units based on pedestrians, for example... Figure 6 The first target 6011 to the third target 6013 shown can correspond to the template set of the first pedestrian to the third pedestrian, respectively; wherein, the feature template 602 of the first target can include the first template 6021 to the third template 6023; for example, the attribute list included in the first template 6021 to the third template 6023 can include location information, appearance features, detection block diagrams and others.

[0180] Figure 7 This is a schematic diagram of the structure for obtaining a new feature template through a mapping network, as provided in an embodiment of this application. Figure 7 As shown, the mapping network 701 can perform tracking and fusion processing on the feature data contained in the first template 6021 to the third template 6023 in the feature template 602 of the first target, thereby obtaining the fourth template 7021 to the sixth template 7023 in the first template set 702.

[0181] like Figure 7 As shown, the mapping network 701 performs tracking and fusion processing on the feature data contained in the first template 6021 to the third template 6023 in the feature template 602 of the first target. After obtaining the processing result, the linear interpolation module 703 can perform linear weighted fusion on each feature data in the processing result to obtain the seventh template 7041 to the ninth template 7043 in the second template set 704. For example, the linear interpolation module 703 can perform linear weighted fusion on each feature data in the processing result by using the weight parameters 1-t and t in the above embodiment, i.e., in formula (1), to perform linear weighting on each feature data in the processing result, and determine the result of linear weighting as the seventh template 7041 to the ninth template 7043 in the second template set 704.

[0182] As can be seen from the above, the mapping network in this embodiment can perform tracking and fusion processing on any feature data or any number of feature data, determine the result of the tracking and fusion processing as a new feature template, and then add the new feature template to the feature template library, thereby expanding the feature template library. Thus, this not only provides a method for expanding feature templates in implicit space, but also reduces the dependence on fixed thresholds when matching feature data in related technologies, thereby enabling high-precision and flexible expansion of the feature template library.

[0183] Based on the foregoing embodiments, this application also provides an electronic device. Figure 8A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 8. As shown in FIG. 8, the electronic device 8 can include a processor 801 and a memory 802. The memory 802 stores a computer program which, when executed by the processor 801, can implement the feature set extension method according to any one of the preceding embodiments. Figure 8

[0184] The processor can be at least one of an ASIC, a DSP, a DSPD, a PLD, an FPGA, a CPU, a controller, a microcontroller, a microprocessor. The memory can be a volatile memory (for example, a Random Access Memory, RAM) or a non-volatile memory (for example, a Read-Only Memory, ROM, a flash memory, a Hard Disk Drive, HDD or a Solid State Disk, SSD) or a combination thereof, and provides the processor with instructions and data.

[0185] Based on the foregoing embodiments, the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor of an electronic device, the feature set extension method according to any one of the preceding embodiments can be implemented.

[0186] The above description of the various embodiments tends to emphasize the differences between the various embodiments, and the same or similar parts can be mutually referred to for brevity. The above description will not be repeated here.

[0187] The methods disclosed in the various method embodiments of the present application can be combined arbitrarily without conflict to obtain new method embodiments.

[0188] The features disclosed in the various product embodiments of the present application can be combined arbitrarily without conflict to obtain new product embodiments.

[0189] The features disclosed in the various method or device embodiments of the present application can be combined arbitrarily without conflict to obtain new method or device embodiments.

[0190] ​It should be noted that the computer readable storage medium above can be a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Ferromagnetic Random Access Memory (FRAM), a Flash Memory, a magnetic surface memory, an optical disc, or a Compact Disc Read-Only Memory (CD-ROM) memory, etc. It can also be various electronic devices including one or any combination of the above memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc.

[0191] It should be noted that in this paper, the term "include", "contain" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0192] The above sequence numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0193] Through the above description of the embodiments, those skilled in the art can clearly understand that the above embodiment method can be realized by software plus necessary general hardware nodes, of course, it can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disc, optical disc) and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) execute the methods described in various embodiments of the present application.

[0194] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks. Figure 1 one or more flow or blocks.

[0195] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flow or blocks. Figure 1 one or more flow or blocks.

[0196] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks. Figure 1 Figure 1 one or more flow or blocks.

[0197] The above merely provides the preferred embodiment of the present application, and is not intended to limit the patent scope of the present application, and any equivalent structure or equivalent flow transformation made by using the content of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A feature set extension method, characterized by, The method comprises: obtaining the ith feature data; wherein the ith feature data is associated with feature data in a feature set to the same type of object; the feature data in the feature set comprises a verification feature for identifying the same type of object; i is an integer greater than or equal to 1; the ith feature data comprises a posture feature and / or an appearance feature of a pedestrian obtained by identifying a pedestrian image or video, and the posture feature comprises a walking posture and / or a gesture of the pedestrian; performing tracking fusion processing on at least part of the data in the ith feature data to obtain the jth feature data; wherein the degree of association between the jth feature data and the feature data in the feature set is greater than an association threshold; j is a positive integer different from i; the feature represented by the jth feature data is an associated feature of the feature represented by the feature data in the feature set; adding the jth feature data to the feature set to expand the feature set; wherein the expanded feature set is used for identifying the same type of object.

2. The method of claim 1, wherein, The tracking fusion processing on at least part of the data in the ith feature data to obtain the jth feature data comprises: performing tracking fusion processing on at least part of the data in the ith feature data based on the feature data in the feature set to obtain the jth feature data.

3. The method of claim 1, wherein, The ith feature data at least comprises the mth feature data and the nth feature data, wherein m and n are different positive integers; the tracking fusion processing on at least part of the data in the ith feature data to obtain the jth feature data comprises: performing tracking fusion processing on at least part of the data in the mth feature data to obtain a first feature component; performing tracking fusion processing on at least part of the data in the nth feature data to obtain a second feature component; processing the first feature component and the second feature component to obtain the jth feature data.

4. The method of claim 3, wherein, The processing of the first feature component and the second feature component to obtain the jth feature data comprises: determining a weight parameter; based on the weight parameter, performing weighted fusion on the first feature component and the second feature component to obtain the jth feature data.

5. The method of claim 1, wherein, The tracking fusion processing on at least part of the data in the ith feature data to obtain the jth feature data comprises: performing tracking identification and fusion processing on at least part of the data in the ith feature data through a mapping network to obtain the jth feature data; The mapping network is obtained by training the mapping network based on a feature sample pair through a target re-identification FAST-ReID model and a text-image matching pre-trained CLIP model; wherein, The FAST-ReID model is used for feature extraction on the feature sample, and an extraction result is output; the mapping network of the intermediate state is used for processing the extraction result output by the FAST-ReID model, and a processing result is output; the CLIP model is used for feature extraction on the feature sample, and an extraction result is output; and the processing result output by the mapping network of the intermediate state and the extraction result output by the CLIP model are used to adjust parameters of the mapping network of the intermediate state, so as to obtain the mapping network.

6. The method of claim 5, wherein, The mapping network of the intermediate state is trained based on the feature sample by using the FAST-ReID model and the CLIP model, and the mapping network is obtained, including: First data and second data are obtained from the feature sample; The FAST-ReID model is used for feature extraction on the first data and the second data respectively, and first extraction result and second extraction result are obtained; The mapping network of the intermediate state is used for processing the first extraction result and the second extraction result respectively, and first result and second result are obtained; The CLIP model is used for feature extraction on the first data and the second data, and third result is obtained; Based on the first result, the second result and the third result, the parameters of the mapping network of the intermediate state are adjusted, and the mapping network is obtained.

7. The method of claim 6, wherein, The mapping network of the intermediate state is trained based on the feature sample by using the FAST-ReID model and the CLIP model, and the mapping network is obtained, including: The first result and the second result are weighted and fused to obtain a first fusion result; The mapping network of the intermediate state is used for processing the first fusion result, and a mapping result is obtained; Based on the matching degree between the third result and the mapping result, the parameters of the mapping network of the intermediate state are adjusted, and the mapping network is obtained.

8. The method of claim 6, wherein, The CLIP model is used for feature extraction on the first data and the second data, and third result is obtained, including: The first data and the second data are weighted and fused to obtain a second fusion result; The CLIP model is used for feature extraction on the second fusion result, and the third result is obtained.

9. The method according to any one of claims 6 to 8, characterized in that, The mapping network of the intermediate state is trained in the following way: Third data is obtained from the feature sample; The FAST-ReID model is used for feature extraction on the third data, and third extraction result is obtained; The mapping network of the initial state is used for processing the third extraction result, and fourth result is obtained; The CLIP model is used for feature extraction on the third data, and fifth result is obtained; Based on the matching degree between the fourth result and the fifth result, the parameters of the mapping network of the initial state are adjusted, and the mapping network of the intermediate state is obtained.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program; when the computer program is executed by the processor of the electronic device, the feature set expansion method according to any one of claims 1 to 9 can be implemented.

Images