Data processing method and apparatus, device, and storage medium

Abstract

This application provides a data processing method, apparatus, device, and storage medium to improve the accuracy of data labels during data migration. The method includes: performing feature analysis on a dataset in a first data domain and constructing a feature matrix; strengthening the feature matrix to determine a strengthening matrix; and analyzing the strengthening matrix to determine the labels of the dataset in a second data domain. This method can strengthen sample pairs in the feature matrix, improving sample accuracy and label accuracy.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data processing method and apparatus, an electronic device, and a storage medium. Background Technology

[0002] When data is migrated from one data domain to another and processed in the other data domain, unsupervised learning is typically employed.

[0003] In unsupervised learning, data needs to be processed to determine its labels. This can be done by analyzing the features of the data to determine the corresponding labels, and then training is performed based on these labels. Therefore, the accuracy of the labels affects the accuracy of unsupervised learning.

[0004] Therefore, a technical problem that urgently needs to be solved by those skilled in the art is: how to improve the accuracy of data labels during data migration. Summary of the Invention

[0005] This application provides a data processing method to improve the accuracy of data tags during data migration.

[0006] Accordingly, embodiments of this application also provide a data processing device, an electronic device, and a storage medium to ensure the implementation and application of the above methods.

[0007] To address the aforementioned issues, this application discloses a data processing method, comprising: performing feature analysis on a dataset in a first data domain and constructing a feature matrix; enhancing the feature matrix to determine an enhancement matrix; and analyzing the enhancement matrix to determine the label of the dataset in a second data domain.

[0008] This application also discloses a data processing apparatus, comprising: a matrix determination module for performing feature analysis on a dataset in a first data domain and constructing a feature matrix; an enhancement module for enhancing the feature matrix and determining an enhancement matrix; and a label determination module for analyzing the enhancement matrix and determining the labels of the database in a second data domain.

[0009] This application also discloses an electronic device, including: a processor; and a memory storing executable code thereon, which, when executed, causes the processor to perform one or more methods as described in this application.

[0010] This application also discloses one or more machine-readable media storing executable code thereon, which, when executed, causes a processor to perform one or more methods as described in this application.

[0011] Compared with the prior art, the embodiments of this application have the following advantages:

[0012] In this embodiment of the application, feature analysis is performed on the dataset of the first data domain, and a feature matrix is ​​constructed. Then, the feature matrix is ​​enhanced to determine the enhancement matrix, thereby enhancing the sample pairs in the feature matrix and improving the accuracy of the samples. Then, cluster analysis is performed based on the enhancement matrix to determine the labels of the database in the second data domain, thereby improving the accuracy of the labels. Attached Figure Description

[0013] Figure 1 This is a schematic diagram illustrating an example of image set processing according to an embodiment of this application;

[0014] Figure 2 This is a schematic diagram illustrating a pedestrian re-identification scenario according to an embodiment of this application.

[0015] Figure 3 This is a flowchart illustrating the steps of an embodiment of a data processing method according to this application;

[0016] Figure 4 This is a flowchart illustrating the steps of an embodiment of a preprocessing method for pedestrian recognition according to this application;

[0017] Figure 5 This is a structural block diagram of an embodiment of a data processing apparatus according to this application;

[0018] Figure 6 This is a schematic diagram of the structure of a device provided in an embodiment of this application. Detailed Implementation

[0019] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] This application can be applied to the field of data domain migration. For various types of sample data, unsupervised learning training can be used during the data domain migration process to improve data accuracy by strengthening features, thereby improving training accuracy during the unsupervised learning training process. The data types can include various types, such as images, text, audio, etc. Each type of data has cross-domain processing scenarios. A data domain can be understood as a dataset composed of data with certain characteristics. The data in this data domain have the same characteristics, such as being collected in the same time period, collected by the same type of collection equipment, or collected in the same region.

[0021] Taking cross-domain pedestrian re-identification as an example, migrating data from one data domain to a new data domain requires training the model to adapt to the new domain. In cross-domain pedestrian re-identification scenarios, different data domains can be divided in various ways, including by different times and different geographical locations. For example, video data collected by the same camera in different seasons belongs to different data domains; similarly, video data collected in different cities or other geographical locations can also belong to different data domains; and indoor (e.g., shopping malls, office buildings) and outdoor (e.g., streets) environments can also belong to different data domains. Pedestrian re-identification refers to the ability to identify pedestrians (or users) from image, video, and other data, and to determine whether the pedestrian is the same person in different images or videos.

[0022] Taking e-commerce scenarios as an example, product-related data obtained from other types of websites such as social networking sites, lifestyle sharing sites, and video websites can be considered data from different data domains than product data on e-commerce websites. To illustrate the migration of product-related data from other types of websites to e-commerce websites, the product-related data obtained from social networking sites and other types of websites can be considered the dataset of the first data domain, while the dataset of product data from the e-commerce website can be considered the dataset of the second data domain. The dataset from the first data domain can be migrated to the second data domain, and the corresponding tags for the dataset in the second data domain can be determined so that the dataset can be used in the second data domain based on these tags. For example, when providing outfit recommendations or beauty tutorials on an e-commerce website, links can be made to the corresponding products. Alternatively, the product dataset from the e-commerce website can be considered the dataset of the first data domain, while the dataset from the social networking site can be considered the dataset of the second data domain. By migrating the data from the first data domain to the second data domain, the tags for the second data domain can be determined, and then tag-based processing can be performed, such as associating recommended products on the social networking site with corresponding product objects on the e-commerce website.

[0023] For unsupervised learning in cross-domain scenarios, the quality of the cross-domain data affects the quality of the trained model. Furthermore, in some cross-domain scenarios, the accuracy of the labels after data transfer also affects the usability of the transferred data. This application's embodiments enhance the sample data of the dataset to improve data quality and the accuracy of the identified labels.

[0024] Reference Figure 1 The diagram illustrates an example of image set processing according to an embodiment of this application.

[0025] Step 102: Perform feature analysis on the dataset of the first data domain and construct a feature matrix.

[0026] Depending on the context, datasets can be categorized by data type, such as image datasets (including video datasets), audio datasets, and text datasets. Migrating a dataset from one data domain to another requires unsupervised learning; the original data domain before migration can be called the first data domain, and the data domain to which it is migrated can be called the second data domain.

[0027] In one scenario, users can upload data from multiple data domains to the server, which then stores the data across these different domains. This data can be collected from different times, locations, and data sources. For example, in a pedestrian re-identification scenario, videos taken in different seasons and locations can be used as data for different domains. Similarly, in a text recognition scenario, text data from different data sources such as academic papers, social media, and e-commerce websites can be collected to form data for different domains.

[0028] Correspondingly, the server can provide data migration services, performing data migration and corresponding unsupervised learning, and then deploying the trained model to the cloud. During this data migration and training process, users can also describe the data, and the cloud can perform migrations between different data domains based on different user needs. For example, migrating image data from the data domain corresponding to a live streaming scenario to the data domain corresponding to an e-commerce scenario, or migrating text data from the data domain corresponding to an e-commerce scenario to the data domain corresponding to a social networking scenario.

[0029] In this embodiment, to facilitate user access to the data migration service, a user interface is provided. Users can upload data from various data domains and input their requirements, allowing the server to perform data migration and training based on these requirements. This enables the server to select the desired data domain for migration and training. During this process, the results of each stage are provided for user viewing, allowing for adjustments. Once the model for the desired data domain is obtained, corresponding processing is performed.

[0030] This application embodiment performs feature analysis on a dataset in the first data domain, identifying various features from the dataset and constructing a corresponding feature matrix based on these features. This feature matrix can be a matrix constructed from features, or it can be a matrix constructed based on differences such as distance and similarity between features; this application embodiment does not impose any limitations on this. The features or differences such as distance and similarity that constitute the matrix can be used as elements of the matrix.

[0031] In this embodiment, the step of performing feature analysis on the dataset of the first data domain and constructing a feature matrix includes: performing feature analysis on the dataset of the first data domain using more than one analysis method to extract object features of the target object; and constructing a feature matrix of the dataset based on the features of the target object. For the dataset of the first data domain, multiple analysis methods can be used for feature analysis. The data analysis methods can be the same or different depending on the data type. Specifically, multiple analysis methods can be determined based on the data type, and then feature analysis is performed to extract object features of the target object. The feature matrix of the dataset is then constructed based on the extracted object features. The feature matrix can be composed of features. In other scenarios, the feature matrix can also be constructed based on the differences between features. One way to express the difference between features is by distance. The difference between two features is represented by calculating the distance between them. The larger the feature difference, the larger the distance; the smaller the difference, the smaller the distance. Therefore, in some optional embodiments, the step of constructing the feature matrix of the dataset based on the features of the target object includes: determining the distance between the object features of any two target objects; and constructing the feature matrix of the dataset using the distance.

[0032] For a given dataset, feature analysis can be performed using various methods to extract the features of the identified target object. In a pedestrian re-identification scenario, pedestrians can be identified from image data, and their features can be extracted. These extracted features can be determined based on the analysis method, such as body posture, walking posture, and facial features. Furthermore, based on the features of the identified pedestrians, the distance between the features of any two pedestrians can be calculated, i.e., the differences in features between two pedestrians can be analyzed. A feature matrix is ​​constructed based on these distances, and this feature matrix describes the distribution of pedestrian features in the image dataset.

[0033] This application embodiment performs feature analysis on the dataset using multiple analysis methods to obtain the features of each sample, thereby obtaining the feature matrix of the dataset. Correspondingly, each analysis method can correspond to a feature matrix. Here, a sample can be understood as a target object. For example, in pedestrian re-identification, the identified pedestrians are samples, and the features of the pedestrians can be obtained through feature analysis.

[0034] The analysis methods include at least one of the following: data augmentation, local feature analysis, time-domain feature analysis, and bias removal.

[0035] Data augmentation is a type of data enhancement method. Taking image datasets as an example, it involves randomly rotating, flipping, cropping, randomly adjusting the brightness and contrast of images, and standardizing the data to increase the sample size before feature extraction. Similarly, for text datasets, it can be augmented using methods such as synonym replacement and text pair retrieval, followed by analysis and feature extraction.

[0036] Local feature analysis is a method of analyzing the local features of data, such as the local features of images and audio. It can extract features based on different parts of the data. For example, Beyond Part Models can use local (part) features to describe pedestrian images and extract local (part) features from images. Another example is the extraction of local features based on partially convolutional pooling (PCB-RPP) networks.

[0037] Temporal feature analysis is a method of feature extraction based on the time domain. For example, for image datasets, during training, the use of a stochastic optimizer causes the model parameters to change rapidly. Therefore, models from different training epochs may produce contradictory inference results. By combining models from multiple epochs, more time-stable features can be extracted. Similarly, temporal features can be extracted from audio data.

[0038] In ReID problems, camera bias can significantly impact feature extraction. For example, images of the same person taken by different cameras may differ in pose, lighting conditions, and occlusion, greatly affecting feature extraction. Therefore, bias removal techniques can be employed to eliminate camera bias and improve feature extraction. This can be achieved by assuming that images taken with the same camera follow a specific distribution. The average of all features from that camera can reflect some characteristics of that camera, and subtracting this average feature from the original features can eliminate some of the camera's influence. Furthermore, in audio processing scenarios, some audio data contains noise such as echoes, which can be addressed through echo cancellation and other methods to remove bias.

[0039] The above examples illustrate several analysis methods. In actual processing, other analysis methods can be adopted based on requirements, and this application does not limit this. Various analysis methods can process datasets, analyze the features of target objects in the datasets, extract corresponding object features, and then calculate the distance between different object features to obtain the corresponding feature matrix.

[0040] Taking image dataset processing as an example: For data augmentation, data can be augmented through random flipping, cropping, rotation, and contrast adjustment. Then, feature extraction is performed to determine the object features of the target objects identified in each image. The distance between the object features of two target objects is then calculated to construct the feature matrix of the image dataset. Another example is local feature analysis, which identifies target objects in the images of the image dataset and extracts local features. The distance between the object features of two target objects is then calculated to construct the feature matrix of the image dataset. Yet another example is temporal feature analysis, which performs multiple rounds of analysis on the image data in the image dataset to extract the temporal features of the target objects. The distance between the object features of two target objects is then calculated to construct the feature matrix of the image dataset. Finally, a bias removal method extracts features from the image data in the image dataset, removes camera bias to obtain the corresponding object features, and then calculates the distance between the object features of two target objects to construct the feature matrix of the image dataset.

[0041] Step 104: Strengthen the feature matrix and determine the strengthening matrix.

[0042] After obtaining the feature matrix corresponding to each analysis method, the distances of target objects (samples) in the feature matrix can be analyzed for each feature matrix. Unlike existing methods that discard samples with low confidence, this embodiment can filter and strengthen the elements in the feature matrix. For example, for a feature matrix composed of distances between features, where the elements are distances, distances can be filtered. This process can retain difficult samples and improve the quality of training data. Through strengthening, the differences between samples with large differences can be increased, while the differences between samples with small differences can be decreased. For example, the strengthening step for the target distance includes: strengthening the target distance according to a strengthening coefficient to obtain the strengthened distance, wherein strengthening increases target distances above a first threshold and decreases target distances below a second threshold.

[0043] Hard samples refer to samples that meet certain characteristics, including dissimilar positive samples and similar negative samples. Adding hard samples to the training data can provide stable gradients for model optimization. To distinguish different samples, each sample can be identified by its ID. The distance mentioned above represents the difference between two samples. Hard samples can be pairs of samples with the same ID but large differences (distance), or pairs of samples with different IDs but small differences (distance). In other words, they are pairs of samples identified as having different IDs but with small differences, or pairs of samples identified as having the same ID but with large differences. These types of sample pairs provide more loss during training and can provide stable gradients for the model.

[0044] The process involves setting filtering conditions based on the selection method. Target distances meeting these conditions are selected from the feature matrix, and then these distances are enhanced. Enhancement increases the distances with large differences and decreases the distances with small differences. This enhanced distance is then replaced with the original feature matrix to obtain the enhanced matrix. The enhancement process can be understood as a mapping process, where the selected target distances are enhanced through a mapping function. For example, a mapping function might be y = ax, where a is the mapping coefficient (also called the enhancement coefficient). The mapping coefficient a = 1.2 for target distances above a first threshold and a = 0.8 for target distances below a second threshold. If a target distance x1 = 2 above the first threshold and x2 = 0.4 below the second threshold, then the enhanced distance y1 = 2 * 1.2 = 2.4 is increased, while the enhanced distance y2 = 0.4 * 0.8 = 0.32 is decreased. This achieves distance enhancement based on the mapping coefficient.

[0045] In this embodiment, strengthening the feature matrix and determining the strengthening matrix includes: strengthening target distances in the feature matrix that meet set conditions, and determining the corresponding strengthening matrix. The set conditions can be determined based on a filtering method. For example, when filtering by a threshold range, distances exceeding the threshold range can be considered target distances that meet the set conditions. Similarly, when filtering by a gate function, distances passing through the gate function can be considered target distances that meet the set conditions. The filtering conditions corresponding to various filtering methods, such as threshold ranges and gate functions, can be adjusted based on the specific dataset used, the data domain before and after migration, and the corresponding analysis method. In data domain migration scenarios, the threshold range and gate function will differ when the same dataset is analyzed using different analysis methods. Therefore, they can be adaptively adjusted based on the dataset, data domain, and corresponding analysis method.

[0046] In some optional embodiments, a threshold range for filtering distances can be set, thereby using distances exceeding the threshold range as target distances that meet the set conditions. Since different analysis methods generate different feature matrices, the threshold range can correspond to the analysis method, with each analysis method corresponding to a different threshold range. This allows for distance filtering and enhancement based on the threshold range matching the analysis method. The step of enhancing the target distances in the feature matrix that meet the set conditions and determining the corresponding enhancement matrix includes: obtaining the threshold range corresponding to the feature matrix; using distances in the feature matrix exceeding the threshold range as target distances and enhancing these target distances; and replacing the corresponding target distances in the feature matrix with the enhanced distances to determine the enhancement matrix. Specifically, target distances smaller than the minimum value of the threshold range can be reduced through enhancement, and target distances larger than the maximum value of the threshold range can be increased through enhancement. Corresponding enhancement coefficients can also be set for target distances with different enhancement methods, and the target distances are enhanced based on these coefficients. The first threshold can be the maximum value of the threshold range, and the second threshold can be the minimum value of the threshold range.

[0047] In some alternative embodiments, a gating function can be used to filter and enhance target distances. A gating function can be understood as a function that acts like a gate, allowing data that meets certain conditions to pass through while disallowing data that does not, thus filtering the data. In this embodiment, the gating function can filter distances in the feature matrix, and the distances passing through the gating function are target distances that meet set conditions. The gating function also enhances the target distances that meet the conditions, outputting the enhanced distance. The enhancement coefficient can control the degree of enhancement applied by the gating function. Therefore, enhancing the target distances in the feature matrix that meet the set conditions and determining the corresponding enhancement matrix includes: enhancing the target distances in the feature matrix using a gating function to determine the enhanced distance, where the gating function corresponds to the analysis method; and replacing the corresponding target distances in the feature matrix with the enhanced distances to determine the corresponding enhancement matrix. In this embodiment, the feature matrices extracted by different analysis methods can correspond to different gate functions. That is, the gate function can be dynamically adjusted to adapt to the analysis method. By processing with a gate function adapted to the analysis method, the target distance is strengthened, thereby strengthening samples with high confidence, that is, retaining difficult samples and improving data diversity. Accordingly, the first threshold and the second threshold can be determined based on the gate function.

[0048] In this embodiment, to better meet user needs, the user can set corresponding enhancement coefficients or enhancement requirements. Therefore, an enhancement setting control can be set in the user interface, based on which the corresponding enhancement coefficients or enhancement requirements can be set, facilitating adjustments to the threshold range and gate function. Correspondingly, the user interface may also include an enhancement display interface, where the server can display the enhancement matrix sent to the user terminal, allowing the user to view whether the enhancement matrix meets the requirements and whether further adjustments to the enhancement coefficients are needed.

[0049] Step 106: Analyze the dataset based on the reinforcement matrix to determine the label in the second data domain.

[0050] The reinforcement matrix can be analyzed to obtain its label in the second data domain. One analysis method is cluster analysis, which can determine multiple clusters. Each cluster can include multiple features of the same type, and the sample data corresponding to each feature can be grouped into clusters. The samples (target objects) in each cluster constitute a class, and corresponding labels can be assigned to distinguish them from supervised learning. These labels can also be called pseudo-labels. If multiple analysis methods are used, each analysis method can correspond to a reinforcement matrix. The reinforcement matrices can be fused first, and then clustered to obtain the labels corresponding to the dataset. Alternatively, similarity analysis can be used, such as calculating the similarity between features and the features corresponding to the labels, to determine the corresponding labels.

[0051] In one optional embodiment, the step of performing cluster analysis based on the reinforcement matrix to determine the label of the dataset in the second dataset includes: fusing the reinforcement matrices corresponding to the more than one analysis method to obtain a fusion matrix; and performing cluster analysis on the fusion matrix to determine the label of the dataset in the second dataset.

[0052] After obtaining the enhancement matrices corresponding to various analysis methods, these matrices can be fused to obtain a more accurate feature matrix, known as a fusion matrix. This fusion matrix can then be used for clustering. Image clustering is performed based on the features in the fusion matrix, resulting in multiple clusters. Each cluster can include multiple features of the same type, and the samples corresponding to each feature can be grouped into clusters. The samples (target objects) within each cluster constitute a class, and corresponding labels can be assigned; these labels are called pseudo-labels.

[0053] In this embodiment, the enhanced features in the feature matrices generated by different analysis methods may differ. Each analysis method can be considered an expert, and the feature matrix can be seen as the expert's opinion. By fusing the opinions of multiple experts, a more accurate feature matrix can be obtained. The distances of the feature matrices can be filtered and enhanced using gate functions and threshold ranges. The distances of sample pairs with distances greater than a certain high threshold (first threshold) are enhanced by increasing the distance, while those with distances below a certain low threshold (second threshold) are enhanced by decreasing the distance. These sample pairs can be considered high-confidence sample pairs. Since the feature matrices corresponding to different analysis methods have different characteristics, the values ​​of the two thresholds—that is, the gate function and threshold range—need to be dynamically adjusted according to the distance distribution of each feature matrix. In this way, positive and negative sample pairs considered reliable by different experts are obtained, resulting in a higher-quality clustering result overall.

[0054] like Figure 2 Taking a pedestrian re-identification scenario as an example, assume there are 4 analysis methods (experts) E1, E2, E3, and E4.

[0055] The image dataset is analyzed using various methods to determine the feature matrices of pedestrian characteristics, resulting in feature matrices d1, d2, d3, and d4. Each feature matrix is ​​then enhanced using corresponding gate functions G1, G2, G3, and G4. This allows four experts to extract four feature matrices, which are then enhanced using adaptive gate functions on pairs of samples with high confidence. Finally, the enhanced matrices are fused to obtain a fused feature matrix L.

[0056] After calculating the labels of the dataset in the second data domain, the embodiments of this application can perform related processing in the second data domain based on the labels, such as making recommendations based on the labels, associating data corresponding to the same labels based on the labels, and training models based on the labels.

[0057] In some embodiments, the dataset is trained in a second data domain based on the dataset and labels. After obtaining the labels, the labels can be added to the dataset; the labeled dataset is then used as training data to train the dataset in the second data domain. This utilizes training data with higher accuracy and confidence, effectively improving model performance.

[0058] The above steps can be executed iteratively. During the iteration process, the gate function and threshold range can be adjusted based on the loss function of the model training process to adjust the reinforcement of elements in the matrix. Then, fusion and clustering are performed to adjust the labels. Training is then carried out using the adjusted labels. Through continuous iteration, unsupervised learning of cross-domain data is achieved.

[0059] In this embodiment, the model training results, loss function, etc., can also be fed back to the user terminal and displayed in the user interface so that the user can adjust the model training results, loss function, etc., to obtain a model that better meets the user's needs.

[0060] Taking a Software-as-a-Service (SaaS) scenario as an example, cross-domain model training can be structured as a service, and a service interface can be developed for users. Users can then call this service through the service interface on their client side, selecting cross-domain migrated data. On the server side, the service executes the aforementioned process for model training. During this process, reinforcement matrices, labels, the trained model, and loss functions can be fed back to the user based on their needs, allowing for adjustments and feedback. This process is iterated to obtain the desired model and return it to the client. Alternatively, the model can be deployed on the server side, providing a model interface for users to call and perform necessary processing. For example, in a pedestrian re-identification scenario, the trained pedestrian re-identification model can be deployed on the server side, allowing users to call the model through the interface to identify pedestrians and other target objects from relevant images and videos.

[0061] In summary, feature analysis is performed on the dataset in the first data domain, and a feature matrix is ​​constructed. Then, the feature matrix is ​​enhanced to determine the enhancement matrix, thereby enhancing the sample pairs in the feature matrix and improving the accuracy of the samples. Finally, cluster analysis is performed based on the enhancement matrix to determine the labels of the database in the second data domain, thereby improving the accuracy of the labels.

[0062] Based on the dataset and labels, the dataset is trained in the second data domain to improve the accuracy of the training.

[0063] Based on the above embodiments, this application also provides a data processing method that can incorporate more dimensional information when calculating the feature matrix of a dataset, describe the relationship between images from multiple perspectives, and then perform effective fusion, so that the fused feature matrix has higher accuracy.

[0064] Reference Figure 3 The diagram shows a flowchart of one embodiment of a data processing method according to this application.

[0065] Step 302: For the dataset of the first data domain, perform feature analysis using more than one analysis method to extract the object features of the target object.

[0066] The analysis methods include at least one of the following: data augmentation, local feature analysis, time-domain feature analysis, and bias removal.

[0067] Step 304: Determine the distance between the object features of any two target objects, and construct the feature matrix of the dataset using the distance.

[0068] Step 306: Enhance the target distances in the feature matrix that meet the set conditions, and determine the corresponding enhancement matrix.

[0069] In one example, a threshold range corresponding to the feature matrix can be obtained; the distances in the feature matrix that exceed the threshold range are taken as target distances, and the target distances are enhanced; the enhanced distances are used to replace the corresponding target distances in the feature matrix to determine the enhancement matrix.

[0070] In another example, a gate function is used to enhance the target distance in the feature matrix to determine the enhanced distance. The gate function corresponds to the analysis method. The enhanced distance is used to replace the corresponding target distance in the feature matrix to determine the corresponding enhancement matrix.

[0071] The target distance enhancement step includes: enhancing the target distance according to the enhancement coefficient to obtain the enhanced distance, wherein the target distance above the first threshold is increased by enhancement, and the target distance below the second threshold is decreased by enhancement.

[0072] Step 308: Fuse the enhancement matrices corresponding to the more than one analysis method to obtain a fusion matrix.

[0073] Step 310: Cluster the fusion matrix to determine the labels corresponding to the dataset.

[0074] Step 312: Based on the dataset and labels, train the dataset in the second data domain.

[0075] The labels are added to the target objects in the dataset; the labeled dataset is used as training data to train the model in the second data domain. This process can be performed iteratively, adjusting the enhancement coefficients, labels, and model parameters based on the loss function during execution to train a model that meets the user's needs.

[0076] Based on the above embodiments, this application also provides a preprocessing method for pedestrian recognition, which can incorporate more dimensional information when calculating the feature matrix of an image dataset, describe the relationship between images from multiple perspectives, and then perform effective fusion, so that the fused feature matrix has higher accuracy.

[0077] Reference Figure 4 The diagram shows a flowchart of an embodiment of a preprocessing method for pedestrian recognition according to this application.

[0078] Step 402: For the image dataset in the first data domain, perform feature analysis on the image dataset according to more than one analysis method, and construct the feature matrix of the image dataset. The distance in the feature matrix is ​​determined based on the corresponding features of the identified pedestrian.

[0079] This application's embodiments can be applied to cross-domain pedestrian re-identification scenarios. "Cross-domain" refers to migrating data from one data domain to a new data domain. In cross-domain pedestrian re-identification scenarios, data is migrated from one data domain to a new one, and the corresponding label for the data in the new data domain needs to be determined. The model can then be trained based on the label to adapt to the new domain, or other necessary processing can be performed. In cross-domain pedestrian re-identification scenarios, different data domains can be divided in various ways, including different times, different geographical locations, etc. For example, video data collected by the same camera in different seasons belongs to different data domains; similarly, video data collected in different cities or other geographical locations can also belong to different data domains; furthermore, indoor (e.g., shopping malls, office buildings) and outdoor (e.g., roads) environments can also belong to different data domains.

[0080] The first and second data domains can contain data captured in different seasons and locations. For example, the first data domain could be an indoor data domain, consisting of images and videos captured indoors, while the second data domain could be an outdoor data domain, consisting of images and videos captured outdoors. Similarly, the first data domain could be a winter data domain, consisting of images and videos captured from December to February, while the second data domain could be a summer data domain, consisting of images and videos captured from June to August. During the migration from the first data domain to the second data domain, the image dataset in the first data domain can be subjected to feature analysis using more than one analysis method to extract pedestrian features; the distance between the features of two pedestrians can be determined, and a feature matrix of the surveillance video set can be constructed.

[0081] Step 404: Enhance the target distances in the feature matrix that meet the set conditions, and determine the corresponding enhancement matrix.

[0082] In one example, the threshold range corresponding to the feature matrix can be obtained; the distance in the feature matrix that exceeds the threshold range is taken as the target distance, and the target distance is enhanced; the enhancement matrix is ​​determined.

[0083] In another example, a gate function is used to enhance the target distance in the feature matrix to determine the corresponding enhancement matrix. The gate function corresponds to the analysis method.

[0084] The target distance enhancement step includes: enhancing the target distance according to the enhancement coefficient to obtain the enhanced distance, wherein the target distance above the first threshold is increased by enhancement, and the target distance below the second threshold is decreased by enhancement.

[0085] Step 406: Fuse the enhancement matrices corresponding to the more than one analysis method to obtain a fusion matrix.

[0086] Step 408: Cluster the fusion matrix to determine the labels corresponding to the image dataset.

[0087] Step 410: Based on the image dataset and labels, train the image dataset in the second data domain. Iterate the above process until the trained person re-identification model meets the requirements. The transferred person re-identification model is trained using a labeled image dataset. No additional training parameters are needed; it can be independently incorporated into other domain transfer methods, further improving the effectiveness of domain transfer person re-identification.

[0088] This application embodiment employs a multi-expert system, which fuses the outputs of multiple similar or different models to improve model performance. Each analysis method can be considered an expert, and the feature matrix can be seen as the expert's opinion. The enhanced features in the feature matrices generated by different analysis methods may differ. Therefore, fusing the opinions of multiple experts yields a more accurate feature matrix. The distances of the feature matrices can be filtered and enhanced using gate functions and threshold ranges. The distances of sample pairs with distances greater than a certain high threshold (first threshold) are enhanced by increasing the distance, while the distances of sample pairs with distances below a certain low threshold (second threshold) are enhanced by decreasing the distance. These sample pairs are considered high-confidence sample pairs. Since the feature matrices corresponding to different analysis methods have different characteristics, the values ​​of the two thresholds need to be dynamically adjusted according to the distance distribution of each feature matrix. This method yields positive and negative sample pairs considered reliable by different experts, resulting in a higher-quality clustering result overall.

[0089] In this application embodiment, more dimensional information is incorporated when calculating the target domain feature matrix (distmat), describing the relationship between target domain images from multiple perspectives, and then effectively fused, so that the fused feature matrix has higher accuracy, thereby obtaining high-quality pseudo-labels to guide network training through clustering.

[0090] By introducing a multi-expert system and adaptive gate functions to strengthen the high-confidence portions of the feature matrix generated by each expert, and by not discarding samples with low confidence, the accuracy of the samples is improved.

[0091] The above methods can be used to train corresponding processing models and processors, and can be added as plugins to other pedestrian re-identification model processing systems to improve the accuracy of the system.

[0092] The above example uses a cross-domain pedestrian re-identification scenario. In actual processing, the embodiments of this application can be applied to the cross-domain model training process of various types of data such as images, text, and audio. For example, e-commerce data may include multiple types of data such as text, images, and audio / video. Data from different e-commerce websites can be regarded as data from different data domains. For example, data from other types of websites such as social networking sites and lifestyle sharing sites can be migrated to e-commerce websites, and data from e-commerce websites can be migrated to other types of websites such as social networking sites and lifestyle sharing sites. The required data can be collected to form a corresponding dataset. Then, feature analysis, reinforcement, and analysis are performed according to the above process to determine the labels of target objects such as product objects. Then, training and recommendation are performed by combining the dataset and labels, and the above process is iterated to realize the data migration of different types of websites, as well as the processing based on the migration, such as recommendation, model training, and recognition.

[0093] For example, in logistics scenarios, data from different logistics service providers, e-commerce websites, traffic data, and meteorological data can be considered data from different data domains. Accordingly, during data migration, feature analysis, reinforcement, and analysis can be used to determine labels for target objects such as logistics objects and roads. Then, combining this dataset and labels, recommendations and model training can be performed, and the above process can be iterated to achieve cross-domain logistics data processing. For instance, by migrating meteorological data to the logistics field, corresponding meteorological labels can be determined, thereby identifying labels for extreme weather, delayed weather, and ordinary weather affecting logistics transportation, which can then be applied to the analysis of the logistics transportation process. Similarly, traffic data can be used to determine congestion labels for the analysis of the logistics transportation process and tool selection.

[0094] For example, in the financial field, data from different financial platforms, banks, and e-commerce websites can be considered as data from different data domains. During data migration, feature analysis, reinforcement, and analysis can be used to determine the labels of target objects. Then, combining this dataset and labels, recommendation, model training, recognition, and classification can be performed, iterating through these processes to achieve cross-domain financial data processing. For instance, combining sales data from e-commerce websites to determine labels such as promotional and regular sales can be applied to scenarios like temporary adjustments to user credit limits.

[0095] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of this application.

[0096] Based on the above embodiments, this embodiment also provides a data processing device that is applied in electronic devices such as terminal devices and servers.

[0097] Reference Figure 5 The diagram shows a structural block diagram of an embodiment of a data processing apparatus according to this application, which may specifically include the following modules:

[0098] The matrix determination module 502 is used to perform feature analysis on the dataset of the first data domain and construct a feature matrix.

[0099] The enhancement module 504 is used to enhance the feature matrix and determine the enhancement matrix.

[0100] The tag determination module 506 is used to analyze the reinforcement matrix and determine the tags of the database in the second data domain.

[0101] It also includes a transfer learning module, used to train the dataset in a second data domain based on the dataset and labels.

[0102] In summary, feature analysis is performed on the dataset in the first data domain to construct a feature matrix. This feature matrix is ​​then enhanced to determine an enhancement matrix, thereby strengthening the sample pairs within the feature matrix and improving sample accuracy. Cluster analysis is then performed based on the enhancement matrix to determine the labels of the database in the second data domain, thus improving label accuracy. Furthermore, the dataset can be trained in the second data domain using the dataset and labels, improving training accuracy.

[0103] The matrix determination module 502 is used to perform feature analysis on the dataset of the first data domain using more than one analysis method to extract the object features of the target object; and to construct the feature matrix of the dataset based on the object features of the target object.

[0104] The matrix determination module 502 is used to determine the distance between the object features of any two target objects; and to construct the feature matrix of the dataset using the distance.

[0105] The enhancement module 504 is used to enhance the target distances in the feature matrix that meet the set conditions, and determine the corresponding enhancement matrix.

[0106] In one example, the enhancement module 504 is used to obtain the threshold range corresponding to the feature matrix; take the distance in the feature matrix that exceeds the threshold range as the target distance, and enhance the target distance; replace the corresponding target distance in the feature matrix with the enhanced distance to determine the enhancement matrix.

[0107] In another example, the enhancement module 504 is used to enhance the target distance in the feature matrix using a gate function to determine the enhanced distance, wherein the gate function corresponds to the analysis method; and the enhanced distance is used to replace the corresponding target distance in the feature matrix to determine the corresponding enhancement matrix.

[0108] The enhancement module 504 is used to enhance the target distance according to an enhancement coefficient to obtain an enhanced distance. Specifically, enhancement increases the target distance above a first threshold and decreases the target distance below a second threshold. The enhancement module 504 is also used to send the enhancement matrix to the user terminal so that the user terminal can adjust the enhancement coefficient according to the enhancement matrix.

[0109] Optionally, the analysis method includes at least one of the following: data augmentation method, local feature analysis method, time-domain feature analysis method, and bias removal method.

[0110] The label determination module 506 is used to perform cluster analysis based on the reinforcement matrix to determine the labels of the database in the second data domain.

[0111] The label determination module 506 is used to fuse the reinforcement matrices corresponding to more than one analysis method to obtain a fusion matrix; and to cluster the fusion matrix to determine the labels corresponding to the dataset.

[0112] The target objects include pedestrians.

[0113] This application embodiment employs a multi-expert system, which fuses the outputs of multiple similar or different models to improve model performance. Each analysis method can be considered an expert, and the feature matrix can be seen as the expert's opinion. The enhanced features in the feature matrices generated by different analysis methods may differ. Therefore, fusing the opinions of multiple experts yields a more accurate feature matrix. The distances of the feature matrices can be filtered and enhanced using gate functions and threshold ranges. The distances of sample pairs with a distance greater than a certain high threshold (first threshold) are enhanced by increasing the distance, while those with a distance below a certain low threshold (second threshold) are enhanced by decreasing the distance. These sample pairs are considered high-confidence sample pairs. Since the feature matrices corresponding to different analysis methods have different characteristics, the values ​​of the two thresholds need to be dynamically adjusted according to the distance distribution of each feature matrix. This method yields positive and negative sample pairs considered reliable by different experts, resulting in a higher-quality clustering result overall.

[0114] In this application embodiment, more dimensional information is incorporated when calculating the target domain feature matrix (distmat), describing the relationship between target domain images from multiple perspectives, and then effectively fused, so that the fused feature matrix has higher accuracy, thereby obtaining high-quality pseudo-labels to guide network training through clustering.

[0115] By introducing a multi-expert system and adaptive gate functions to strengthen the high-confidence portions of the feature matrix generated by each expert, and by not discarding samples with low confidence, the accuracy of the samples is improved.

[0116] The above methods can be used to train corresponding processing models and processors, and can be added as plugins to other pedestrian re-identification model processing systems to improve the accuracy of the system.

[0117] This application also provides a non-volatile readable storage medium storing one or more modules (programs). When these modules are applied to a device, they enable the device to execute the instructions for the method steps in this application.

[0118] This application provides one or more machine-readable media storing instructions that, when executed by one or more processors, cause an electronic device to perform one or more of the methods described in the above embodiments. In this application, the electronic device includes various types of devices such as terminal devices and servers (clusters).

[0119] The embodiments of this disclosure can be implemented as an apparatus configured as desired using any suitable hardware, firmware, software, or any combination thereof, including electronic devices such as terminal devices, servers (clusters), etc. Figure 6 An exemplary apparatus 600 is schematically shown that can be used to implement the various embodiments described in this application.

[0120] In one embodiment, Figure 6 An exemplary device 600 is shown, which includes one or more processors 602, a control module (chipset) 604 coupled to at least one of the processors 602, a memory 606 coupled to the control module 604, a non-volatile memory (NVM) / storage device 608 coupled to the control module 604, one or more input / output devices 610 coupled to the control module 604, and a network interface 612 coupled to the control module 604.

[0121] Processor 602 may include one or more single-core or multi-core processors, and processor 602 may include any combination of general-purpose processors or special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In some embodiments, device 600 can serve as a terminal device, server (cluster), or other device as described in the embodiments of this application.

[0122] In some embodiments, the apparatus 600 may include one or more computer-readable media (e.g., memory 606 or NVM / storage device 608) having instructions 614 and one or more processors 602 that are combined with the one or more computer-readable media and configured to execute the instructions 614 to implement the module and thus perform the actions described in this disclosure.

[0123] In one embodiment, the control module 604 may include any suitable interface controller to provide any suitable interface to at least one of the processors 602 and / or any suitable device or component communicating with the control module 604.

[0124] The control module 604 may include a memory controller module to provide an interface to the memory 606. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0125] Memory 606 may be used, for example, to load and store data and / or instructions 614 for device 600. In one embodiment, memory 606 may include any suitable volatile memory, such as suitable DRAM. In some embodiments, memory 606 may include double data rate type quad synchronous dynamic random access memory (DDR4 SDRAM).

[0126] In one embodiment, the control module 604 may include one or more input / output controllers to provide an interface to the NVM / storage device 608 and (one or more) input / output devices 610.

[0127] For example, NVM / storage device 608 may be used to store data and / or instructions 614. NVM / storage device 608 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable (one or more) non-volatile storage devices (e.g., one or more hard disk drives (HDDs), one or more optical disc drives (CDs), and / or one or more digital universal optical disc (DVD) drives).

[0128] NVM / storage device 608 may include storage resources that are physically part of a device on which device 600 is mounted, or that are accessible to the device but do not necessarily have to be part of the device. For example, NVM / storage device 608 may be accessed via a network through one or more input / output devices 610.

[0129] One or more input / output devices 610 may provide an interface for device 600 to communicate with any other suitable device. Input / output devices 610 may include communication components, audio components, sensor components, etc. A network interface 612 may provide an interface for device 600 to communicate via one or more networks. Device 600 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, such as accessing wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G, 5G, etc., or combinations thereof.

[0130] In one embodiment, at least one of the processors 602 may be logically packaged with one or more controllers (e.g., memory controller modules) of the control module 604. In one embodiment, at least one of the processors 602 may be logically packaged with one or more controllers of the control module 604 to form a system-in-package (SiP). In one embodiment, at least one of the processors 602 may be integrated with the logic of one or more controllers of the control module 604 on the same die. In one embodiment, at least one of the processors 602 may be integrated with the logic of one or more controllers of the control module 604 on the same die to form a system-on-a-chip (SoC).

[0131] In various embodiments, device 600 may be, but is not limited to, a server, desktop computing device, or mobile computing device (e.g., laptop, handheld computing device, tablet, netbook, etc.). In various embodiments, device 600 may have more or fewer components and / or different architectures. For example, in some embodiments, device 600 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touchscreen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0132] The detection device can use a main control chip as a processor or control module, and sensor data, position information, etc. can be stored in a memory or NVM / storage device. The sensor group can be used as an input / output device, and the communication interface can include a network interface.

[0133] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0134] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0135] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0136] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0137] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0138] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0139] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0140] The above provides a detailed description of a data processing method and apparatus, an electronic device, and a storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A data processing method, characterized in that, The method includes: For the dataset in the first data domain, feature analysis is performed using more than one analysis method to extract the object features of the target object. The data types in the dataset include at least one of the following: image, text, and audio. Determine the distance between the object features of any two target objects; The feature matrix of the dataset is constructed using the distance. The enhanced distances of targets in the feature matrix that meet the set conditions are enhanced based on the enhancement coefficients, and the enhanced distances are obtained. The target distances that meet the set conditions are distances that exceed a threshold range or distances that pass through a gate function. The corresponding target distance in the feature matrix is ​​replaced with the enhanced distance to determine the corresponding enhancement matrix; Based on the reinforcement matrix, the labels of the dataset in the second data domain are determined.

2. The method according to claim 1, characterized in that, Also includes: Obtain the threshold range corresponding to the feature matrix; Distances in the feature matrix that exceed the threshold range are determined as target distances that meet the set conditions; Specifically, the distance to a target above a first threshold is increased by strengthening the method, and the distance to a target below a second threshold is decreased by strengthening the method. The first threshold is the maximum value of the threshold range, and the second threshold is the minimum value of the threshold range.

3. The method according to claim 1, characterized in that, The enhancement of target distances in the feature matrix that meet set conditions based on enhancement coefficients to obtain enhanced distances includes: The target distance in the feature matrix is ​​enhanced using an enhancement coefficient and a gate function to determine the enhanced distance. The gate function corresponds to the analysis method, and the enhancement coefficient is used to control the degree of enhancement of the target distance by the gate function.

4. The method according to claim 1, characterized in that, The elements in the feature matrix are enhanced according to enhancement coefficients, and the method further includes: The enhancement matrix is ​​sent to the user terminal so that the user terminal can adjust the enhancement coefficient according to the enhancement matrix.

5. The method according to claim 1, characterized in that, The analysis based on the reinforcement matrix to determine the labels of the dataset in the second data domain includes: Cluster analysis is performed based on the reinforcement matrix to determine the label of the dataset in the second data domain.

6. The method according to claim 5, characterized in that, The step of performing cluster analysis based on the reinforcement matrix to determine the labels of the database in the second data domain includes: The enhancement matrices corresponding to more than one analysis method are fused to obtain a fusion matrix; Cluster the fusion matrix to determine the label of the dataset in the second data domain.

7. The method according to claim 1, characterized in that, Also includes: Based on the dataset and labels, the dataset is trained in the second data domain.

8. The method according to claim 3 or 4, characterized in that, The analysis methods include at least one of the following: data augmentation, local feature analysis, time-domain feature analysis, and bias removal.

9. The method according to any one of claims 1-7, characterized in that, The target objects include pedestrians.

10. A data processing apparatus, characterized in that, The device includes: The matrix determination module is used to perform feature analysis on a dataset in the first data domain using more than one analysis method, extract object features of target objects, and the data type in the dataset includes at least one of the following: image, text, and audio; determine the distance between the object features of any two target objects; and construct the feature matrix of the dataset using the distance. The enhancement module is used to enhance the target distances in the feature matrix that meet the set conditions based on the enhancement coefficient, so as to obtain the enhanced distances. The target distances that meet the set conditions are distances that exceed a threshold range or distances that pass through a gate function. The enhanced distances are used to replace the corresponding target distances in the feature matrix to determine the corresponding enhancement matrix. The label determination module is used to analyze the reinforcement matrix and determine the labels of the database in the second data domain.

11. An electronic device, characterized in that, include: processor; and A memory having executable code stored thereon, which, when executed, causes the processor to perform the method as described in any one of claims 1-9.

12. One or more machine-readable media having executable code stored thereon, which, when executed, causes a processor to perform the method as described in any one of claims 1-9.

Images