Labeled data expansion method, platform and storage medium for retail scenarios
By building globally shared conditional label distribution information and federated learning technology in retail scenarios, cross-store data collaboration is achieved while protecting data privacy, generating high-quality, highly diverse, and business-adaptable synthetic labeled data, solving the problems of insufficient data quality and diversity caused by data silos in retail scenarios.
Patent Information
- Application Number
- CN202510798636.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-16
- Publication Date
- 2025-09-05
- Estimated Expiration
- 2045-06-16
AI Technical Summary
In retail scenarios, due to the data silos among multiple stores, the synthetic data generated by the labeled data expansion method has low quality, insufficient diversity, and poor business adaptability, which cannot meet the needs of high-quality model training.
By extracting local conditional labels on each client and uploading them to the server, the server performs cross-node aggregation processing to generate globally shared conditional label distribution information. The client uses local annotated data, conditional labels and distribution information to train the generator and discriminator. The server processes the model parameters based on the weighted summation of data quantity and quality to generate synthetic data that is both globally representative and regionally adaptable.
Without transmitting the original data, high-quality, highly diverse, and business-adaptable synthetic labeled data is generated, which improves the model training effect and business applicability, and solves the problems of data dispersion and sharing limitations caused by data silos.
Smart Images

Figure CN120316273B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a method, platform, and storage medium for expanding labeled data for use in retail scenarios. Background Art
[0002] In the digital transformation of the retail industry, data-driven precision marketing and supply chain optimization have become core means of enhancing corporate competitiveness. Artificial Intelligence (AI) technology is being applied to AI products in the intelligent retail (IR) industry, focusing on data analysis of individual consumer behavior. Based on understanding consumer habits, it predicts product consumption trends. This consumption trend guides product production and manufacturing, providing consumers with diversified and personalized products and services, and achieving precision marketing. At the same time, it places greater emphasis on the value of labeled data, accelerating the integration of technology across all aspects of the retail industry chain, and improving operational efficiency and user experience in the retail industry. For example, patent application publication number CN113392295A, entitled "Data Labeling Method, Platform, Electronic Device, and Computer Storage Medium," describes a method that obtains labeling type information and business scenario information for a data labeling task; the labeling object of the data labeling task includes at least one data sample to be labeled; determines a data labeling tool corresponding to the data sample to be labeled based on the labeling type information and business scenario information; receives labeling operation information from a data labeler using the data labeling tool on the labeled data sample, and obtains a data labeling result for the data sample to be labeled based on the labeling operation information. For example, the patent application with publication number CN116049351A and invention name is A data labeling method, device, system and storage medium. The method first imports original training data and labeled data, then constructs an original labeling model, trains the original labeling model based on the original training data and labeled data to obtain a first labeling model, then imports unlabeled training data, predicts the unlabeled training data based on the first labeling model to obtain predicted data, and finally analyzes the first labeling model based on the unlabeled training data and the predicted data to obtain a second labeling model.
[0003] However, data annotation and expansion in retail scenarios face a fundamental challenge: the problem of multi-store data silos. Due to privacy regulations and internal business competition, different stores or regional platforms of chain retail companies often independently store key information such as customer transaction data and behavior logs, forming physically isolated data silos. For example, a supermarket chain's East China and South China stores may each hold high-value data such as local users' purchasing preferences and promotion sensitivity. However, due to data sharing restrictions, this information cannot be integrated and analyzed at the group level.
[0004] This data silo phenomenon significantly hinders the expansion of labeled data, which in turn affects the effectiveness of model training and its application value. This is specifically manifested in the following three aspects:
[0005] (1) Insufficient data volume and low quality
[0006] The amount of data from a single store or region is often insufficient to meet the requirements for high-quality model training. In retail scenarios, with a wide variety of product categories and complex user behaviors, a single store may only have a small number of labeled samples. This is especially true for niche products (such as organic foods and imported goods) or specific scenarios (such as Member Day promotions), where data scarcity is even more pronounced. Traditional data augmentation methods (such as random sampling and simple transformations) are unable to effectively utilize data dispersed across multiple stores. The resulting synthetic data often lacks broad representativeness and struggles to capture real business patterns. This results in low-quality synthetic data that cannot support high-quality model training.
[0007] (2) Insufficient data diversity
[0008] The value of retail data lies not only in its scale but also in its diversity. Retail data from different regions often exhibit significant variations, such as in user purchasing power and seasonality of product demand. However, traditional data augmentation methods lack cross-store data collaboration mechanisms and are unable to integrate the advantages of multiple data sources. Consequently, the generated synthetic data lacks diversity and struggles to cover complex retail scenarios.
[0009] (3) Dynamic data synchronization lag
[0010] Retail data is highly time-sensitive. Holiday promotions and emergencies, for example, can rapidly alter user behavior patterns. In a data silo architecture, dynamic changes in regional data are difficult to synchronize with the global perspective, causing synthetic annotations generated based on historical data to quickly become outdated. Traditional methods, lacking a dynamic data synchronization mechanism, are unable to promptly reflect the latest business changes.
[0011] In summary, methods for augmenting labeled data in retail scenarios are often limited by data fragmentation and sharing caused by data silos across multiple stores. This results in low-quality, insufficiently diverse, and poorly adaptable synthetic data. Therefore, a method for augmenting labeled data that can break down data silos and enable cross-domain data collaboration is urgently needed to support high-quality model training in retail scenarios. Summary of the Invention
[0012] The embodiments of the present disclosure provide a method, platform, and storage medium for expanding labeled data in retail scenarios, so as to at least solve the technical problems in the prior art of expanding labeled data in retail scenarios, namely, data dispersion and sharing restrictions caused by multi-store data silos, resulting in low quality, insufficient diversity, and poor business adaptability of the generated synthetic data.
[0013] According to one aspect of an embodiment of the present disclosure, a method for expanding labeled data for use in a retail scenario is provided, which is applicable to a data labeling platform. The data labeling platform includes a server and multiple clients. The method includes: each client generates a local conditional label based on local labeled data and uploads the local conditional label to the server. The server aggregates the local conditional labels of all clients to obtain globally shared conditional label distribution information and distributes it back to each client. Each client trains a local generator and a local discriminator using the local labeled data, the local conditional label, and the conditional label distribution information, and uploads the parameters of the trained local generator and local discriminator to the server. The server performs weighted summation on the parameters uploaded by all clients based on the weights of each client, determines the parameters of a global generator and a global discriminator, and distributes them back to each client. The weights are related to the data volume and data quality of the local labeled data of the client. Each client updates its local generator based on the parameters of the global generator, and uses the updated local generator in combination with the local conditional label and the conditional label distribution information to generate synthetic data of the local labeled data.
[0014] According to another aspect of an embodiment of the present disclosure, a storage medium is further provided, the storage medium including a stored program, wherein when the program is run, a processor executes any one of the above methods.
[0015] According to another aspect of an embodiment of the present disclosure, a labeling data expansion platform for use in a retail scenario is also provided, comprising: a server and multiple clients; each client is used to generate a local conditional label based on local labeling data, and upload the local conditional label to the server, the server aggregates the local conditional labels of all clients, obtains globally shared conditional label distribution information, and distributes it back to each client; each client is also used to train a local generator and a local discriminator using the local labeling data, the local conditional label, and the conditional label distribution information, and uploads the parameters of the trained local generator and local discriminator to the server; the server is used to perform weighted summation processing on the parameters uploaded by all clients according to the weights of each client, determine the parameters of the global generator and the global discriminator, and distribute them back to each client; wherein the weight is related to the data volume and data quality of the local labeling data of the client; and each client is also used to update the local generator according to the parameters of the global generator, and use the updated local generator in combination with the local conditional label and the conditional label distribution information to generate synthetic data of the local labeling data.
[0016] This application first extracts local conditional labels based on local annotated data through each client, and uploads the local conditional labels to the server. The server generates globally shared conditional label distribution information through cross-node aggregation processing and distributes it back to each client, thereby achieving the refinement of scattered local knowledge (local conditional labels) into sharable global cognition (conditional label distribution information) without transmitting the original data, thereby avoiding privacy and competition restrictions. This application achieves the refinement of scattered local knowledge (local conditional labels) into sharable global cognition (conditional label distribution information) without transmitting the original data, thereby avoiding privacy and competition restrictions. Then, each client uses the local annotated data, the local conditional labels and the conditional label distribution information to train the local generator and the local discriminator, and uploads the parameters of the trained local generator and the local discriminator to the server, thereby achieving local model training, using the globally shared conditional label distribution information to guide the local model to learn the data distribution pattern across stores, and adopting a mechanism where the data does not move but the model parameters move, to upload the local model parameters to the server, providing a basis for the aggregation of the global model. Afterwards, the server performs weighted summation on the parameters uploaded by all clients based on the weights of each client (related to the data volume and data quality of the client's local annotated data), determines the parameters of the global generator and the global discriminator, and distributes them back to each client, thereby achieving cross-store knowledge fusion and model optimization by fusing the model parameters of each client through weighted averaging, and obtaining the global generator and the global discriminator. Finally, each client updates the local generator based on the parameters of the global generator, and uses the updated local generator to generate synthetic data of the local annotated data in combination with the local conditional labels and the conditional label distribution information, thereby achieving dynamic production of high-quality synthetic annotated data that is both globally representative (inheriting global parameter rules) and regionally adaptable (fusing local conditional labels) under the premise of zero transmission of original data. Therefore, this application, by constructing globally shared conditional label distribution information and combining federated learning with generative adversarial network technology, achieves the technical effect of breaking down multi-store data silos and generating high-quality, highly diverse, and business-adaptable synthetic annotated data while protecting data privacy. This solves the technical problem that the existing method of expanding labeled data in retail scenarios causes data dispersion and sharing restrictions due to multi-store data silos, resulting in low quality, insufficient diversity and poor business adaptability of the generated synthetic data. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of this application. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the drawings:
[0018] Figure 1is a hardware structure block diagram of a computing device for implementing the method according to embodiment 1 of the present disclosure;
[0019] Figure 2 1 is a schematic diagram of the hardware architecture of the annotation data expansion platform applied in the retail scenario according to Example 1 of the present disclosure;
[0020] Figure 3 This is a flowchart of the annotation data expansion method applied in the retail scenario according to Example 1 of the present application. DETAILED DESCRIPTION
[0021] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present disclosure.
[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0023] Example 1
[0024] According to this embodiment, a method embodiment of a method for expanding labeled data in a retail scenario is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0025] The method embodiment provided in this embodiment can be executed in a server or similar computing device. Figure 1 The following is a hardware block diagram of a computing device for implementing a method for expanding labeled data in a retail scenario. Figure 1As shown, a computing device may include one or more processors (the processor may include, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA) or other processing device), a memory for storing data, a transmission device for communication functions, and an input / output interface. The memory, transmission device, and input / output interface are connected to the processor via a bus. In addition, it may also include: a display, a keyboard, and a cursor control device connected to the input / output interface. Those skilled in the art will understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.
[0026] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry." The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be fully or partially integrated into any of the other components of the computing device. As discussed in the embodiments of the present disclosure, the data processing circuitry functions as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).
[0027] The memory can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method for expanding the labeled data in the retail scenario in the embodiment of the present disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the method for expanding the labeled data in the retail scenario of the above-mentioned application. The memory may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely located relative to the processor, and these remote memories may be connected to the computing device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0028] The transmission device is used to receive or send data via a network. Specific examples of the aforementioned network may include a wireless network provided by a communications provider of the computing device. In one embodiment, the transmission device includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0029] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computing device.
[0030] It should be noted that, in some optional embodiments, the above Figure 1 The computing device shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computing device described above.
[0031] Figure 2 Schematic diagram of the labeling data expansion platform used in retail scenarios according to this embodiment. Figure 2 As shown, the platform includes a server 300 and various clients 200a-200n. Server 300 serves as a centralized coordination node, responsible for cross-domain knowledge fusion and global model evolution. Clients 200a-200n are deployed in various retail stores or regional platforms and serve as edge nodes, responsible for local data encapsulation and lightweight generation.
[0032] Consider a retail company with multiple stores or regional platforms across the country. To achieve precision marketing, each store hopes to leverage local private data to train accurate models to support operations such as precision marketing and inventory optimization. However, the sample size of private data for a single store is often limited. Even with annotation, the resulting annotated dataset is insufficient to effectively train complex models. To overcome this data shortage, retailers need to expand their annotated data to generate more diverse and representative synthetic data, thereby improving model training and generalization.
[0033] Specifically, each client (store) can use its local private data to train a local generative adversarial network (GAN). This GAN consists of a local generator and a local discriminator, which learn the distribution characteristics of local data through adversarial training. After training is complete, the client does not upload the original data, but instead uploads the parameters of the local GAN (i.e., the weights of the generator and discriminator) to server 300.
[0034] Server 300 receives and aggregates the GAN parameters uploaded by all clients to generate a global GAN model. This global model incorporates the data features and knowledge of multiple stores, resulting in stronger generalization and data generation capabilities. Therefore, server 300 transmits the parameters of the global GAN model back to each client.
[0035] Each client then uses the received global GAN model parameters to update its local GAN model and uses the updated model to generate synthetic data. This synthetic data combines local features with global knowledge, retaining the uniqueness of local data while incorporating data patterns from other stores, thereby improving data diversity and quality.
[0036] Ultimately, clients can use the expanded annotated datasets (including both local real-world and synthetic data) to train various business models, such as precision marketing models and inventory forecasting models. These trained models are then applied to real-world business scenarios, helping retail businesses improve operational efficiency, optimize the customer experience, and enhance their market competitiveness.
[0037] It should be noted that the server 300 and each client 200a-200n may be applicable to the hardware structure described above.
[0038] Under the above operating environment, according to the first aspect of this embodiment, a method for expanding labeled data in a retail scenario is provided. Figure 2 The platform implementation shown in . Figure 3 A schematic diagram showing the process of the method is shown in FIG. Figure 3 As shown, the method includes:
[0039] S302: Each client generates a local conditional tag based on the local annotation data and uploads the local conditional tag to the server. The server aggregates the local conditional tags of all clients to obtain globally shared conditional tag distribution information and distributes it back to each client.
[0040] S304: Each client trains a local generator and a local discriminator using the local annotated data, the local conditional labels, and the conditional label distribution information, and uploads the trained parameters of the local generator and local discriminator to the server;
[0041] S306: The server performs weighted summation on the parameters uploaded by all clients according to the weights of each client, determines the parameters of the global generator and the global discriminator, and distributes them back to each client; wherein the weights are related to the amount and quality of the local annotated data of the client; and
[0042] S308: Each client updates the local generator according to the parameters of the global generator, and uses the updated local generator in combination with the local conditional labels and the conditional label distribution information to generate synthetic data of the local annotated data.
[0043] In an embodiment of the present invention, clients 200a-200n deployed in each store / regional platform extract structured business features as local conditional tags based on local annotated data (e.g., user transaction records with tagged information and product inventory data) and upload them to server 300. The tag information included in user transaction records and product inventory data includes, for example, user transaction record tag information (including user ID, transaction time, product ID, transaction amount, and discount information) and product inventory data tag information (including product ID, inventory quantity, storage time, and shelf life). Local conditional tags include, for example, user tags (e.g., purchase frequency (high / low frequency), promotion sensitivity (high / medium / low), category preference (fresh food / daily chemicals / home appliances), etc.) and product tags (e.g., price range (high-end / mid-end / low-end), inventory turnover rate (fast-moving consumer goods / slow-moving goods), and probability of associated sales (high / low)). Server 300 then aggregates the local conditional tags uploaded by clients 200a-200n using a federated learning framework, generates globally shared conditional tag distribution information, and distributes it back to clients 200a-200n. This globally shared conditional tag distribution information might include, for example, that high-frequency fresh food users account for 15% of the country's total, and that the average inventory turnover rate for high-end products is 3.2 times per week.
[0044] In this way, it is possible to refine scattered local knowledge (local conditional labels) into shareable global cognition (conditional label distribution information) without transmitting the original data, circumventing privacy and competition restrictions, and providing important global perspectives and constraints for subsequent local generator and discriminator training.
[0045] Then, each client 200a~200n uses its own local annotated data, local conditional labels, and the globally shared conditional label distribution information distributed by the server 300 to train the local generator and the local discriminator. During the training process, the local generator learns to generate synthetic data that conforms to local features and global features based on the local conditional labels and the globally shared conditional label distribution information distributed by the server 300. The local discriminator discriminates the generated synthetic data to ensure that the generated synthetic data not only conforms to local features but also has global representativeness. Through adversarial training of the local generator and the local discriminator, the data generation capability of the local generator is continuously improved. After the training is completed, each client 200a~200n uploads the parameters of the trained local generator and local discriminator to the server 300. These parameters contain local data features and knowledge, providing a basis for subsequent global model aggregation.
[0046] By introducing globally shared conditional label distribution information, we ensure that local models can incorporate global business knowledge during training, improving the model's generalization capabilities and data generation quality. We enable local model training, using globally shared conditional label distribution information to guide local models in learning data distribution patterns across stores. We also employ a mechanism where model parameters are dynamic while data remains fixed, uploading local model parameters to the server to provide a foundation for global model aggregation.
[0047] Next, the server 300 performs a weighted summation process on the parameters of the local generators and local discriminators uploaded by all clients based on the weights of each client, thereby determining the parameters of the global generator and global discriminator, and distributing these global parameters back to each client. Among them, the weight of the client is closely related to the data volume and data quality of its local annotated data. For example, the larger the data volume and the higher the data quality of the client, the greater its weight may be. During the weighted summation process, the server 300 will comprehensively consider the model parameters uploaded by each client and the corresponding weights, and calculate the parameters of the global generator and global discriminator through existing algorithms (such as the federated average algorithm). These global parameters integrate the data features and knowledge of multiple clients and have stronger generalization and data generation capabilities. After determining the global parameters, the server 300 will distribute them back to each client.
[0048] This enables cross-store knowledge fusion and global model evolution. The global model parameters aggregate data features and knowledge from multiple stores, improving the model's generalization capabilities and data generation quality, and providing stronger support for subsequent synthetic data generation. Furthermore, by assigning weights based on client data volume and quality, the global model more accurately reflects the actual conditions and needs of each store.
[0049] Finally, each client 200a-200n updates its local generator based on the parameters of the global generator distributed from server 300. The updated local generator incorporates the globally shared conditional tag distribution information. When generating synthetic data, each client combines its local conditional tags, such as purchase frequency (high / low frequency), promotion sensitivity (high / medium / low), category preference (fresh food / daily chemicals / home appliances), etc., with the globally shared conditional tag distribution information. In this way, the generated synthetic data retains the uniqueness of local data, such as the purchasing habits and preferences of users in a specific region, while incorporating data patterns from other stores, such as the nationwide proportion of high-frequency fresh food users and the inventory turnover rate of high-end products.
[0050] As a result, it is possible to dynamically produce high-quality and diverse synthetic annotation data that is both globally representative (inheriting global parameter rules) and regionally adaptable (integrating local conditional labels) without any transmission of original data.
[0051] As described in the background, methods for augmenting annotated data in retail scenarios are often limited by data fragmentation and sharing caused by data silos across multiple stores. This results in low-quality, insufficiently diverse, and poorly adaptable synthetic data. Therefore, a method for augmenting annotated data that breaks down data silos and enables cross-domain data collaboration is urgently needed to support high-quality model training in retail scenarios.
[0052] In view of this, the present application first extracts local conditional labels based on local annotated data through each client, and uploads the local conditional labels to the server. The server generates globally shared conditional label distribution information through cross-node aggregation processing and distributes it back to each client, thereby realizing the refinement of dispersed local knowledge (local conditional labels) into sharable global cognition (conditional label distribution information) without transmitting the original data, thereby avoiding privacy and competition restrictions. It realizes the refinement of dispersed local knowledge (local conditional labels) into sharable global cognition (conditional label distribution information) without transmitting the original data, thereby avoiding privacy and competition restrictions. Then, each client uses the local annotated data, the local conditional labels and the conditional label distribution information to train the local generator and the local discriminator, and uploads the parameters of the trained local generator and the local discriminator to the server, thereby realizing local model training, using the globally shared conditional label distribution information to guide the local model to learn the data distribution pattern across stores, and adopting a mechanism where the data is fixed but the model parameters are dynamic, to upload the local model parameters to the server, providing a basis for the aggregation of the global model. Afterwards, the server performs weighted summation on the parameters uploaded by all clients based on the weights of each client (related to the data volume and data quality of the client's local annotated data), determines the parameters of the global generator and the global discriminator, and distributes them back to each client, thereby achieving cross-store knowledge fusion and model optimization by fusing the model parameters of each client through weighted averaging, and obtaining the global generator and the global discriminator. Finally, each client updates the local generator based on the parameters of the global generator, and uses the updated local generator to generate synthetic data of the local annotated data in combination with the local conditional labels and the conditional label distribution information, thereby achieving dynamic production of high-quality synthetic annotated data that is both globally representative (inheriting global parameter rules) and regionally adaptable (fusing local conditional labels) under the premise of zero transmission of original data. Therefore, this application, by constructing globally shared conditional label distribution information and combining federated learning with generative adversarial network technology, achieves the technical effect of breaking down multi-store data silos and generating high-quality, highly diverse, and business-adaptable synthetic annotated data while protecting data privacy. This solves the technical problem that the existing method of expanding labeled data in retail scenarios causes data dispersion and sharing restrictions due to multi-store data silos, resulting in low quality, insufficient diversity and poor business adaptability of the generated synthetic data.
[0053] Optionally, the operation of each client generating a local condition label based on the local annotation data includes: the client extracting key features from the local annotation data; wherein the key features include user behavior features and product attribute features; the client uses a clustering analysis algorithm based on the extracted key features to generate an initial local condition label related to the business goal; and the client optimizes and adjusts the initial local condition label based on business rule constraints to obtain a final local condition label.
[0054] Specifically, each client first extracts key features from local annotated data. These key features cover both user behavior and product attributes. For example, from user transaction records, we can extract user behavior features such as purchase frequency (e.g., high, low), promotion sensitivity (e.g., high, medium, low), and category preferences (e.g., fresh produce, daily necessities, home appliances). Simultaneously, from product inventory data, we can extract product attributes such as price range (e.g., high, mid-range, low-end), inventory turnover (e.g., fast-moving consumer goods, slow-moving goods), and associated sales probability (e.g., high, low).
[0055] Next, the client uses a clustering analysis algorithm based on the extracted key features to generate initial local conditional labels relevant to the business objectives. For example, a K-means clustering algorithm can be used to segment users into different groups based on their purchase frequency and promotion sensitivity, such as those with "high frequency and high promotion sensitivity" and those with "low frequency and low promotion sensitivity." These initial local conditional labels can reflect the characteristics of different user groups and provide strong support for subsequent business model training.
[0056] Finally, the client optimizes and adjusts the initial local conditional labels based on business rule constraints to obtain the final local conditional labels. Business rule constraints include label mutual exclusivity, completeness, and business relevance. For example, when optimizing user category preference labels, we can ensure that each user is classified as having only one primary category preference, avoiding overlapping or conflicting labels. Furthermore, labels can be merged or split based on business needs to better meet the needs of business model training.
[0057] Through the above steps, the client can generate local conditional labels that meet local business characteristics and needs. These labels not only reflect the uniqueness of local data, but also incorporate the requirements of business rules and business goals, providing a high-quality foundation for subsequent global model aggregation and synthetic data generation.
[0058] Optionally, the server aggregates the local conditional tags of all clients to obtain globally shared conditional tag distribution information, including: the server initializes and aggregates the local conditional tags of all clients to obtain a preliminary global conditional tag distribution; the server calculates the similarity score between the local conditional tag of each client and the preliminary global conditional tag distribution; the server uses the Softmax function to calculate the attention weight of each client based on the similarity score; and the server performs weighted summation of the local conditional tags of all clients based on the attention weight to obtain globally shared conditional tag distribution information.
[0059] Specifically, server 300 first performs an initial aggregation of all client local conditional tags to obtain a preliminary global conditional tag distribution. This step aims to initially integrate the local conditional tags uploaded by all clients to form a global tag distribution overview. For example, the server can simply aggregate the user tags and product tags of all clients and count the frequency of occurrence of each tag to obtain a preliminary global tag distribution.
[0060] Next, server 300 calculates a similarity score between each client's local conditional label and the preliminary global conditional label distribution. The purpose of this step is to quantify the degree of difference between each client's local conditional label and the global label distribution. For example, the server can use an algorithm such as cosine similarity or KL divergence to calculate a similarity score between each client's local conditional label vector and the preliminary global label vector. The higher the score, the closer the client's local conditional label is to the global label distribution.
[0061] Then, the server 300 uses the Softmax function to calculate the attention weight of each client based on the similarity score. The Softmax function can convert the similarity score into a probabilistic form, so that the attention weight of each client is between 0 and 1, and the sum of the weights of all clients is 1. In this way, the attention weight reflects the relative importance of each client in the global label distribution aggregation. For example, the higher the similarity of the client with the preliminary global label distribution, the greater its attention weight may be, indicating that its contribution to the global label distribution is higher.
[0062] Finally, server 300 performs a weighted summation of the local conditional tags of all clients based on the attention weights to obtain globally shared conditional tag distribution information. This step fuses the local conditional tags of all clients according to their attention weights through weighted summation, thereby generating a more accurate and comprehensive global conditional tag distribution. For example, the server can perform a weighted average of the local user tags of all clients based on their attention weights to obtain a global user tag distribution, such as the percentage of users nationwide who "frequently purchase fresh food and are highly sensitive to promotions."
[0063] Through the above steps, the server generates a globally shared conditional label distribution that incorporates the local features and knowledge of multiple clients. This global distribution not only reflects the unique business characteristics of each store or region, but also ensures the accuracy and representativeness of the global label distribution through the attention weighting mechanism, providing an important global perspective and constraints for subsequent training of the local generator and local discriminator.
[0064] Optionally, each client uses the local annotated data, the local conditional label and the conditional label distribution information to train the local generator and the local discriminator, including: step 1: dividing the local annotated data into a training set and a validation set; initializing the network parameters of the local generator and the local discriminator; step 2: splicing the local conditional label and the conditional label distribution information of each local annotated data in the training set, and inputting random noise and the spliced local conditional label and conditional label distribution information into the local generator to be trained, and outputting corresponding target synthetic data; wherein the target synthetic data carries corresponding label information; step 3: inputting the target synthetic data output by the local generator into the validation set; The synthetic data and the local annotated data are input into the local discriminator to be trained, and the authenticity evaluation result and the conditional consistency evaluation result of the target synthetic data are output; Step 4: Based on the authenticity evaluation result, the conditional consistency evaluation result output by the local discriminator and the accuracy of the label information of the synthetic data, a preset joint loss function is used to calculate the loss values of the local generator and the local discriminator respectively; Step 5: According to the corresponding loss values, the network parameters of the local generator and the local discriminator are updated through the back propagation algorithm, and the performance of the model is tested using the validation set; and Step 6: Iteratively execute the above steps 2 to 5 until the training termination condition is met.
[0065] Specifically, the client divides the local annotated data into a training set and a validation set. The training set is used for model training, while the validation set is used to evaluate the model's performance during training, prevent overfitting, and serve as a basis for model tuning.
[0066] Next, the client initializes the network parameters of the local generator and local discriminator. Initialization typically uses random initialization or pre-trained initialization to provide a starting point for model training. Proper parameter initialization helps accelerate model convergence and improve training performance.
[0067] The client then concatenates the local conditional labels and conditional label distribution information for each piece of locally annotated data in the training set, feeds random noise and the concatenated label information into the local generator to be trained, and outputs the corresponding target synthetic data. In this step, the local conditional labels reflect the client's local business characteristics, while the conditional label distribution information provides a global business perspective. By concatenating these two pieces of information, the local generator can learn data generation patterns that are both locally characteristic and globally representative. The introduction of random noise increases the diversity of the generated data. The output target synthetic data carries the corresponding label information, which will be used for subsequent discriminator evaluation.
[0068] The client then feeds the target synthesized data and the locally labeled data output by the local generator into the local discriminator to be trained, which then outputs the authenticity assessment results and conditional consistency assessment results for the target synthesized data. The local discriminator evaluates the authenticity of the target synthesized data by comparing the differences between the target synthesized data and the real data. It also checks whether the target synthesized data meets the preset conditional label requirements and evaluates its conditional consistency. These two evaluation results together reflect the data generation quality of the local generator.
[0069] Based on the authenticity evaluation results and conditional consistency evaluation results of the local discriminator output, as well as the label accuracy of the target synthetic data, the client uses a preset joint loss function to calculate the loss values of the local generator and local discriminator respectively. The joint loss function typically consists of two parts: the generator loss and the discriminator loss, which are used to quantify the model's performance in generating and evaluating data. By minimizing the joint loss function, the model parameters can be optimized, improving the performance of the local generator and local discriminator.
[0070] Finally, the client uses the backpropagation algorithm to update the network parameters of the local generator and local discriminator based on the corresponding loss values and tests the model's performance using the validation set. The backpropagation algorithm calculates the gradients of the parameters based on the loss values and updates the parameters to minimize the loss function. After each parameter update, the client uses the validation set to evaluate model performance, such as accuracy and recall. Steps 2 through 5 are iterated until the training termination criteria are met (e.g., reaching the preset number of training rounds or no further improvement in validation set performance), thus completing the model training process.
[0071] Through the above steps, the client can train a high-performance local generator and local discriminator. These models can generate high-quality synthetic data and combine local features with global knowledge for data evaluation, providing a solid foundation for subsequent global model aggregation and synthetic data generation.
[0072] Optionally, the joint loss function of the local generator is:
[0073] ;
[0074] ;
[0075] ;
[0076] ;
[0077] Where, is the joint loss value of the local generator, is the adversarial loss term, is the feature matching loss term, is the classification loss term, 、 and is the weight coefficient of each loss; N is the number of samples, is a random noise vector, is the concatenated local conditional label and conditional label distribution information, G is the local generator, D is the local discriminator, It is the target synthetic data generated by the local generator based on random noise and the concatenated local conditional labels and conditional label distribution information. It is the weighted sum of the authenticity evaluation results and conditional consistency evaluation results of the local discriminator on the target synthetic data; represents the feature extraction function of the intermediate layer of the local discriminator, is a real data sample, Represents the L2 norm; classification loss term It is used to measure the difference between the label information of the target synthetic data generated by the local generator and the spliced local conditional labels and conditional label distribution information. C represents the number of categories of local labeled data in the training set. is the one-hot encoding of the local conditional label. When the local conditional label reflects the true category =1, when the local conditional label reflects a non-real category =0, is the label probability of the target synthetic data output by the local generator.
[0078] In this embodiment of the present invention, the local generator uses a specific joint loss function to optimize its performance. This loss function consists of three key components: an adversarial loss term, a feature matching loss term, and a classification loss term. This design enables the local generator to comprehensively consider data authenticity, feature consistency, and label accuracy during training, thereby generating high-quality synthetic data.
[0079] Specifically, the adversarial loss term is a core component in the Generative Adversarial Network (GAN), which measures the extent to which the synthetic data generated by the generator is recognized as real data by the discriminator. is calculated as the negative log-likelihood of the weighted sum of the local discriminator's evaluation of the authenticity of the synthesized data and the conditional consistency evaluation results. Specifically, the local generator attempts to minimize this loss term so that the generated synthetic data can deceive the local discriminator as much as possible and be considered as real data by the local discriminator.
[0080] The feature matching loss term is used to ensure that the synthetic data generated by the generator is consistent with the real data in the feature space. This is achieved by calculating the L2 norm difference between the features extracted by the local discriminator's intermediate layer for the real data and the synthetic data. Specifically, the local generator attempts to minimize this difference so that the generated synthetic data is indistinguishable from the real data at the feature level.
[0081] The classification loss measures the difference between the label information of the synthetic data generated by the generator and the expected label. The cross-entropy loss is used to calculate the difference between the synthetic data label probability and the one-hot encoding of the concatenated local conditional label and the conditional label distribution information. Specifically, the local generator attempts to minimize this cross-entropy loss to ensure that the generated synthetic data is not only realistic in terms of features but also accurate in terms of label information.
[0082] Combining these three loss terms, we get the joint loss function of the local generator. 、 and are weight coefficients for each loss, used to balance the importance of different loss terms during training. By adjusting these weight coefficients, we can control the degree to which the local generator focuses on authenticity, feature consistency, and label accuracy during training.
[0083] During training, the local generator updates its network parameters using the backpropagation algorithm based on the value of the joint loss function to minimize the loss function. Simultaneously, the local discriminator also updates its own parameters based on its evaluation results to more accurately assess the authenticity and conditional consistency of the synthesized data. This adversarial training process continues iteratively until the training termination criteria are met, resulting in the generation of high-quality synthetic data.
[0084] Therefore, the above design of the local generator joint loss function comprehensively considers the authenticity, feature consistency and label accuracy of the data, enabling the local generator to generate high-quality synthetic data that is both realistic and meets business needs.
[0085] Optionally, the joint loss function of the local discriminator is:
[0086] ;
[0087] - ;
[0088] ;
[0089] ;
[0090] Where, is the joint loss of the local discriminator, is the authenticity loss term, is the conditional consistency loss term, is the label accuracy loss term, 、 and is the weight coefficient of each loss; x represents the distribution of real data Locally labeled data samples sampled from , is a random noise vector, It is the concatenated local conditional label and conditional label distribution information. It is the target synthetic data generated by the local generator based on random noise and the concatenated local conditional labels and conditional label distribution information. represents the probability that the local discriminator judges that the sample x is true, Represents the judgment of the local discriminator is the true probability, Represents the real data distribution The mathematical expectation of the sample x in is calculated. Represents the noise distribution Noise in Perform mathematical expectation calculations; Indicates that the label of the local discriminator prediction sample x belongs to the concatenated local conditional label and conditional label distribution information The probability of represents the local discriminator prediction The label belongs to the local conditional label and conditional label distribution information after splicing The probability of represents the local discriminator prediction The label belongs to the local conditional label probability.
[0091] In this embodiment of the present invention, the local discriminator also uses a specific joint loss function to optimize its performance. This loss function consists of three key components: an authenticity loss term, a conditional consistency loss term, and a label accuracy loss term. This design enables the local discriminator to comprehensively consider data authenticity, conditional consistency, and label accuracy during training, thereby more accurately evaluating the synthetic data produced by the local generator.
[0092] Specifically, the authenticity loss term is the basis for the local discriminator to evaluate the authenticity of the data. It is measured by calculating the difference between the logarithmic loss of the probability that the real data sample x is judged as real by the local discriminator and the logarithmic loss of the probability that the target synthetic data is judged as real by the local discriminator. Specifically, the local discriminator attempts to maximize the probability that the real data is judged as real while minimizing the probability that the target synthetic data is judged as real. This design encourages the local discriminator to more accurately distinguish between real data and synthetic data.
[0093] The conditional consistency loss ensures that the local discriminator fully considers the conditional label information when evaluating the data. It is measured by calculating the difference between the probability that the local discriminator predicts that the real data sample x belongs to the concatenated local conditional label and the conditional label distribution information, and the probability that the target synthetic data belongs to the local conditional label. Specifically, the local discriminator needs to ensure that its evaluation is not only based on the authenticity of the data, but also consistent with the given conditional label. This helps to improve the local discriminator's sensitivity to conditional labels, enabling it to more accurately assess whether the synthetic data meets the preset conditional requirements.
[0094] The label accuracy loss term is used to measure the accuracy of the local discriminator's predicted labels. It is measured by calculating the cross-entropy loss between the labels of the real data sample x predicted by the local discriminator and the actual labels. Specifically, the local discriminator needs to predict the labels of the real data as accurately as possible in order to provide more reliable label information when evaluating synthetic data. This helps improve the performance of the local discriminator on label classification tasks, enabling it to more accurately evaluate the label information of synthetic data.
[0095] Combining these three loss terms, we get the joint loss function of the local discriminator. 、 and is the weight coefficient of each loss, which is used to balance the importance of different loss terms during training. By adjusting these weight coefficients, the degree of attention paid by the local discriminator to authenticity, conditional consistency, and label accuracy during training can be controlled.
[0096] During training, the local discriminator updates its network parameters using the backpropagation algorithm based on the value of the joint loss function to minimize the loss function. Simultaneously, the local generator also updates its own parameters based on the evaluation results of the local discriminator to generate synthetic data that better meets the local discriminator's evaluation criteria. This adversarial training process continues iteratively until the training termination criteria are met, resulting in a local discriminator with excellent performance.
[0097] Therefore, the above design of the joint loss function of the local discriminator comprehensively considers the authenticity, conditional consistency and label accuracy of the data, enabling the local discriminator to more accurately evaluate the synthetic data generated by the local generator, providing strong support for subsequent global model aggregation and synthetic data generation.
[0098] In addition, reference Figure 1 As shown, according to a second aspect of this embodiment, a storage medium is provided, wherein the storage medium includes a stored program, wherein when the program is run, a processor executes any one of the above methods.
[0099] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that the present invention is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.
[0100] Through the description of the above embodiments, those skilled in the art will clearly understand that the methods according to the above embodiments can be implemented using software plus the necessary general-purpose hardware platform. Of course, hardware can also be used, but in many cases the former is a more preferred embodiment. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, or optical disk) and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention.
[0101] Example 2
[0102] According to the labeling data expansion platform for retail scenarios described in this embodiment, the method described in Example 1 is run on the platform. The labeling data expansion platform includes: a server and multiple clients; each client is used to generate local conditional labels based on local labeling data and upload the local conditional labels to the server, the server aggregates the local conditional labels of all clients, obtains globally shared conditional label distribution information, and distributes it back to each client; each client is also used to train a local generator and a local discriminator using the local labeling data, the local conditional labels, and the conditional label distribution information, and uploads the parameters of the trained local generator and local discriminator to the server; the server is used to perform weighted summation processing on the parameters uploaded by all clients based on the weights of each client, determine the parameters of the global generator and the global discriminator, and distribute them back to each client; wherein the weights are related to the data volume and data quality of the local labeling data of the client; and each client is also used to update the local generator based on the parameters of the global generator, and use the updated local generator in combination with the local conditional labels and the conditional label distribution information to generate synthetic data of the local labeling data.
[0103] Optionally, the operation of each client generating a local condition label based on the local annotation data includes: the client extracting key features from the local annotation data; wherein the key features include user behavior features and product attribute features; the client uses a clustering analysis algorithm based on the extracted key features to generate an initial local condition label related to the business goal; and the client optimizes and adjusts the initial local condition label based on business rule constraints to obtain a final local condition label.
[0104] Optionally, the server aggregates the local conditional tags of all clients to obtain globally shared conditional tag distribution information, including: the server initializes and aggregates the local conditional tags of all clients to obtain a preliminary global conditional tag distribution; the server calculates the similarity score between the local conditional tag of each client and the preliminary global conditional tag distribution; the server uses the Softmax function to calculate the attention weight of each client based on the similarity score; and the server performs weighted summation of the local conditional tags of all clients based on the attention weight to obtain globally shared conditional tag distribution information.
[0105] Thus, according to this embodiment, each client first extracts local conditional labels based on local annotated data and uploads the local conditional labels to the server. The server then aggregates the data across nodes to generate globally shared conditional label distribution information and distributes it back to each client. This allows the decentralized local knowledge (local conditional labels) to be refined into a shareable global understanding (conditional label distribution information) without transmitting the original data, thus circumventing privacy and competition restrictions. This allows the decentralized local knowledge (local conditional labels) to be refined into a shareable global understanding (conditional label distribution information) without transmitting the original data, thus circumventing privacy and competition restrictions. Each client then uses the local annotated data, the local conditional labels, and the conditional label distribution information to train a local generator and a local discriminator, and uploads the parameters of the trained local generator and local discriminator to the server. This allows for local model training, using the globally shared conditional label distribution information to guide the local model in learning the data distribution pattern across stores. Using a data-fixed, model-parameter-dynamic mechanism, the local model parameters are uploaded to the server, providing a foundation for global model aggregation. Afterwards, the server performs weighted summation on the parameters uploaded by all clients based on the weights of each client (related to the data volume and data quality of the client's local annotated data), determines the parameters of the global generator and the global discriminator, and distributes them back to each client, thereby achieving cross-store knowledge fusion and model optimization by fusing the model parameters of each client through weighted averaging, and obtaining the global generator and the global discriminator. Finally, each client updates the local generator based on the parameters of the global generator, and uses the updated local generator to generate synthetic data of the local annotated data in combination with the local conditional labels and the conditional label distribution information, thereby achieving dynamic production of high-quality synthetic annotated data that is both globally representative (inheriting global parameter rules) and regionally adaptable (fusing local conditional labels) under the premise of zero transmission of original data. Therefore, this application, by constructing globally shared conditional label distribution information and combining federated learning with generative adversarial network technology, achieves the technical effect of breaking down multi-store data silos and generating high-quality, highly diverse, and business-adaptable synthetic annotated data while protecting data privacy. This solves the technical problem that the existing method of expanding labeled data in retail scenarios causes data dispersion and sharing restrictions due to multi-store data silos, resulting in low quality, insufficient diversity and poor business adaptability of the generated synthetic data.
[0106] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.
[0107] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0108] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
[0109] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.
[0110] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.
[0111] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), a mobile hard drive, a magnetic disk, or an optical disk.
[0112] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.
Claims
1. A method for expanding labeled data in a retail scenario, characterized in that: Applicable to a data annotation platform, the data annotation platform includes a server and multiple clients, and the annotation data expansion method includes: Each client generates a local conditional tag based on the local annotation data and uploads the local conditional tag to the server. The server aggregates the local conditional tags of all clients, obtains globally shared conditional tag distribution information, and distributes it back to each client. Each client trains a local generator and a local discriminator using the local annotated data, the local conditional labels, and the conditional label distribution information, and uploads the parameters of the trained local generator and local discriminator to the server; The server performs weighted summation on the parameters uploaded by all clients based on the weights of each client, determines the parameters of the global generator and the global discriminator, and distributes them back to each client; wherein the weights are related to the amount and quality of the local annotated data of the client; and Each client updates a local generator according to the parameters of the global generator, and uses the updated local generator in combination with the local conditional labels and the conditional label distribution information to generate synthetic data of the local annotated data; The operations of each client generating local conditional labels based on local annotation data include: The client extracts key features from the local annotated data; wherein the key features include user behavior features and product attribute features; The client generates initial local condition labels related to the business objectives using a cluster analysis algorithm based on the extracted key features; and The client optimizes and adjusts the initial local condition label based on business rule constraints to obtain a final local condition label; The server aggregates the local condition tags of all clients to obtain globally shared condition tag distribution information, including: The server initializes and aggregates the local condition tags of all clients to obtain a preliminary global condition tag distribution; The server calculates a similarity score between the local condition tag of each client and the preliminary global condition tag distribution; The server uses a Softmax function to calculate the attention weight of each client according to the similarity score; and The server performs weighted summation on the local conditional labels of all clients according to the attention weights to obtain globally shared conditional label distribution information; The operations of each client using the local annotated data, the local conditional labels, and the conditional label distribution information to train a local generator and a local discriminator include: Step 1: Divide the local labeled data into a training set and a validation set; initialize the network parameters of the local generator and the local discriminator; Step 2: Concatenate the local conditional labels and conditional label distribution information of each piece of local annotated data in the training set, input random noise and the concatenated local conditional labels and conditional label distribution information into the local generator to be trained, and output the corresponding target synthetic data; wherein the target synthetic data carries the corresponding label information; Step 3: Input the target synthetic data output by the local generator and the local annotated data into the local discriminator to be trained, and output the authenticity evaluation result and conditional consistency evaluation result of the target synthetic data; Step 4: Based on the authenticity evaluation result and conditional consistency evaluation result output by the local discriminator and the accuracy of the label information of the synthetic data, a preset joint loss function is used to calculate the loss values of the local generator and the local discriminator respectively; Step 5: Based on the corresponding loss values, update the network parameters of the local generator and the local discriminator through the back-propagation algorithm, and use the validation set to test the performance of the model; and Step 6: Iterate steps 2 to 5 above until the training termination condition is met.
2. The method according to claim 1, characterized in that The joint loss function of the local generator is: ; ; ; ; Where, is the joint loss value of the local generator, is the adversarial loss term, is the feature matching loss term, is the classification loss term, 、 and is the weight coefficient of each loss; N is the number of samples, is a random noise vector, is the concatenated local conditional label and conditional label distribution information, G is the local generator, D is the local discriminator, It is the target synthetic data generated by the local generator based on random noise and the concatenated local conditional labels and conditional label distribution information. It is the weighted sum of the authenticity evaluation results and conditional consistency evaluation results of the local discriminator on the target synthetic data; represents the feature extraction function of the intermediate layer of the local discriminator, is a real data sample, Represents the L2 norm; classification loss term It is used to measure the difference between the label information of the target synthetic data generated by the local generator and the spliced local conditional labels and conditional label distribution information. C represents the number of categories of local labeled data in the training set. is the one-hot encoding of the local conditional label. When the local conditional label reflects the true category =1, when the local conditional label reflects a non-real category =0, is the label probability of the target synthetic data output by the local generator.
3. The method according to claim 1, characterized in that The joint loss function of the local discriminator is: ; ; ; ; Where, is the joint loss of the local discriminator, is the authenticity loss term, is the conditional consistency loss term, is the label accuracy loss term, 、 and is the weight coefficient of each loss; x represents the distribution of real data Locally labeled data samples sampled from , is a random noise vector, It is the concatenated local conditional label and conditional label distribution information. It is the target synthetic data generated by the local generator based on random noise and the concatenated local conditional labels and conditional label distribution information. represents the probability that the local discriminator judges that the sample x is true, Represents the judgment of the local discriminator is the true probability, Represents the real data distribution The mathematical expectation of the sample x in is calculated. Represents the noise distribution Noise in Perform mathematical expectation calculations; Indicates that the label of the local discriminator prediction sample x belongs to the concatenated local conditional label and conditional label distribution information The probability of represents the local discriminator prediction The label belongs to the local conditional label and conditional label distribution information after splicing The probability of represents the local discriminator prediction The label belongs to the local conditional label probability.
4. A storage medium, characterized in that The storage medium includes a stored program, wherein when the program is run, the processor executes the method according to any one of claims 1 to 3.
5. A labeling data expansion platform for retail scenarios, characterized by: Includes a server and multiple clients; Each client is used to generate a local conditional tag based on the local annotation data and upload the local conditional tag to the server. The server aggregates the local conditional tags of all clients to obtain globally shared conditional tag distribution information and distributes it back to each client. Each client is further configured to train a local generator and a local discriminator using the local annotated data, the local conditional labels, and the conditional label distribution information, and upload the parameters of the trained local generator and local discriminator to the server; The server is used to perform weighted summation processing on the parameters uploaded by all clients according to the weights of each client, determine the parameters of the global generator and the global discriminator, and distribute them back to each client; wherein the weights are related to the data volume and data quality of the local annotated data of the client; and Each client is further configured to update a local generator according to the parameters of the global generator, and use the updated local generator in combination with the local conditional labels and the conditional label distribution information to generate synthetic data of the local annotated data; The operations of each client generating local conditional labels based on local annotation data include: The client extracts key features from the local annotated data; wherein the key features include user behavior features and product attribute features; The client generates initial local condition labels related to the business objectives using a cluster analysis algorithm based on the extracted key features; and The client optimizes and adjusts the initial local condition label based on business rule constraints to obtain a final local condition label; The server aggregates the local condition tags of all clients to obtain globally shared condition tag distribution information, including: The server initializes and aggregates the local condition tags of all clients to obtain a preliminary global condition tag distribution; The server calculates a similarity score between the local condition tag of each client and the preliminary global condition tag distribution; The server uses a Softmax function to calculate the attention weight of each client according to the similarity score; and The server performs weighted summation on the local conditional labels of all clients according to the attention weights to obtain globally shared conditional label distribution information; The operations of each client using the local annotated data, the local conditional labels, and the conditional label distribution information to train a local generator and a local discriminator include: Step 1: Divide the local labeled data into a training set and a validation set; initialize the network parameters of the local generator and the local discriminator; Step 2: Concatenate the local conditional labels and conditional label distribution information of each piece of local annotated data in the training set, input random noise and the concatenated local conditional labels and conditional label distribution information into the local generator to be trained, and output the corresponding target synthetic data; wherein the target synthetic data carries the corresponding label information; Step 3: Input the target synthetic data output by the local generator and the local annotated data into the local discriminator to be trained, and output the authenticity evaluation result and conditional consistency evaluation result of the target synthetic data; Step 4: Based on the authenticity evaluation result and conditional consistency evaluation result output by the local discriminator and the accuracy of the label information of the synthetic data, a preset joint loss function is used to calculate the loss values of the local generator and the local discriminator respectively; Step 5: Based on the corresponding loss values, update the network parameters of the local generator and the local discriminator through the back-propagation algorithm, and use the validation set to test the performance of the model; and Step 6: Iterate steps 2 to 5 above until the training termination condition is met.
Citation Information
Patent Citations
Data annotation method and platform, electronic equipment and computer storage medium
CN113392295A
Data annotation method, device and system and storage medium
CN116049351A
Method, device, equipment for generating multi-task model, medium and program product
CN112541124A
Data augmentation method and system based on multimodal image federated segmentation
CN116580188A