Product data processing method, device, storage medium and processor

Abstract

The present invention discloses a method, device, storage medium and processor for processing product data. The method comprises: obtaining an image data set of a product object, wherein the image data set comprises: multiple images and multiple image frames of a video; distributing multiple images in the image data set to multiple processes for distributed calculation, and at least one process obtains a local gradient in the back propagation process; aggregating at least one local gradient obtained to obtain an aggregated gradient; and using the aggregated gradient to train a data representation model. The present invention solves the technical problem of inaccurate structured modeling results of product data.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a method, device, storage medium and processor for processing product data. Background Art

[0002] Currently, supervised learning methods are usually used to model product data in a structured manner.

[0003] The supervised learning method is to manually label the product information of the goods, and then complete the structured modeling by learning the mapping relationship between the product information and the manual labels. However, this supervised learning method cannot make full use of larger-scale unlabeled data to improve the model accuracy, and it is difficult to cover complex and changing scenes, product types and shooting conditions, resulting in inaccurate structured modeling results for product data.

[0004] Currently, no effective solution has been proposed to the above-mentioned technical problem of inaccurate structured modeling results of product data. Summary of the invention

[0005] Embodiments of the present invention provide a method, device, storage medium, and processor for processing product data, so as to at least solve the technical problem of inaccurate structured modeling results of product data.

[0006] According to one aspect of an embodiment of the present invention, a method for processing product data is provided. The method may include: obtaining an image dataset of a product object, wherein the image dataset includes: multiple images and multiple image frames of a video, and the images in the image dataset are not labeled; distributing the multiple images in the image dataset to multiple processes for distributed calculation, and at least one process obtains a local gradient in the back propagation process; aggregating at least one local gradient obtained to obtain an aggregated gradient; using the aggregated gradient to train a data representation model

[0007] According to another aspect of an embodiment of the present invention, another method for processing product data is also provided. The method may include: entering an image data set of a product object on an input page on an operation interface, wherein the image data set includes: multiple images and multiple image frames of a video, and none of the images in the image data set are labeled; sensing a model generation instruction in the operation interface, distributing multiple images in the image data set to multiple processes for distributed calculation, wherein at least one process obtains a local gradient in the back propagation process, and aggregating at least one local gradient obtained to obtain an aggregated gradient; displaying a prompt message of a training data representation model on the operation interface, wherein the training data representation model is trained using an aggregated gradient.

[0008] According to another aspect of an embodiment of the present invention, another method for processing product data is also provided. The method may include: displaying an image data set of a product object on an interactive interface, wherein the image data set includes: multiple images and multiple image frames of a video, and none of the images in the image data set are labeled; sensing a model generation instruction in the interactive interface; responding to the model generation instruction, selecting multiple images from the image data set, and distributing the multiple images to multiple processes for distributed calculation, wherein at least one process obtains a local gradient in the back propagation process, and aggregates at least one local gradient obtained to obtain an aggregated gradient; displaying prompt information of a training data representation model on the interactive interface, wherein the training data representation model is trained using an aggregated gradient.

[0009] According to another aspect of an embodiment of the present invention, another method for processing product data is also provided. The method may include: a front-end client uploading an image dataset of a product object, wherein the image dataset includes: multiple images and multiple image frames of a video, and none of the images in the image dataset are labeled; the front-end client transmits the image dataset to a back-end server; the front-end client receives a trained data representation model returned by the back-end server, wherein multiple images in the image dataset are distributed by the back-end server to multiple processes for distributed calculation, at least one process obtains a local gradient in the back-propagation process, aggregates the at least one local gradient obtained, obtains an aggregated gradient, and uses the aggregated gradient to train the data representation model.

[0010] According to another aspect of an embodiment of the present invention, a device for processing product data is also provided. The device may include: an acquisition unit, used to acquire an image data set of a product object, wherein the image data set includes: multiple images and multiple image frames of a video, and the images in the image data set are not labeled; a distribution unit, used to distribute the multiple images in the image data set to multiple processes for distributed calculation, and at least one process obtains the local gradient in the back propagation process; an aggregation unit, used to aggregate at least one local gradient obtained to obtain an aggregated gradient; and a training unit, used to use the aggregated gradient to train a data representation model.

[0011] According to another aspect of an embodiment of the present invention, another device for processing product data is provided. The device may include: an input unit, used to input an image data set of a product object on an input page on an operation interface, wherein the image data set includes: multiple images and multiple image frames of a video, and none of the images in the image data set are labeled; a first sensing unit, used to sense a model generation instruction in the operation interface, distribute multiple images in the image data set to multiple processes for distributed calculation, wherein at least one process obtains a local gradient in the back propagation process, and aggregates at least one local gradient obtained to obtain an aggregated gradient; a first display unit, used to display prompt information of the training data representation model on the operation interface, wherein the training data representation model is trained using an aggregated gradient.

[0012] According to another aspect of an embodiment of the present invention, another device for processing product data is provided. The device may include: a second display unit, used to display an image data set of a product object on an interactive interface, wherein the image data set includes: multiple images and multiple image frames of a video, and the images in the image data set are not labeled; a second sensing unit, used to sense a model generation instruction in the interactive interface; a response unit, used to respond to the model generation instruction, select multiple images from the image data set, and distribute the multiple images to multiple processes for distributed calculation, wherein at least one process obtains a local gradient in the back propagation process, and aggregates at least one local gradient obtained to obtain an aggregated gradient; a third display unit, used to display prompt information of the training data representation model on the interactive interface, wherein the training data representation model is trained using aggregated gradients.

[0013] According to another aspect of an embodiment of the present invention, another device for processing product data is provided. The device may include: an uploading unit, used to enable a front-end client to upload an image data set of a product object, wherein the image data set includes: multiple images and multiple image frames of a video, and the images in the image data set are not labeled; a transmission unit, used to enable the front-end client to transmit the image data set to a back-end server; a receiving unit, used to enable the front-end client to receive a trained data representation model returned by the back-end server, wherein multiple images in the image data set are distributed by the back-end server to multiple processes for distributed calculation, at least one process obtains a local gradient in the back-propagation process, aggregates the at least one local gradient obtained to obtain an aggregated gradient, and uses the aggregated gradient to train the data representation model.

[0014] According to another aspect of the present invention, a computer-readable storage medium is provided, which includes a stored program, wherein when the program is executed by a processor, the device where the computer-readable storage medium is located is controlled to execute the method for processing product data of the present invention.

[0015] According to another aspect of an embodiment of the present invention, a processor is further provided, wherein the processor is used to run a program, wherein when the program is run, the method for processing product data of an embodiment of the present invention is executed.

[0016] According to another aspect of an embodiment of the present invention, a product data processing system is also provided. The system may include: a processor; a memory connected to the processor, and used to provide the processor with instructions for processing the following processing steps: obtaining an image data set of a product object, wherein the image data set includes: multiple images and multiple image frames of a video, and none of the images in the image data set are labeled; distributing multiple images in the image data set to multiple processes for distributed calculation, and at least one process obtains a local gradient in the back propagation process; aggregating at least one local gradient obtained to obtain an aggregated gradient; and using the aggregated gradient to train a data representation model.

[0017] In an embodiment of the present invention, an image data set of a product object is obtained, wherein the image data set includes: multiple images and multiple image frames of a video, and the images in the image data set are not labeled; the multiple images in the image data set are distributed to multiple processes for distributed calculation, and at least one process obtains the local gradient in the back propagation process; the at least one local gradient obtained is aggregated to obtain an aggregated gradient; and the aggregated gradient is used to train the data representation model. In other words, the present application uses a supervised machine learning method, which can be distributedly trained only based on the image data set of unlabeled product objects, thereby effectively constructing a data representation model, which is completely unaffected by other large amounts of noise, avoiding the inability of supervised learning methods to fully utilize larger-scale unlabeled data to improve model accuracy, and also avoiding the difficulty of a small amount of labeled data to cover complex and changeable scenes, product types and shooting conditions, and the model constructed on these data may have certain data deviations, thereby solving the technical problem of inaccurate structured modeling results of product data, and achieving the technical effect of improving the accuracy of structured modeling results of product data. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0019] Figure 1is a hardware structure block diagram of a computer terminal (or mobile device) for implementing product data processing according to an embodiment of the present invention;

[0020] Figure 2 is a flow chart of a method for processing product data according to an embodiment of the present invention;

[0021] Figure 3 is a flowchart of another method for processing product data according to an embodiment of the present invention;

[0022] Figure 4 is a flowchart of another method for processing product data according to an embodiment of the present invention;

[0023] Figure 5 is a flowchart of another method for processing product data according to an embodiment of the present invention;

[0024] Figure 6 is a schematic diagram of a noise contrast loss function according to an embodiment of the present invention;

[0025] Fig. 7A is a schematic diagram of a distributed training process according to an embodiment of the present invention;

[0026] Figure 7B is a schematic diagram of an interactive interface of a method for processing product data according to an embodiment of the present invention;

[0027] Figure 7C is a scenario schematic diagram of a method for processing product data according to an embodiment of the present invention;

[0028] Figure 8 is a schematic diagram of a device for processing product data according to an embodiment of the present invention;

[0029] Fig. 9 is a schematic diagram of another device for processing product data according to an embodiment of the present invention;

[0030] Fig.10 is a schematic diagram of another device for processing product data according to an embodiment of the present invention;

[0031] Fig.11 is a schematic diagram of another device for processing product data according to an embodiment of the present invention; and

[0032] Fig.12 It is a structural block diagram of a computer terminal according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0036] Deep learning is an artificial neural network structure with a high number of layers, which is used to realize intelligent image detection, classification and other functions.

[0037] Self-supervision is a type of machine learning method that can build models directly from the data itself without human-provided data labels;

[0038] The data representation model is a model obtained by structured modeling of the data of the product object;

[0039] Standard Product Unit (SPU) is a set of reusable and easily searchable standardized information that describes the characteristics of a product.

[0040] Noise Contrastive Estimation (NCE) is used to calculate local loss based on the local similarity matrix;

[0041] Transfer Learning is a machine learning method that transfers knowledge from one field (source field) to another field (target field) so that the target field can achieve better learning results.

[0042] Example 1

[0043] According to an embodiment of the present invention, an embodiment of a method for processing product data is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0044] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 1 is a hardware structure block diagram of a computer terminal (or mobile device) for implementing product data processing according to an embodiment of the present invention. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0045] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0046] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the method for processing product data in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the method for processing product data of the above-mentioned application program is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0047] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0048] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0049] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0050] It should be noted that, in some optional embodiments, the above Figure 1 The computer device (or mobile device) shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the above-described computer device (or mobile device).

[0051] exist Figure 1 In the operating environment shown, this application provides Figure 2 It should be noted that the method for processing product data in this embodiment can be Figure 1The illustrated embodiment is executed by a mobile terminal.

[0052] Figure 2 FIG. 1 is a flow chart of a method for processing product data according to an embodiment of the present invention. Figure 2 As shown, the method may include the following steps:

[0053] Step S202: acquiring an image dataset of the product object, wherein the image dataset includes: a plurality of pictures and a plurality of image frames of a video, and none of the images in the image dataset are labeled.

[0054] In the technical solution provided in the above step S202 of the present invention, the product object may be a commodity object, for example, a new commodity to be released by the seller. An image dataset of the product object is obtained, and the image dataset may be data of multiple images obtained by photographing the product object, and may include multiple pictures of the product object and multiple image frames of a video, wherein, when the product object is a commodity, the picture of the product object may be referred to as commodity picture data. Optionally, this embodiment obtains the image dataset of the above product object from an e-commerce platform, and the e-commerce platform is not specifically limited here.

[0055] In this embodiment, the images in the above-mentioned image data set are not labeled, that is, the image data set can be large-scale (massive) unlabeled data, so the product data processing method of this embodiment is for the unlabeled image data set.

[0056] Step S204: distribute the multiple images in the image data set to multiple processes for distributed calculation, and at least one process obtains the local gradient in the back propagation process.

[0057] In the technical solution provided in the above step S204 of the present invention, after obtaining the image data set of the product object, multiple images in the image data set are distributed to multiple processes for distributed calculation, and at least one process obtains the local gradient in the back propagation process.

[0058] This embodiment can be a distributed training scheme for parallel processes, distributing multiple images in an image data set to multiple processes for distributed calculation, wherein the multiple processes can be all current processes, such as process 1, process 2...process N. At least one of the multiple processes can be used to calculate a local similarity matrix and a local loss, and optionally, the at least one process can perform distributed calculations on multiple images in the image data set on a corresponding graphics processing unit (GPU). Optionally, during the back propagation process, this embodiment returns local gradients to each process, thereby achieving efficient training of the model, wherein the local gradients are also gradient information.

[0059] Step S206: Aggregate the at least one local gradient obtained to obtain an aggregated gradient.

[0060] In the technical solution provided in the above step S206 of the present invention, after at least one process obtains the local gradient in the back-propagation process, at least one local gradient obtained can be aggregated in at least one process to obtain the aggregated gradient under the above at least one process to complete the optimization of the model on each process, that is, the aggregated gradient of this embodiment is for at least one process, and this training process improves the training efficiency of the data representation model.

[0061] Step S208: Use the aggregated gradient to train the data representation model.

[0062] In the technical solution provided in the above step S208 of the present invention, after aggregating at least one local gradient obtained to obtain an aggregated gradient, the aggregated gradient is used to train the data representation model, for example, the aggregated gradient is used to learn (optimize) a data representation model with strong discriminability, and the data representation model can be called a pre-trained model. Optionally, the data representation model of this embodiment can be used as a feature extractor and a classifier, wherein the feature extractor is used to realize feature extraction, and the classifier is used to realize feature classification, and the data representation model can be migrated to the product recognition task to improve the accuracy and generalization performance of product object recognition.

[0063] It should be noted that the above method of this embodiment does not require artificially provided image data labels, and can directly construct a data representation model from the image data set itself. Therefore, the above method of this embodiment is a self-supervised machine learning method.

[0064] This embodiment constructs a data representation model in a self-supervised learning manner based only on information of a large amount of image data sets themselves, and can fine-tune the data representation model by migrating it to a small amount of manually annotated data to achieve higher performance of the model.

[0065] This embodiment can achieve the purpose of continuously optimizing the data characterization model by cyclically executing the above steps S202 to S208, so as to improve the performance of the data characterization model.

[0066] The method for processing product data of this embodiment is a data structured modeling process of a product object. Through the above steps S202 to S208 of this application, an image data set of a product object is obtained, wherein the image data set includes: multiple image frames of multiple pictures and videos, and the images in the image data set are not labeled; the multiple images in the image data set are distributed to multiple processes for distributed calculation, and at least one process obtains the local gradient in the back propagation process; the at least one local gradient obtained is aggregated to obtain an aggregated gradient; and the aggregated gradient is used to train the data representation model. In other words, this embodiment uses a supervised machine learning method, and can perform distributed training on the image data set of unlabeled product objects only, thereby effectively constructing a data representation model, which is completely unaffected by other large amounts of noise, avoiding the inability of supervised learning methods to fully utilize larger unlabeled data to improve model accuracy, and also avoiding the difficulty of a small amount of labeled data to cover complex and changeable scenes, product types and shooting conditions, and the model constructed on these data may have certain data deviations, thereby solving the technical problem of inaccurate structured modeling results of product data, and achieving the technical effect of improving the accuracy of structured modeling results of product data.

[0067] The above method of this embodiment is further introduced below.

[0068] As an optional implementation, step S204 distributes multiple images in the image data set to multiple processes for distributed calculation, and at least one process obtains the local gradient in the back propagation process, including: obtaining image features of each image in the image data set, and distributing the image features of each image to different processes; calculating the similarity matrix between image features, and obtaining the local similarity matrix generated under at least one process; performing local loss calculation on the local similarity matrix generated under at least one process, and generating the local loss under at least one process; performing gradient calculation based on the local loss obtained by at least one process, and obtaining the local gradient in the back propagation process.

[0069] In this embodiment, the image data set includes multiple images. In the process of training the data characterization model, the algorithm needs to aggregate all image features stored in different machines and different processes. Therefore, this embodiment distributes multiple images in the image data set to multiple processes for distributed calculation. When at least one process obtains the local gradient in the back propagation process, image features can be first extracted from each image in the image data set, and each image can be input into the characterization model to obtain the image features of each image, and then the image features of each image are distributed to different processes. For example, the image features of each image are evenly distributed to different processes, and at least one process is only used to calculate the local similarity matrix and the local loss. The similarity matrix calculation and the loss calculation can be evenly distributed to each process to complete the calculation of the local loss, thereby realizing a distributed training scheme based on parallel processes.

[0070] Optionally, this embodiment calculates the similarity matrix between the image features of each image through at least one process, obtains the local similarity matrix generated under at least one process according to the similarity matrix, and then performs local loss calculation for the local similarity matrix generated under at least one process to generate the local loss (loss value) under at least one process. It should be noted that the data transmission amount, calculation amount and storage amount of the above process of this embodiment are relatively large.

[0071] After generating the local loss of at least one process, gradient calculation can be performed based on the local loss obtained by at least one process, that is, the local gradients are distributed and calculated on each GPU to obtain multiple local gradients in the back propagation process. The local gradients on multiple GPUs can be averaged to process data on a large scale, so as to achieve the purpose of aggregating multiple local gradients and training the data representation model according to multiple local gradients, thereby improving the training efficiency of the data representation model.

[0072] As an optional implementation, image features of each image in the image data set are obtained: a neural network model is used to identify image features of each image in the image data set, and image features of each image are extracted.

[0073] In this embodiment, when acquiring the image features of each image in the image data set, a pre-trained neural network model may be read, and the neural network model may include but is not limited to a convolutional neural network model, and then the above neural network model is used to identify the image features of each image in the image data set. After identifying the image features of each image in the image data set, the image features may be extracted from each image to distribute the image features of each image to different processes, thereby realizing a distributed training scheme for the data representation model.

[0074] As an optional implementation, a similarity matrix between image features is calculated to obtain a local similarity matrix generated under at least one process, including: in the process of calculating the local similarity matrix in any process, image features generated in other processes are pulled; similarity is calculated using the pulled image features and image features in the current process to generate a local similarity matrix under the current process.

[0075] In this embodiment, when calculating the similarity matrix between image features and obtaining the local similarity matrix generated under at least one process, it can be that in the process of calculating the local similarity matrix in any one process, image features generated in other processes except the above process in multiple processes are also pulled at the same time, that is, all image features are pulled from other GPUs and aggregated on one GPU to calculate the local similarity matrix, wherein any one process can also be called the current process, and the pulled image features can be used to perform similarity calculation with the image features in the current process, thereby generating the local similarity matrix under the current process.

[0076] It should be noted that the local similarity matrix of at least one process among the above-mentioned multiple processes can be obtained according to the above-mentioned method.

[0077] As an optional implementation, a noise contrast loss function is used to perform local loss calculation on the local similarity matrix generated under at least one process to generate a local loss under at least one process, wherein the noise contrast loss function is used to reduce the image feature distance between different transformations in the same image, so that the image feature distance between different images is expanded.

[0078] In this embodiment, when calculating the local loss according to the local similarity matrix, the noise contrast loss function may be read, and then the noise contrast loss function is used to calculate the local loss for the local similarity matrix generated in at least one process. For example, the noise contrast loss function may be expressed by the following formula:

[0079]

[0080] Among them, i and j are used to represent two transformations of the image of the same product object, which transformation may include but is not limited to scaling, cutting out, color transformation, etc., wherein the meaning of the image is not changed in various transformations, and k is used to represent images of other product objects.

[0081] Through the above-mentioned noise contrast loss function of this embodiment, the distance between image features of different transformations in the same image can be gradually reduced, that is, the distance between image features of the image between different transformations can be gradually shortened; the noise contrast loss function of this embodiment can also gradually expand the distance between image features of different images, that is, the distance between image features of the image between different transformations can be gradually extended. Taking image A and image B of a product object as an example, the noise contrast loss function can require that the distance between image features of the same image between different transformations be as close as possible, while the distance between image features between different transformations can be as far as possible, so as to effectively learn a data representation model with strong discriminative ability.

[0082] As an optional implementation, step S206, aggregating at least one local gradient obtained to obtain an aggregated gradient, includes: distributing the local gradient under any process to other processes; at least one process uses the local gradient in the current process and the local gradient distributed from other processes to aggregate to generate an aggregated gradient under at least one process.

[0083] In this embodiment, when aggregating at least one acquired local gradient to obtain an aggregated gradient, the local gradient under any one process may be distributed to other processes among multiple processes except any one process, and at least one process is determined as the current process, so that at least one process uses the local gradient in the current process and the local gradient distributed from other processes to aggregate, thereby generating an aggregated gradient under the at least one process.

[0084] Optionally, when any process aggregates the gradient, the local gradients of other processes may be pulled; the pulled local gradients are aggregated with the local gradients in the current process to generate the aggregated gradient of the current process.

[0085] As an optional implementation, the frequency of gradient aggregation is determined based on the communication volume, computing volume, and access times between at least one process.

[0086] In this embodiment, at least one local gradient obtained is aggregated, and the obtained aggregated gradient has a certain frequency, which can be determined based on the communication volume, calculation volume and access times between at least one process, wherein the communication volume is the amount of data communicated between at least one process, the calculation volume is the amount of data calculated between at least one process, and the access times is the number of times data is stored and read between at least one process.

[0087] As an optional implementation, in step S208, after using the aggregated gradient to train the data representation model, the method further includes: obtaining a recognition task of product information; and using the trained data representation model to recognize the recognition task to obtain product information.

[0088] In this embodiment, the trained data representation model can be transferred to the product information recognition task through transfer learning, wherein the product information can be a text description content used to describe the product object, for example, commodity information, which can include but is not limited to the product title, product selling points and other information of the product object, and can also be the product object's category, attribute, category, Standard Product Unit (Standard Product Unit, referred to as SPU) and other structured information, which can be generated by acquiring product data of the product object, analyzing the product data, generating multimodal information of the product object, and using a multimodal network model to process the multimodal information, wherein the product data includes at least one of the following: product image information, video information and text information; analyzing the product data to generate multimodal information of the product object, wherein the multimodal information includes: feature sequences between different modal information, wherein the multimodal network model can be a multimodal transformer network model.

[0089] This embodiment uses the trained data representation model to identify the recognition task, which can be to copy the model parameters of the trained data representation model to the model of the product information recognition task as initialization, and then continue to train the model of the product information recognition task, wherein the recognition task can be a task of distinguishing different product objects, for example, the recognition task is a task of distinguishing shirts from T-shirts. Optionally, in the case where the product object is a commodity, the model of the product information recognition task can be called a commodity recognition model, and the recognition task is recognized by the model, so as to recognize the above-mentioned product information used to describe the product object, so that the accuracy of the recognition of the category, attribute, category, etc. of the product object is higher and the performance is more stable, thereby achieving the improvement of the accuracy, generalization performance and cross-domain capability of product information recognition.

[0090] As an optional implementation, after using the trained data representation model to identify product information, the method further includes: generating multiple types of product materials based on the product information; and publishing multiple product materials.

[0091] In this embodiment, after generating product information for describing the product object, for example, after generating the product title and product selling points of the product object, various types of product materials can be generated therefrom. The product materials are the materials required for publishing the product object, and can be picture product materials, video product materials, text product materials, etc. Each type of product material can include the above-mentioned product information, and then publish multiple product materials.

[0092] As an optional implementation, after generating multiple types of product materials, the method also includes: uploading product materials to be released, and extracting multiple product contents to be verified in the product materials to be released; determining whether at least one product content to be verified meets the entry criteria; if so, successfully entering the product material into the release template; otherwise, preprocessing the product content that failed the verification, and entering the product material into the release template if the preprocessed product content meets the entry criteria.

[0093] In this embodiment, after the product material to be published is generated, the product material to be published can be uploaded to the product publishing platform, and multiple product contents to be verified can be extracted from the product material to be published. The multiple product contents are the contents in the product material that need to be entered into the publishing template. At least one product content to be verified can be determined from the multiple product contents, and then it is determined whether the at least one product content to be verified meets the entry standard. The entry standard is also a standard used to determine whether the at least one product content meets the specification. If it is determined that the at least one product content to be verified meets the entry standard, it is determined that the product material verification is successful, and the product material can be successfully entered into the publishing template; if it is determined that the at least one product content to be verified does not meet the standard, it is determined that the at least one product content verification fails, and the product content that fails the verification is preprocessed, for example, the product content that fails the verification is modified and adjusted, and it is determined whether the preprocessed product content meets the entry standard. If it is determined that the preprocessed product content meets the entry standard, the product material can be entered into the publishing template, so that the product object can be published on the product publishing platform through the publishing template.

[0094] The embodiment of the present invention also provides another method for processing product data from the perspective of human-computer interaction.

[0095] Figure 3 FIG. 4 is a flowchart of another method for processing product data according to an embodiment of the present invention. Figure 3 As shown, the method may include the following steps:

[0096] Step S302, inputting an image data set of a product object on an input page on an operation interface, wherein the image data set includes: multiple pictures and multiple image frames of a video, and none of the images in the image data set are labeled.

[0097] In the technical solution provided in the above step S302 of the present invention, an input page is displayed on the operation interface, and the input page is used to input the image data set of the product object. The product object can be a commodity object, for example, a new commodity to be released by the seller. The image data set can be data of multiple images obtained by shooting the product object, and can include multiple pictures of the product object and multiple image frames of the video.

[0098] It should be noted that the images in the above image dataset are not labeled, that is, the image dataset may be large-scale (massive) unlabeled data, so the product data processing method of this embodiment is for unlabeled image datasets.

[0099] Step S304, sensing the model generation instruction in the operation interface, distributing multiple images in the image data set to multiple processes for distributed calculation, wherein at least one process obtains the local gradient in the back propagation process, and aggregates the at least one local gradient obtained to obtain an aggregated gradient.

[0100] In the technical solution provided by the above step S304 of the present invention, after the image data set of the product object is entered on the input page of the operation interface, a model generation instruction can be received and sensed in the operation interface, and the model generation instruction is used to instruct the generation of a data representation model. Optionally, the operation interface of this embodiment can respond to the above model generation instruction to distribute multiple images in the image data set to multiple processes for distributed calculation, wherein the multiple processes can be all current processes. At least one process can be used to calculate the local similarity matrix and the local loss, and optionally, at least one process can perform distributed calculations on multiple images in the image data set on the corresponding GPU. Optionally, this embodiment returns local gradients to each process during back propagation, thereby achieving efficient training of the model.

[0101] Step S306: displaying prompt information of the training data characterization model on the operation interface, wherein the training data characterization model is trained using aggregated gradients.

[0102] In the technical solution provided in the above step S306 of the present invention, after aggregating at least one local gradient obtained to obtain an aggregated gradient, the aggregated gradient can be used to train the data characterization model, and then a prompt information is displayed on the operation interface. The prompt information can be used to indicate that the data characterization model is being trained, or that the training of the data characterization model has been completed.

[0103] This embodiment uses aggregated gradients to learn (optimize) a data representation model with strong discriminativeness. Optionally, the data representation model of this embodiment can be used as a feature extractor and classifier, and can be transferred to the product recognition task to improve the accuracy and generalization performance of product object recognition.

[0104] It should be noted that the above method of this embodiment does not require artificially provided image data labels, and can directly construct a data representation model from the image data set itself. Therefore, the above method of this embodiment is a self-supervised machine learning method.

[0105] This embodiment constructs a data representation model in a self-supervised learning manner based only on information of a large amount of image data sets themselves, and can achieve higher performance of the model by fine-tuning the data representation model by migrating it to a small amount of manually annotated data.

[0106] As an optional implementation, in step S306, after the prompt information of the training data representation model is displayed on the operation interface, the method also includes: displaying the recognition task of product information on the operation interface; displaying product information on the operation interface, wherein the product information is obtained by identifying the recognition task using the trained data representation model.

[0107] In this embodiment, the trained data representation model can be transferred to the product information recognition task, wherein the product information can be a text description content used to describe the product object, for example, commodity information, which can include but is not limited to the product title, product selling points and other information of the product object. After the prompt information of the training data representation model is displayed on the operation interface, the product information recognition task can be displayed on the operation interface, and then the task recognition operation instruction can be received and responded to on the operation interface, and the recognition task can be recognized using the trained data representation model.

[0108] Optionally, this embodiment copies the model parameters of the trained data characterization model to the model of the product information recognition task as initialization, and then continues to train the model of the product information recognition task. When the product object is a commodity, the model of the product information recognition task can be called a commodity recognition model. The recognition task is identified by the model to identify the above-mentioned product information used to describe the product object, and then the product information is displayed on the operation interface, so that the recognition accuracy of the product object's category, attribute, category, etc. is higher and the performance is more stable, thereby achieving the improvement of the accuracy of product information recognition, generalization performance and cross-domain capability.

[0109] As an optional implementation, after displaying the product information on the operation interface, the method also includes: popping up guidance information on the operation interface, wherein the guidance information includes defect information existing in the product information; displaying creative materials generated based on the guidance information on the operation interface, wherein the creative materials are basic information constituting the product materials; generating various types of product materials based on the creative materials; and publishing multiple product materials.

[0110] In this embodiment, after the product information used to describe the product object is displayed on the operation interface, guidance information may also pop up on the operation interface, and the guidance information may include defect information existing in the product information, and the defect information is used to indicate problems existing in the product information when generating product materials, and may be used to guide the generation of creative materials, wherein the creative materials are basic information constituting the product materials. This embodiment may generate creative materials based on the above guidance information, for example, check for omissions and fill in gaps based on the guidance information to generate creative materials, and then display the above creative materials on the operation interface.

[0111] After the creative materials generated based on the guidance information are displayed on the operation interface, various types of product materials can be generated based on the creative materials. The product materials are the materials needed when publishing product objects, and can be picture product materials, video product materials, text product materials, etc. Each type of product material can include the above-mentioned product information, and then publish multiple product materials.

[0112] The embodiment of the present invention also provides another method for processing product data from the perspective of human-computer interaction.

[0113] Figure 4 FIG. 1 is a flowchart of another method for processing product data according to an embodiment of the present invention. Figure 4 As shown, the method may include the following steps:

[0114] Step S402: displaying an image data set of the product object on an interactive interface, wherein the image data set includes: a plurality of pictures and a plurality of image frames of a video, and none of the images in the image data set are labeled.

[0115] In the technical solution provided in the above step S402 of the present invention, the image data set obtained by identifying the product object can be directly displayed on the interactive interface. The image data set can be data of multiple images obtained by shooting the product object, and can include multiple pictures of the product object and multiple image frames of the video.

[0116] It should be noted that the images in the above-mentioned image data set of this embodiment are not labeled, that is, the image data set can be large-scale (massive) unlabeled data, so the product data processing method of this embodiment is for the unlabeled image data set.

[0117] Step S404, sensing a model generation instruction in the interactive interface.

[0118] In the technical solution provided in the above step S404 of the present invention, after the image data set of the product object is displayed on the interactive interface, a model generation instruction can be sensed in the interactive interface. The model generation instruction is used to instruct the generation of a data representation model, and can be an instruction for generating a data representation model triggered by the user on the operation interface.

[0119] Step S406, in response to the model generation instruction, multiple images are selected from the image data set, and the multiple images are distributed to multiple processes for distributed calculation, wherein at least one process obtains the local gradient in the back propagation process, and the at least one local gradient obtained is aggregated to obtain an aggregated gradient.

[0120] In the technical solution provided in the above step S406 of the present invention, after the model generation instruction is sensed in the interactive interface, in response to the model generation instruction, multiple images are first selected from the image data set, for example, 1000 images are randomly extracted from the image data set for training. After selecting multiple images from the image data set, the selected multiple images are distributed to multiple processes for distributed calculation, wherein the multiple processes may be all current processes. At least one process may be used to calculate the local similarity matrix and the local loss, and optionally, at least one process may perform distributed calculation on multiple images in the image data set on the corresponding GPU. Optionally, in the process of back propagation, this embodiment returns the local gradient to each process, thereby achieving efficient training of the model.

[0121] Step S408, displaying prompt information of the training data characterization model on the interactive interface, wherein the training data characterization model is trained using aggregated gradients.

[0122] In the technical solution provided in the above step S408 of the present invention, after at least one process obtains the local gradient in the back-propagation process and aggregates at least one local gradient to obtain the aggregated gradient, a prompt information of the training data representation model can be displayed on the interactive interface. The prompt information can be used to indicate that the data representation model is being trained or that the training data representation model has been completed.

[0123] This embodiment uses aggregated gradients to learn (optimize) a data representation model with strong discriminativeness. Optionally, the data representation model of this embodiment can be used as a feature extractor and classifier, and can be transferred to the product recognition task to improve the accuracy and generalization performance of product object recognition.

[0124] It should be noted that the above method of this embodiment does not require artificially provided image data labels, and can directly construct a data representation model from the image data set itself. Therefore, the above method of this embodiment is a self-supervised machine learning method.

[0125] This embodiment constructs a data representation model in a self-supervised learning manner based only on information of a large amount of image data sets themselves, and can achieve higher performance of the model by fine-tuning the data representation model by migrating it to a small amount of manually annotated data.

[0126] The embodiment of the present invention also provides another method for processing product data from the perspective of human-computer interaction.

[0127] Figure 5 FIG. 1 is a flowchart of another method for processing product data according to an embodiment of the present invention. Figure 5 As shown, the method may include the following steps:

[0128] Step S502: The front-end client uploads an image data set of a product object, wherein the image data set includes: multiple pictures and multiple image frames of a video, and none of the images in the image data set are labeled.

[0129] In the technical solution provided in the above step S502 of the present invention, the front-end client can be a merchant publishing end, which can receive an upload operation instruction on the operation interface, respond to the upload operation instruction, and start uploading an image data set of a product object, wherein the product object can be a commodity object, and the image data set can be data of multiple images obtained by photographing the product object, and can include multiple pictures of the product object and multiple image frames of a video.

[0130] It should be noted that the images in the image data set of the product objects uploaded by the above-mentioned front-end client are not labeled, that is, the image data set can be large-scale (massive) unlabeled data, and thus the product data processing method of this embodiment is for the unlabeled image data set.

[0131] Step S504: the front-end client transmits the image data set to the back-end server.

[0132] In the technical solution provided in the above step S504 of the present invention, after the front-end client uploads the image data set of the product object, the front-end client can transmit the image data set of the product object to the back-end server.

[0133] In this embodiment, a communication connection is established between the front-end client and the back-end server, and an image data set of a product object can be transmitted to the back-end server, so that the back-end server generates a data representation model based on the image data set.

[0134] Step S506: the front-end client receives the trained data representation model returned by the back-end server.

[0135] In the technical solution provided in the above step S506 of the present invention, multiple images in the image data set are distributed by the background server to multiple processes for distributed calculation, at least one process obtains the local gradient in the back propagation process, aggregates at least one local gradient obtained to obtain the aggregated gradient, and uses the aggregated gradient to train the data characterization model.

[0136] In this embodiment, after the background server receives the image data set of the product object, the background server distributes multiple images in the image data set to multiple processes for distributed calculation, and at least one process obtains the local gradient in the back propagation process.

[0137] Optionally, the background server of this embodiment can be a distributed training scheme for parallel processes, distributing multiple images in the image data set to multiple processes for distributed calculation. At least one process can be used to calculate the local similarity matrix and local loss. Optionally, at least one process can perform distributed calculations on multiple images in the image data set on the corresponding GPU. Optionally, during the back propagation process of this embodiment, the background server returns the local gradient to each process, thereby achieving efficient training of the model.

[0138] After at least one process obtains the local gradient in the back-propagation process, the background server can aggregate the at least one local gradient obtained in the at least one process, thereby obtaining the aggregated gradient under the at least one process to complete the optimization of the model on each process.

[0139] After the backend server aggregates at least one local gradient obtained to obtain an aggregated gradient, the aggregated gradient is used to train the data representation model, for example, the aggregated gradient is used to learn (optimize) a data representation model with strong discriminability. Optionally, the data representation model of this embodiment can be used as a feature extractor and classifier. After the front-end server has trained the data representation model, the data representation model can be returned to the front-end client.

[0140] It should be noted that the above method of this embodiment does not require artificially provided image data labels, and can directly construct a data representation model from the image data set itself. Therefore, the above method of this embodiment is a self-supervised machine learning method.

[0141] This embodiment constructs a data representation model in a self-supervised learning manner based only on information of a large amount of image data sets themselves, and can fine-tune the data representation model by migrating it to a small amount of manually annotated data to achieve higher performance of the model.

[0142] In the related art, supervised learning methods and weakly supervised learning methods are used to model the structured modeling of product data. Among them, the supervised learning method is to manually label the product information of the goods, and then complete the structured modeling by learning the mapping relationship between the product information and the manual labels. However, the supervised learning method cannot fully utilize the larger scale of unlabeled data to improve the model accuracy, and it is difficult to cover complex and changing scenes, product types and shooting conditions; while the weakly supervised learning method is to extract effective information from the product description as a label, and construct it with the help of the mapping relationship between the product data and the description information provided externally, but the construction result is greatly affected by the noise information in the description text, and both have the technical problem of inaccurate structured modeling results of product data.

[0143] However, this embodiment is a complete product object recognition pre-training solution based on self-supervised learning algorithms, distributed training and transfer learning. It can effectively utilize massive unlabeled product object image data sets to build data representation models and migrate to product information recognition tasks. The solution aims to make full use of the massive product object image data sets of e-commerce platforms to complete structured modeling, alleviate cross-domain problems (reduced recognition rate for products that are significantly different from labeled data), and improve the accuracy of product object recognition to enhance the user experience of sellers and buyers.

[0144] Example 2

[0145] The preferred implementation of the above method of this embodiment is further introduced below, specifically taking the product object as a commodity as an example.

[0146] In e-commerce platforms, it is necessary to perform structured modeling on product data, and it is usually necessary to design algorithms to intelligently identify structured information such as product categories, attributes, categories, SPU, etc. Existing algorithms usually rely on manual labeling of product information, and complete structured modeling by learning the mapping relationship between product data and manual labels. Most of these algorithms are supervised learning methods, based on data-driven modeling methods represented by deep learning, which require intensive manual labeling and data review, and the data cost is high. Due to the high cost of manual labeling and the difficulty of scalability, this part of labeled data only accounts for a very small part of the total product data. Therefore, on the one hand, the above-mentioned supervised learning methods cannot make full use of larger-scale unlabeled data to improve model accuracy; on the other hand, manually labeled data usually only accounts for a very small part of the massive product data, and it is difficult to cover complex and changeable scenes, product types, shooting angles, shooting environments, etc. The models built on these data may have certain data deviations, and the performance may decline when applied to uncommon scenes, that is, facing the "cross-domain problem" in machine learning. Therefore, the above-mentioned algorithms cannot make full use of large-scale product data, and there are certain cross-domain problems (the recognition rate of products with large differences from labeled data is reduced).

[0147] One solution to the above problem is to use the product description information provided by merchants as weak supervision information to build a product model. For example, the product name, introduction text, etc. can be segmented and screened and used as labels for product data. This method can greatly reduce manual intervention, save costs, and facilitate scalability. However, there is a lot of text noise in the product description information, which will cause significant interference in the structured modeling process and greatly reduce the model effect. Therefore, the existing product data structured modeling process has the following limitations:

[0148] (1) Over-reliance on manual labeling and review, high data costs, and difficulty in scalability;

[0149] (2) The utilization rate of commodity data is low, and there is a lack of solutions to effectively utilize unlabeled data;

[0150] (3) There are cross-domain issues. When faced with scenes and product categories that do not appear in the labeled data, the recognition accuracy may be reduced.

[0151] Related technologies can extract effective information from product descriptions as labels, and build a supervised model with the help of the mapping relationship between product data and the description information provided by the seller. This type of method does not require additional manual labels and belongs to weakly supervised learning methods. However, this type of method is greatly affected by the noise information in the description text and the effect is not good. At present, there is no solution that can effectively use massive unlabeled product data to build a data representation model and migrate it to the product recognition task. Therefore, it is necessary to study self-supervised product recognition pre-training methods that effectively use massive unlabeled data.

[0152] However, this embodiment provides a complete set of product recognition pre-training solutions based on self-supervised learning algorithms, distributed training and transfer learning, mainly aiming to fully utilize the massive data of the e-commerce platform to complete structured modeling, alleviate cross-domain problems, and improve the accuracy of product recognition, so as to enhance the user experience of sellers and buyers. The method of this embodiment is further introduced below.

[0153] This embodiment learns a highly discriminative data representation model by minimizing the distance between different transformations of the same product image and maximizing the distance between different product images based on the noise contrast estimation loss function on large-scale unlabeled product image data, and migrates the representation model to the product recognition task to improve the accuracy and generalization performance of product recognition. The data representation model can be a feature extractor and a classifier.

[0154] During the training of the data representation model, the algorithm needs to aggregate all image features stored in different machines and processes, calculate the similarity matrix between them, and then calculate the loss value. This process requires a large amount of data transmission, calculation and storage.

[0155] Optionally, this embodiment randomly extracts 1,000 images from the image data set of the commodity for training, iteration, and calculation of the similarity matrix of the 1,000 images, which can be distributed to three GPUs to obtain image features, and then aggregated to one GPU to calculate the similarity matrix between image features, obtain the local similarity matrix generated under at least one process, perform local loss calculation for the local similarity matrix generated under at least one process, generate the local loss under at least one process, perform gradient calculation based on the local loss obtained by at least one process, and obtain the local gradient in the back propagation process. The frequency of gradient aggregation is determined based on the communication volume, calculation volume, and access times between at least one process (between GPUs).

[0156] Optionally, this embodiment evenly distributes the similarity and loss calculation tasks to all processes, at least one process is only used to calculate the local similarity matrix and local loss, and during the back propagation process, the gradient information is distributed back to each process, thereby achieving efficient training.

[0157] The training method of the data representation model of this embodiment may include the following steps:

[0158] S1, obtain the image features of each of the multiple images of the product.

[0159] S2, calculates the similarity matrix between image features (here it is necessary to obtain the image features of each image and distribute the image features of each image to different processes, that is, it is necessary to pull all image features from other GPUs).

[0160] S3, obtaining a local similarity matrix generated under at least one process according to the similarity matrix, using a noise contrast loss function to perform local loss calculation on the local similarity matrix generated under at least one process, and generating a local loss under at least one process.

[0161] S4: Perform gradient calculation based on the local loss obtained by at least one process to obtain the local gradient in the back propagation process (distribute and calculate the local gradient on each GPU).

[0162] S5, distribute the local gradient of any process to other processes (distribute to each GPU), and at least one process aggregates the local gradient in the current process and the local gradient distributed by other processes to generate an aggregated gradient of at least one process. The local gradients on multiple GPUs can be averaged to process data on a large scale.

[0163] S6, using aggregated gradients to train the data representation model.

[0164] Repeat the above steps S1 to S6.

[0165] In this embodiment, the noise contrast loss function can be expressed by the following formula:

[0166]

[0167] Among them, i and j are used to represent two transformations of the same product image (for example, scaling, cropping, color change, etc., where various transformations do not change the meaning of the image), and k is used to represent images of other products.

[0168] Figure 6 is a schematic diagram of a noise contrast loss function according to an embodiment of the present invention. Figure 6 As shown, taking product image A and product image B as examples, product image A undergoes transformation 1, and product image A undergoes transformation 2. Image features are extracted from product image A after transformation 1 through a convolutional neural network, image features are extracted from product image A after transformation 2 through a convolutional neural network, image features are extracted from product image B after transformation 1 through a convolutional neural network, and image features are extracted from product image B after transformation 2 through a convolutional neural network. The noise contrast loss function of this embodiment requires that the distance between image features of the same image between different transformations be as close as possible, while the distance between image features of different images be as far apart as possible. Among them, Figure 6The solid line is used to indicate the distance between two sets of image features is shortened, and the dotted line is used to indicate the distance between two sets of image features is extended.

[0169] Fig. 7A FIG. 1 is a schematic diagram of a distributed training process according to an embodiment of the present invention. Fig. 7A As shown, there are multiple processes, including process 1, process 2...process N, and multiple images in the image data set are distributed to multiple processes for distributed calculation. A neural network model can be used in at least one process to identify the image features of each image in the image data set, calculate the similarity matrix between the image features, and then obtain the local similarity matrix generated in at least one process. In the process of calculating the local similarity matrix in any process, the image features generated in other processes are pulled, and the pulled image features are used to calculate the similarity with the image features in the current process to generate the local similarity matrix in the current process.

[0170] This embodiment performs local loss calculation on the local similarity matrix generated under at least one process to generate local loss under at least one process. This embodiment can distribute the local gradient under any process to other processes; at least one process uses the local gradient in the current process and the local gradient distributed from other processes to aggregate and generate an aggregated gradient under at least one process, thereby completing the optimization purpose of the data representation model on at least one process.

[0171] Figure 7B is a schematic diagram of an interactive interface of a method for processing product data according to an embodiment of the present invention. Figure 7B As shown, the user can enter the image data set of the product object on the operation interface, wherein the image data set includes at least one of the following: multiple pictures B and multiple image frames P of the video, and the images in the image data set are not labeled. By clicking the "Model Training" button, the multiple images in the image data set are distributed to multiple processes for distributed calculation, at least one process obtains the local gradient in the back propagation process, and the at least one local gradient obtained is aggregated to obtain the aggregated gradient, and the aggregated gradient is used to train the data characterization model, and then the prompt information of the training data characterization model is displayed on the operation interface.

[0172] Figure 7C FIG. 1 is a schematic diagram of a method for processing product data according to an embodiment of the present invention. Figure 7CAs shown, the computing device obtains an image data set of a product object, wherein the image data set includes: multiple images and multiple image frames of a video, and the images in the image data set are not labeled, and the product data of the above product object can be displayed on the interactive interface of the computing device. Then, a model generation instruction is sensed in the interactive interface of the computing device, and in response to the model generation instruction, multiple images are selected from the image data set, and the multiple images are distributed to multiple processes for distributed calculation, wherein at least one process obtains the local gradient in the back propagation process, and the obtained at least one local gradient is aggregated to obtain an aggregated gradient, and then the prompt information of the training data representation model is displayed on the interactive interface of the computing device, wherein the training data representation model is trained using the aggregated gradient.

[0173] Compared with the solution of "building a supervised model with the mapping relationship between product data and description information provided by sellers" in the related art, the method of self-supervised product identification pre-training in this embodiment is not affected by the large amount of noise contained in the description information of the product. Instead, it builds a pre-trained model based on the information of a large number of product images themselves in a self-supervised learning manner, and achieves higher performance by migrating the pre-trained model to a small amount of manually annotated data for fine-tuning. In terms of technical effects, the model after migration in this embodiment has higher accuracy in identifying product categories, attributes, categories, etc., and more stable performance.

[0174] This embodiment proposes a complete set of commodity recognition pre-training solutions based on self-supervised learning algorithms, distributed training and transfer learning, which can use a large amount of unlabeled data to complete model pre-training and transfer to commodity recognition tasks to improve recognition accuracy and cross-domain capabilities. Specifically, it includes the following aspects:

[0175] (1) A complete set of product recognition pre-training solutions, including self-supervised learning, distributed training, and transfer learning.

[0176] (2) The image self-supervision method based on noise contrast estimation loss gradually shortens the distance between image features of different transformations of the same image and gradually increases the distance between image features of different images during the training process, which can effectively learn a data representation model with strong discriminative ability.

[0177] (3) Based on the distributed training scheme of parallel processes, the similarity matrix calculation and loss calculation are evenly distributed to each process. After completing the local loss calculation in each process, the gradient is aggregated to complete the optimization of the data representation model on at least one process, thereby improving the training efficiency of the data representation model.

[0178] (4) Transfer learning scheme: After completing the self-supervised training on unlabeled data, the parameters of the trained data representation model can be copied to the model of the product recognition task as initialization, and the product recognition model can be further trained. This transfer learning improves the accuracy and cross-domain capability of product recognition.

[0179] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0180] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0181] Example 3

[0182] According to an embodiment of the present invention, there is also provided a method for implementing the above Figure 2 The product data processing method shown is a product data processing device.

[0183] Figure 8 FIG. 1 is a schematic diagram of a device for processing product data according to an embodiment of the present invention. Figure 8 As shown, the product data processing device 80 may include: an acquisition unit 81 , a distribution unit 82 , an aggregation unit 83 and a training unit 84 .

[0184] The acquisition unit 81 is used to acquire an image data set of a product object, wherein the image data set includes: multiple pictures and multiple image frames of a video, and the images in the image data set are not labeled.

[0185] The distribution unit 82 is used to distribute multiple images in the image data set to multiple processes for distributed calculation, and at least one process obtains the local gradient in the back propagation process.

[0186] The aggregation unit 83 is used to aggregate the acquired at least one local gradient to obtain an aggregated gradient.

[0187] The training unit 84 is used to train the data representation model using the aggregated gradient.

[0188] It should be noted that the acquisition unit 81, distribution unit 82, aggregation unit 83 and training unit 84 correspond to steps 202 to S208 in Example 1, and the four units and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned units, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0189] According to an embodiment of the present invention, another method for implementing the above Figure 3 The product data processing method shown is a product data processing device.

[0190] Fig. 9 FIG. 1 is a schematic diagram of another device for processing product data according to an embodiment of the present invention. Fig. 9 As shown, the product data processing device 90 may include: an input unit 91 , a first sensing unit 92 and a first display unit 93 .

[0191] The input unit 91 is used to input an image data set of a product object on an input page of an operation interface, wherein the image data set includes: multiple pictures and multiple image frames of a video, and none of the images in the image data set are labeled.

[0192] The first sensing unit 92 is used to sense the model generation instruction in the operation interface, distribute multiple images in the image data set to multiple processes for distributed calculation, wherein at least one process obtains the local gradient in the back propagation process, and aggregates the at least one local gradient obtained to obtain an aggregated gradient.

[0193] The first display unit 93 is used to display prompt information of the training data representation model on the operation interface, wherein the training data representation model is trained using aggregated gradients.

[0194] It should be noted that the above-mentioned input unit 91, the first sensing unit 92 and the first display unit 93 correspond to steps 302 to S306 in Example 1, and the three units and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned units, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0195] According to an embodiment of the present invention, another method for implementing the above Figure 4The product data processing method shown is a product data processing device.

[0196] Fig.10 FIG. 1 is a schematic diagram of another device for processing product data according to an embodiment of the present invention. Fig.10 As shown, the product data processing device 100 may include: a second display unit 101 , a second sensing unit 102 , a response unit 103 and a third display unit 104 .

[0197] The second display unit 101 is used to display an image data set of a product object on an interactive interface, wherein the image data set includes: multiple pictures and multiple image frames of a video, and the images in the image data set are not labeled.

[0198] The second sensing unit 102 is used to sense the model generation instruction in the interactive interface.

[0199] The response unit 103 is used to respond to the model generation instruction, select multiple images from the image data set, and distribute the multiple images to multiple processes for distributed calculation, wherein at least one process obtains the local gradient in the back propagation process, and aggregates the at least one local gradient obtained to obtain an aggregated gradient.

[0200] The third display unit 104 is used to display prompt information of the training data representation model on the interactive interface, wherein the training data representation model is trained using aggregated gradients.

[0201] It should be noted that the second display unit 101, the second sensing unit 102, the response unit 103 and the third display unit 104 correspond to steps 402 to S408 in Embodiment 1, and the four units and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in Embodiment 1. It should be noted that the above units, as part of the device, can be run in the computer terminal 10 provided in Embodiment 1.

[0202] According to an embodiment of the present invention, another method for implementing the above Figure 5 The product data processing method shown is a product data processing device.

[0203] Fig.11 FIG. 1 is a schematic diagram of another device for processing product data according to an embodiment of the present invention. Fig.11 As shown, the product data processing device 110 may include: an uploading unit 111 , a transmission unit 112 and a receiving unit 113 .

[0204] The uploading unit 111 is used to enable the front-end client to upload an image data set of a product object, wherein the image data set includes: multiple pictures and multiple image frames of a video, and the images in the image data set are not labeled.

[0205] The transmission unit 112 is used to enable the front-end client to transmit the image data set to the back-end server.

[0206] The receiving unit 113 is used to enable the front-end client to receive the trained data representation model returned by the background server, wherein multiple images in the image data set are distributed by the background server to multiple processes for distributed calculation, at least one process obtains the local gradient in the back propagation process, aggregates at least one local gradient obtained to obtain an aggregated gradient, and uses the aggregated gradient to train the data representation model.

[0207] In the product data processing device of this embodiment, a supervised machine learning method is used, and distributed training can be performed only based on the image data set of unlabeled product objects, so as to effectively construct a data representation model, which is completely unaffected by other large amounts of noise. This avoids the inability of supervised learning methods to fully utilize larger-scale unlabeled data to improve model accuracy, and also avoids the possibility that the model constructed on the data may have certain data deviations due to the difficulty of a small amount of labeled data to cover complex and changeable scenes, product types and shooting conditions. This solves the technical problem of inaccurate structured modeling results of product data and achieves the technical effect of improving the accuracy of the structured modeling results of product data.

[0208] Example 4

[0209] The embodiment of the present invention can provide a product data processing system, which can include a computer terminal, which can be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal can also be replaced by a terminal device such as a mobile terminal.

[0210] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of the computer network.

[0211] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the product data processing method of the application: obtaining an image data set of the product object, wherein the image data set includes: multiple pictures and multiple image frames of videos, and the images in the image data set are not labeled; distributing the multiple images in the image data set to multiple processes for distributed calculation, and at least one process obtains the local gradient in the back propagation process; aggregating at least one local gradient obtained to obtain an aggregated gradient; and using the aggregated gradient to train the data representation model.

[0212] Optionally, Fig.12 is a structural block diagram of a computer terminal according to an embodiment of the present invention. Fig.12 As shown, the computer terminal A may include: one or more (only one is shown in the figure) processors 1202 , a memory 1204 , and a transmission device 1206 .

[0213] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the method and device for processing product data in the embodiment of the present invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned method for processing product data. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal A via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0214] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain an image data set of the product object, wherein the image data set includes: multiple pictures and multiple image frames of videos, and the images in the image data set are not labeled; distribute the multiple images in the image data set to multiple processes for distributed calculation, and at least one process obtains the local gradient in the back propagation process; aggregate the at least one local gradient obtained to obtain an aggregated gradient; and use the aggregated gradient to train the data characterization model.

[0215] Optionally, the processor may also execute program code of the following steps: obtaining image features of each image in an image data set, and distributing the image features of each image to different processes; calculating a similarity matrix between image features, and obtaining a local similarity matrix generated under at least one process; performing local loss calculation on the local similarity matrix generated under at least one process, and generating a local loss under at least one process; performing gradient calculation based on the local loss obtained by at least one process, and obtaining a local gradient in the back propagation process.

[0216] Optionally, the processor may also execute program code of the following steps: using a neural network model to identify image features of each image in the image data set, and extracting image features of each image.

[0217] Optionally, the processor may also execute the following program code: in the process of calculating the local similarity matrix in any process, image features generated in other processes are pulled; similarity is calculated using the pulled image features and image features in the current process to generate a local similarity matrix for the current process.

[0218] Optionally, the processor may also execute the program code of the following steps: using a noise contrast loss function to perform local loss calculation on a local similarity matrix generated under at least one process, and generating a local loss under at least one process, wherein the noise contrast loss function is used to reduce the image feature distance between different transformations in the same image, thereby increasing the image feature distance between different images.

[0219] Optionally, the processor may also execute program code of the following steps: distributing the local gradient of any process to other processes; at least one process aggregates the local gradient in the current process and the local gradient distributed from other processes to generate an aggregated gradient for at least one process.

[0220] Optionally, the processor may further execute program code of the following steps: determining a frequency of gradient aggregation based on a communication volume, a computation volume, and a number of accesses between at least one process.

[0221] Optionally, the processor may also execute program code of the following steps: after using aggregated gradients to train a data representation model, obtaining a task of identifying product information; and using the trained data representation model to identify the recognition task to obtain product information.

[0222] Optionally, the processor may also execute program codes of the following steps: after using the trained data representation model to identify product information, generating multiple types of product materials based on the product information; and publishing multiple product materials.

[0223] Optionally, the processor may also execute program code for the following steps: after generating multiple types of product materials, uploading the product materials to be released, and extracting multiple product contents to be verified in the product materials to be released; determining whether at least one product content to be verified meets the entry criteria; if so, successfully entering the product material into the release template; otherwise, preprocessing the product content that failed the verification, and if the preprocessed product content meets the entry criteria, entering the product material into the release template.

[0224] As an optional implementation, the processor can call the information and application stored in the memory through the transmission device to perform the following steps: enter the image data set of the product object on the entry page on the operation interface, wherein the image data set includes: multiple pictures and multiple image frames of videos, and the images in the image data set are not labeled; sense the model generation instruction in the operation interface, distribute the multiple images in the image data set to multiple processes for distributed calculation, wherein at least one process obtains the local gradient in the back propagation process, aggregates the at least one local gradient obtained, and obtains the aggregated gradient; display the prompt information of the training data characterization model on the operation interface, wherein the training data characterization model is trained using the aggregated gradient.

[0225] Optionally, the processor may also execute the program code of the following steps: after displaying the prompt information of the training data representation model on the operation interface, displaying the recognition task of the product information on the operation interface; displaying the product information on the operation interface, wherein the product information is obtained by identifying the recognition task using the trained data representation model.

[0226] Optionally, the processor may also execute the program code of the following steps: after displaying product information on the operation interface, popping up guidance information on the operation interface, wherein the guidance information includes defect information existing in the product information; displaying creative materials generated based on the guidance information on the operation interface, wherein the creative materials are basic information constituting the product materials; generating various types of product materials based on the creative materials; and publishing multiple product materials.

[0227] As an optional implementation, the processor can call the information and application stored in the memory through the transmission device to perform the following steps: displaying an image data set of the product object on the interactive interface, wherein the image data set includes: multiple pictures and multiple image frames of videos, and the images in the image data set are not labeled; sensing a model generation instruction in the interactive interface; in response to the model generation instruction, selecting multiple images from the image data set, and distributing the multiple images to multiple processes for distributed calculation, wherein at least one process obtains the local gradient in the back propagation process, and aggregates at least one local gradient obtained to obtain an aggregated gradient; displaying prompt information of the training data characterization model on the interactive interface, wherein the training data characterization model is trained using aggregated gradients.

[0228] As an optional implementation, the processor can call the information and application stored in the memory through the transmission device to perform the following steps: the front-end client uploads an image data set of the product object, wherein the image data set includes: multiple pictures and multiple image frames of videos, and the images in the image data set are not labeled; the front-end client transmits the image data set to the back-end server; the front-end client receives the trained data representation model returned by the back-end server, wherein the multiple images in the image data set are distributed by the back-end server to multiple processes for distributed calculation, at least one process obtains the local gradient in the back-propagation process, aggregates the at least one local gradient obtained to obtain the aggregated gradient, and uses the aggregated gradient to train the data representation model.

[0229] According to an embodiment of the present invention, a method for processing product data is provided. An image data set of a product object is obtained, wherein the image data set includes: multiple images and multiple image frames of a video, and the images in the image data set are not labeled; the multiple images in the image data set are distributed to multiple processes for distributed calculation, and at least one process obtains the local gradient in the back propagation process; the at least one local gradient obtained is aggregated to obtain an aggregated gradient; and the aggregated gradient is used to train a data representation model. That is, using a supervised machine learning method, it is possible to perform distributed training on the unlabeled image data set of the product object only, thereby effectively constructing a data representation model, which is completely unaffected by a large amount of other noise, avoiding the inability of the supervised learning method to fully utilize a larger scale of unlabeled data to improve the model accuracy, and also avoiding the fact that a small amount of labeled data is difficult to cover complex and changeable scenes, product types and shooting conditions, and the model constructed on these data may have certain data deviations, thereby solving the technical problem of inaccurate structured modeling results of product data, and achieving the technical effect of improving the accuracy of structured modeling results of product data.

[0230] It can be understood by those skilled in the art that Fig.12 The structure shown is for illustration only, and the computer terminal A may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a handheld computer, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Fig.12 It does not limit the structure of the above-mentioned computer terminal A. For example, the computer terminal A may also include Fig.12 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Fig.12 Different configurations shown.

[0231] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0232] Example 5

[0233] The embodiment of the present invention further provides a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store the program code executed by the product data processing method provided in the first embodiment.

[0234] Optionally, in this embodiment, the computer-readable storage medium may be located in any one of the computer terminals in a computer terminal group in a computer network, or in any one of the mobile terminals in a mobile terminal group.

[0235] Optionally, in this embodiment, a computer-readable storage medium is configured to store program code for executing the following steps: obtaining an image dataset of a product object, wherein the image dataset includes: multiple pictures and multiple image frames of a video, and none of the images in the image dataset are labeled; distributing the multiple images in the image dataset to multiple processes for distributed calculations, and at least one process obtains local gradients in the back propagation process; aggregating at least one local gradient obtained to obtain an aggregated gradient; and using the aggregated gradient to train a data characterization model.

[0236] Optionally, the computer-readable storage medium is also configured to store program code for performing the following steps: obtaining image features of each image in an image data set, and distributing the image features of each image to different processes; calculating a similarity matrix between image features, and obtaining a local similarity matrix generated under at least one process; performing local loss calculation on the local similarity matrix generated under at least one process, and generating a local loss under at least one process; performing gradient calculation based on the local loss obtained by at least one process, and obtaining a local gradient in the back propagation process.

[0237] Optionally, the computer-readable storage medium is further configured to store program code for performing the following steps: using a neural network model to identify image features of each image in the image data set, and extracting image features of each image.

[0238] Optionally, the computer-readable storage medium is also configured to store program code for executing the following steps: in the process of calculating a local similarity matrix in any process, image features generated in other processes are pulled; similarity is calculated using the pulled image features and image features in the current process to generate a local similarity matrix for the current process.

[0239] Optionally, the computer-readable storage medium is also configured to store program code for executing the following steps: using a noise contrast loss function to perform local loss calculation on a local similarity matrix generated under at least one process, and generating a local loss under at least one process, wherein the noise contrast loss function is used to reduce the image feature distance between different transformations in the same image, so that the image feature distance between different images is expanded.

[0240] Optionally, the computer-readable storage medium is also configured to store program codes for executing the following steps: distributing the local gradient of any process to other processes; and aggregating the local gradient in the current process and the local gradient distributed from other processes to generate an aggregated gradient for at least one process.

[0241] Optionally, the computer-readable storage medium is further configured to store program code for executing the following step: determining a frequency of gradient aggregation based on a communication volume, a computational volume, and an access number between at least one process.

[0242] Optionally, the computer-readable storage medium is also configured to store program code for performing the following steps: after using aggregated gradients to train a data characterization model, obtaining a task of identifying product information; and using the trained data characterization model to identify the identification task to obtain product information.

[0243] Optionally, the computer-readable storage medium is also configured to store program code for executing the following steps: after using the trained data characterization model to identify product information, the method further includes: generating multiple types of product materials based on the product information; and publishing multiple product materials.

[0244] Optionally, the computer-readable storage medium is also configured to store program code for executing the following steps: after generating multiple types of product materials, uploading the product materials to be released, and extracting multiple product contents to be verified in the product materials to be released; determining whether at least one product content to be verified meets the entry criteria; if so, successfully entering the product material into the release template; otherwise, preprocessing the product content that failed the verification, and if the preprocessed product content meets the entry criteria, entering the product material into the release template.

[0245] As an optional example, a computer-readable storage medium is configured to store program code for executing the following steps: entering an image dataset of a product object on an entry page on an operation interface, wherein the image dataset includes: multiple pictures and multiple image frames of a video, and none of the images in the image dataset are labeled; sensing a model generation instruction in the operation interface, distributing multiple images in the image dataset to multiple processes for distributed calculations, wherein at least one process obtains local gradients in the back propagation process, and aggregating at least one local gradient obtained to obtain an aggregated gradient; and displaying prompt information of a training data characterization model on the operation interface, wherein the training data characterization model is trained using aggregated gradients.

[0246] Optionally, the computer-readable storage medium is also configured to store program code for executing the following steps: after displaying the prompt information of the training data characterization model on the operation interface, displaying the recognition task of the product information on the operation interface; displaying the product information on the operation interface, wherein the product information is obtained by identifying the recognition task using the trained data characterization model.

[0247] Optionally, the computer-readable storage medium is also configured to store program codes for executing the following steps: after displaying product information on the operation interface, popping up guidance information on the operation interface, wherein the guidance information includes defect information existing in the product information; displaying creative materials generated based on the guidance information on the operation interface, wherein the creative materials are basic information constituting the product materials; generating various types of product materials based on the creative materials; and publishing multiple product materials.

[0248] As an optional example, a computer-readable storage medium is configured to store program code for performing the following steps: displaying an image dataset of a product object on an interactive interface, wherein the image dataset includes: multiple pictures and multiple image frames of a video, and none of the images in the image dataset are labeled; sensing a model generation instruction in the interactive interface; in response to the model generation instruction, selecting multiple images from the image dataset, and distributing the multiple images to multiple processes for distributed calculation, wherein at least one process obtains local gradients in the back propagation process, and aggregates at least one local gradient obtained to obtain an aggregated gradient; and displaying prompt information of a training data characterization model on the interactive interface, wherein the training data characterization model is trained using aggregated gradients.

[0249] As an optional example, a computer-readable storage medium is configured to store program code for executing the following steps: a front-end client uploads an image dataset of a product object, wherein the image dataset includes: multiple pictures and multiple image frames of a video, and none of the images in the image dataset are labeled; the front-end client transmits the image dataset to a back-end server; the front-end client receives a trained data representation model returned by the back-end server, wherein multiple images in the image dataset are distributed by the back-end server to multiple processes for distributed calculation, at least one process obtains local gradients in the back-propagation process, aggregates at least one local gradient obtained to obtain an aggregated gradient, and uses the aggregated gradient to train the data representation model.

[0250] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0251] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0252] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0253] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0254] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0255] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0256] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for processing product data, characterized in that: include: Acquire an image data set of a product object, wherein the image data set includes: a plurality of images and a plurality of image frames of a video, the image data set includes the image frames obtained by photographing the product object, and the image frames are not labeled; Obtaining image features of each image in the image data set, and distributing the image features of each image to different processes; calculating a similarity matrix between the image features, and obtaining a local similarity matrix generated under the at least one process; performing local loss calculation on the local similarity matrix generated under the at least one process, and generating a local loss under the at least one process, so that the at least one process can obtain a local gradient in the back propagation process; Aggregating at least one local gradient obtained to obtain an aggregated gradient; The aggregated gradients are used to train a data representation model.

2. The method according to claim 1, characterized in that Based on the local loss of the at least one process, so that the at least one process obtains the local gradient in the back propagation process, including: Based on the local loss obtained by the at least one process, the gradient calculation is performed to obtain the local gradient in the back propagation process.

3. The method according to claim 1, characterized in that Obtaining image features of each image in the image dataset, including: A neural network model is used to identify image features of each image in the image dataset, and the image features of each image are extracted.

4. The method according to claim 1, characterized in that: Calculating the similarity matrix between the image features and obtaining the local similarity matrix generated in the at least one process includes: When any process calculates the local similarity matrix, it pulls the image features generated by other processes; The similarity between the pulled image features and the image features in the current process is calculated to generate a local similarity matrix for the current process.

5. The method according to claim 1, characterized in that The local similarity matrix generated under the at least one process is subjected to the local loss calculation using a noise contrast loss function to generate a local loss under the at least one process, wherein the noise contrast loss function is used to reduce the image feature distance between different transformations in the same image so as to increase the image feature distance between different images.

6. The method according to any one of claims 1 to 5, characterized in that Aggregating the at least one local gradient obtained to obtain an aggregated gradient includes: Distribute the local gradient of any process to other processes; The at least one process aggregates the local gradient in the current process and the local gradients distributed from other processes to generate the aggregated gradient under the at least one process.

7. The method according to claim 6, characterized in that The frequency of gradient aggregation is determined based on the communication volume, the computation volume, and the access times between the at least one process.

8. The method according to any one of claims 1 to 5, characterized in that After using the aggregated gradient to train a data representation model, the method further includes: Recognition tasks to obtain product information; The trained data representation model is used to identify the identification task to obtain the product information.

9. The method according to claim 8, characterized in that After using the trained data representation model to identify the product information, the method further includes: Based on the product information, generate various types of product materials; Publish a plurality of said product materials.

10. The method according to claim 9, characterized in that After generating multiple types of product materials, the method further includes: Uploading the product material to be released, and extracting multiple product contents to be verified from the product material to be released; Determining whether the at least one product content to be verified meets the entry criteria; If satisfied, the product material is successfully entered into the publishing template; Otherwise, the product content that fails the verification is preprocessed, and if the preprocessed product content meets the entry standard, the product material is entered into the publishing template.

11. A method for processing product data, characterized in that: include: Entering an image data set of a product object on an input page of an operation interface, wherein the image data set includes: multiple images and multiple image frames of a video, the image data set includes the image frames obtained by photographing the product object, and the image frames are not labeled; A model generation instruction is sensed in the operation interface, image features of each image in the image data set are obtained, and the image features of each image are distributed to different processes; a similarity matrix between the image features is calculated to obtain a local similarity matrix generated under the at least one process; a local loss calculation is performed on the local similarity matrix generated under the at least one process to generate a local loss under the at least one process, so that the at least one process can obtain a local gradient in the back propagation process, and the obtained at least one local gradient is aggregated to obtain an aggregated gradient; Prompt information of a training data characterization model is displayed on the operation interface, wherein the training data characterization model is trained using the aggregated gradient.

12. The method according to claim 11, characterized in that After displaying prompt information of the training data characterization model on the operation interface, the method further includes: Displaying product information on the operation interface; The product information is displayed on the operation interface, wherein the product information is obtained by using the trained representation model to identify the recognition task.

13. The method according to claim 12, characterized in that After displaying the product information on the operation interface, the method further includes: popping up guidance information on the operation interface, wherein the guidance information includes defect information existing in the product information; Displaying the creative materials generated based on the guidance information on the operation interface, wherein the creative materials are basic information constituting product materials; Based on the creative materials, generate various types of product materials; Publish a plurality of said product materials.

14. A method for processing product data, characterized in that: include: Displaying an image data set of a product object on an interactive interface, wherein the image data set includes: a plurality of images and a plurality of image frames of a video, the image data set includes the image frames obtained by photographing the product object, and the image frames are not labeled; Sensing a model generation instruction in the interactive interface; In response to the model generation instruction, multiple images are selected from the image data set, and image features of each image in the image data set are obtained, and the image features of each image are distributed to different processes; a similarity matrix between the image features is calculated to obtain a local similarity matrix generated under the at least one process; a local loss calculation is performed on the local similarity matrix generated under the at least one process to generate a local loss under the at least one process, so that the at least one process can obtain a local gradient in the back propagation process, and the obtained at least one local gradient is aggregated to obtain an aggregated gradient; Prompt information of a training data characterization model is displayed on the interactive interface, wherein the training data characterization model is trained using the aggregated gradient.

15. A method for processing product data, characterized in that: include: The front-end client uploads an image data set of a product object, wherein the image data set includes: multiple images and multiple image frames of a video, the image data set includes the image frames obtained by photographing the product object, and the image frames are not labeled; The front-end client transmits the image data set to the back-end server; The front-end client receives the trained data representation model returned by the back-end server, wherein image features of each image in the image data set are obtained, and the image features of each image are distributed to different processes; a similarity matrix between the image features is calculated to obtain a local similarity matrix generated under the at least one process; a local loss calculation is performed on the local similarity matrix generated under the at least one process to generate a local loss under the at least one process, so that the at least one process can obtain the local gradient in the back-propagation process, the at least one local gradient obtained is aggregated to obtain an aggregated gradient, and the aggregated gradient is used to train the data representation model.

16. A device for processing product data, characterized in that: include: An acquisition unit, configured to acquire an image data set of a product object, wherein the image data set includes: a plurality of images and a plurality of image frames of a video, the image data set includes the image frames obtained by photographing the product object, and the image frames are not labeled; A distribution unit, configured to obtain image features of each image in the image data set, and distribute the image features of each image to different processes; calculate a similarity matrix between the image features, and obtain a local similarity matrix generated under the at least one process; perform local loss calculation on the local similarity matrix generated under the at least one process, and generate a local loss under the at least one process, so that the at least one process can obtain a local gradient in the back propagation process; An aggregation unit, used for aggregating the acquired at least one local gradient to obtain an aggregated gradient; A training unit is used to train a data representation model using the aggregated gradient.

17. A product data processing device, characterized in that: include: An input unit, used to input an image data set of a product object on an input page on an operation interface, wherein the image data set includes: a plurality of images and a plurality of image frames of a video, the image data set includes the image frames obtained by photographing the product object, and the image frames are not labeled; A first sensing unit is used to sense a model generation instruction in the operation interface, obtain image features of each image in the image data set, and distribute the image features of each image to different processes; calculate a similarity matrix between the image features to obtain a local similarity matrix generated under the at least one process; perform local loss calculation on the local similarity matrix generated under the at least one process to generate a local loss under the at least one process, so that the at least one process can obtain a local gradient in the back propagation process, and aggregate the at least one local gradient obtained to obtain an aggregated gradient; The first display unit is used to display prompt information of the training data representation model on the operation interface, wherein the training data representation model is trained using the aggregated gradient.

18. A product data processing device, characterized in that: include: A second display unit is used to display an image data set of a product object on an interactive interface, wherein the image data set includes: a plurality of images and a plurality of image frames of a video, the image data set includes the image frames obtained by photographing the product object, and the image frames are not labeled; A second sensing unit, used to sense a model generation instruction in the interactive interface; A response unit is used to respond to the model generation instruction, obtain the image features of each image in the image data set, and distribute the image features of each image to different processes; calculate the similarity matrix between the image features, and obtain the local similarity matrix generated under the at least one process; perform local loss calculation on the local similarity matrix generated under the at least one process, and generate the local loss under the at least one process, so that the at least one process can obtain the local gradient in the back propagation process, and aggregate the at least one local gradient obtained to obtain an aggregated gradient; A third display unit is used to display prompt information of the training data representation model on the interactive interface, wherein the training data representation model is trained using the aggregated gradient.

19. A product data processing device, characterized in that: include: An uploading unit, configured to enable a front-end client to upload an image data set of a product object, wherein the image data set includes: a plurality of images and a plurality of image frames of a video, the image data set includes the image frames obtained by photographing the product object, and the image frames are not labeled; A transmission unit, used to enable the front-end client to transmit the image data set to a back-end server; A receiving unit is used to enable the front-end client to receive the trained data representation model returned by the back-end server, wherein the image features of each image in the image data set are obtained, and the image features of each image are distributed to different processes; the similarity matrix between the image features is calculated to obtain the local similarity matrix generated under the at least one process; the local loss calculation is performed on the local similarity matrix generated under the at least one process to generate the local loss under the at least one process, so that the at least one process can obtain the local gradient in the back-propagation process, the at least one local gradient obtained is aggregated to obtain the aggregated gradient, and the aggregated gradient is used to train the data representation model.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed by a processor, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 15.

21. A processor, characterized in that: The processor is used to run a program, wherein the program executes the method according to any one of claims 1 to 15 when running.

22. A product data processing system, characterized in that: include: processor; A memory is connected to the processor and is used to provide the processor with instructions for processing the following processing steps: obtaining an image data set of a product object, wherein the image data set includes: multiple image frames of multiple pictures and videos, and the image data set includes the image frames obtained by shooting the product object, and the image frames are not labeled; obtaining image features of each image in the image data set, and distributing the image features of each image to different processes; calculating a similarity matrix between the image features, and obtaining a local similarity matrix generated under the at least one process; performing local loss calculation on the local similarity matrix generated under the at least one process, and generating a local loss under the at least one process, so that the at least one process obtains a local gradient in the back propagation process; aggregating the at least one local gradient obtained to obtain an aggregated gradient; and using the aggregated gradient to train a data representation model.

Images