Image-text model training method, commodity repetition judgment method, equipment and medium
Through the graphic and text model training method, combined with the double tower and single tower models, the text and picture information of the product are extracted, which solves the problems of low efficiency and poor accuracy of traditional product weight judgment methods, and achieves fast and accurate product weight judgments.
Patent Information
- Application Number
- CN202310640909.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-01
- Publication Date
- 2024-12-03
AI Technical Summary
The traditional product judging method relies on factors such as title similarity, image similarity and word co-occurrence, which leads to high time consumption and poor results, and it is impossible to effectively identify product duplication in user-generated content.
Using the graphic and text model training method, by obtaining text and picture data related to the product, extracting named entity information and picture information, using a combination of double tower and single tower models to train the graphic and text model, calculate the similarity of the product, thereby realizing the product Quick and accurate judgment.
It significantly improves the efficiency and accuracy of product quota, can quickly identify duplicate products in the product library, reduces the time to judge and improves the accuracy of results.
Smart Images

Figure CN119068280A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a graphic model training method, a commodity weight determination method, equipment and medium. Background Art
[0002] Users will publish and share user-generated content (UGC) with other users on the Internet, and the content may be about a certain product.
[0003] In order to identify the products that appear in these user-generated contents, it is necessary to build a product library first. An important technology required for building and maintaining a product library is to determine whether two products are repeated, and if so, merge the two products.
[0004] Traditional methods for detecting duplicate products mainly rely on the similarity of product titles and images, as well as the similarity and co-occurrence of words after word segmentation, combined with post-processing strategies to determine whether two products are identical. However, this method takes a long time and the effect of detecting duplicate products is not good. Summary of the invention
[0005] In order to solve the above problems, the present application provides a graphic model training method, a commodity weight determination method, a device and a medium.
[0006] The first aspect of the present application discloses a method for training a graphic model for an electronic device, the method comprising:
[0007] Acquire training data, wherein the training data includes text and images related to a plurality of commodities;
[0008] For each product, extract the named entity information of the product from the text related to the product, and extract the image information of the product from the image related to the product;
[0009] Using the texts, pictures, named entity information and picture information of the multiple commodities to train a pre-built model to obtain the trained picture-text model,
[0010] The pre-built model includes a first model and a second model, and the text, the picture, the named entity information and the picture information of two of the multiple products are respectively input into the first model and the second model to calculate the similarity of the two products, and the first model only fuses the text, the picture, the named entity information and the picture information of the same product, and the second model fuses the text, the picture, the named entity information and the picture information of the two products.
[0011] Optionally, the named entity information includes one or more of a brand, a series, and a category of a product.
[0012] Optionally, the image information includes a cropped image corresponding to the product detection frame.
[0013] Optionally, the first model is a dual-tower model and the second model is a single-tower model.
[0014] The second aspect of the present application discloses a commodity weight determination method for electronic equipment, the method comprising:
[0015] Receiving data to be processed and commodity library data, wherein the data to be processed includes text and pictures related to the commodity to be processed, and the commodity library data includes text and pictures related to commodities in multiple commodity libraries;
[0016] The data to be processed and the commodity library data are used as inputs of the graphic model, and a matching result between the commodity to be processed and each commodity library commodity in the plurality of commodity library commodities is output.
[0017] Wherein, the graphic model is trained according to the method described in the first aspect of the present application, and wherein, the first model of the graphic model determines multiple candidate product library products that match the product to be processed, and the second model of the graphic model determines a matching product library product that matches the product to be processed from the multiple candidate product library products.
[0018] The third aspect of the present application discloses a graphic model training device, the device comprising:
[0019] An acquisition module is configured to acquire training data, wherein the training data includes texts and pictures related to a plurality of commodities;
[0020] An extraction module is configured to extract, for each product, named entity information of the product from the text related to the product, and image information of the product from the image related to the product;
[0021] a training module configured to train a pre-built model using the text, the image, the named entity information and the image information of the plurality of commodities to obtain the trained image-text model;
[0022] The pre-built model includes a first model and a second model, and the text, the picture, the named entity information and the picture information of two of the multiple products are respectively input into the first model and the second model to calculate the similarity of the two products, and the first model only fuses the text, the picture, the named entity information and the picture information of the same product, and the second model fuses the text, the picture, the named entity information and the picture information of the two products.
[0023] The fourth aspect of the present application discloses a commodity weight determination device, the device comprising:
[0024] A receiving module, configured to receive data to be processed and commodity library data, wherein the data to be processed includes text and pictures related to the commodity to be processed, and the commodity library data includes text and pictures related to commodities in multiple commodity libraries;
[0025] a matching module configured to use the data to be processed and the commodity library data as inputs of the graphic model, and output a matching result between the commodity to be processed and each commodity library commodity in the plurality of commodity library commodities,
[0026] Wherein, the graphic model is trained according to the method described in the first aspect of the present application, and wherein, the first model of the graphic model determines multiple candidate product library products that match the product to be processed, and the second model of the graphic model determines a matching product library product that matches the product to be processed from the multiple candidate product library products.
[0027] The fifth aspect of the present application discloses an electronic device, which includes a memory storing computer-executable instructions and a processor; when the computer-executable instructions are executed by the processor, the electronic device implements the method described in the first aspect of the present application, or implements the method described in the second aspect of the present application.
[0028] The sixth aspect of the present application discloses a computer-readable medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the method described in the first aspect of the present application, or to implement the method described in the second aspect of the present application.
[0029] The seventh aspect of the present application discloses a computer program product, including a computer program, which, when executed by a processor, implements the method according to the first aspect of the present application, or implements the method according to the second aspect of the present application.
[0030] Compared with the existing technology, this application can further improve the efficiency and accuracy of product weight determination through multi-mode information fusion. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 is a schematic diagram of an application scenario according to an embodiment of the present application;
[0032] Figure 2 is a flowchart of a method for training a graphic model according to an embodiment of the present application;
[0033] Figure 3A and Figure 3B They are schematic diagrams of a method for training a graphic-text model according to an embodiment of the present application;
[0034] Figure 4 is a flow chart of a method for determining the weight of a commodity according to an embodiment of the present application;
[0035] Figure 5 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION
[0036] The present application will be further described below in conjunction with specific embodiments and accompanying drawings. It will be appreciated that the specific embodiments described herein are only intended to explain the present application and are not intended to limit the present application. In addition, for ease of description, only some but not all structures or processes related to the present application are shown in the accompanying drawings. It should be noted that in this specification, similar reference numerals and letters represent similar items in the following accompanying drawings.
[0037] It should be understood that although the terms "first", "second", etc. may be used herein to describe various features, these features should not be limited by these terms. These terms are used only to distinguish and should not be understood as indicating or implying relative importance. For example, a first feature may be referred to as a second feature, and similarly a second feature may be referred to as a first feature without departing from the scope of the exemplary embodiments.
[0038] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.
[0039] Figure 1 : is a schematic diagram of an application scenario according to an embodiment of the present application. In this application scenario, the device involved may include a model training device for training a graphic model, and the model training device may be a server or a terminal. Figure 1 Take the model training device as the first server 101 as an example. On the first server 101, the pre-built model can be trained based on the training data containing text and pictures to obtain a trained picture and text model.
[0040] like Figure 1 As shown, the training data may be collected in advance and stored in a database.
[0041] like Figure 1 As shown, the device involved in the application scenario may also include a processing device for processing the data to be processed through the graphic model, and the processing device may be a server or a terminal. Figure 1 For example, the processing device is the second server 102. After the graph model is trained based on the training data on the first server 101, the graph model on the first server 101 is deployed on the second server 102, and the data to be processed is processed on the second server 102 by the graph model.
[0042] like Figure 1 As shown, the device involved in the application scenario may also include a terminal 103 , and the terminal 103 may send the data to be processed to the second server 102 . The second server 102 processes the data to be processed through a graphic model and returns the result to the terminal 103 .
[0043] Among them, the client 103 can be one or more, and the number of the client 103 should not be used as a limiting condition of the present application. The client 103 can be a computer device such as a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., which has computing power, input and output functions, and network connection functions. The servers 101 and 102 can be a single server device, or a server cluster, a cloud server, etc., which can provide application services to the outside.
[0044] The technical solution of the present application and how the technical solution of the present application solves the technical problem are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.
[0045] Figure 2 It is a flowchart of a method for training a graphic model according to an embodiment of the present application.
[0046] like Figure 2 As shown, in S201, training data is obtained, and the training data includes texts and pictures related to multiple commodities.
[0047] As an implementation method, the training data may be derived from a large amount of user-generated content, wherein each user-generated content includes textual content describing one or more products (such as the title and body of the user-generated content), and includes image content containing background information and / or products. As an example, the training data includes text and images related to product 1, text and images related to product 2, and text and images related to product 3, etc. It is worth noting that the acquisition, storage, use, and processing of data in the technical solutions disclosed in the embodiments of this application are in compliance with the relevant provisions of national laws and regulations.
[0048] In S202 , for each product, named entity information of the product is extracted from text related to the product, and image information of the product is extracted from images related to the product.
[0049] As an implementation, based on Name Entity Recognition (NER) technology, named entity information of the product is extracted from the text related to the product, and the named entity information may include one or more of the brand, series, and category of the product. As an example, the text related to the product may be "Brand A, Series B Bracelet", and the named entity information of the product may be "Brand A", "Series B", and "Bracelet".
[0050] As an implementation manner, the image information may include a cutout corresponding to the commodity detection frame. The number of the cutout may be one or more.
[0051] Those skilled in the art will appreciate that the extraction of the above-mentioned picture information can be achieved by utilizing the commodity recognition technology based on image detection in the prior art, which will not be elaborated herein.
[0052] In S203, a pre-built model is trained using the multiple texts and pictures related to the products obtained in S201 and the named entity information and picture information of the multiple products obtained in S202 to obtain a trained graphic-text model, wherein the pre-built model includes a first model and a second model, and the text, picture, named entity information and picture information of two products among the multiple products are respectively input into the first model and the second model to calculate the similarity of the two products, and wherein the first model only integrates the text, picture, named entity information and picture information of the same product, and the second model integrates the text, picture, named entity information and picture information of the two products.
[0053] As an implementation method, the image information in the training data can be manually annotated, that is, it is manually determined that two products match (that is, the similarity between the two is high), and two products do not match (that is, the similarity between the two is low). Matching product pairs constitute positive examples, while mismatched product pairs constitute negative examples, and the text, pictures, named entity information and picture information of the positive or negative examples are respectively input into the first model and the second model to calculate the similarity between the two products in the product pair. Based on this, the pre-built model can learn the alignment relationship between the original text and pictures of the two products and the cutouts, and then learn the information relied on when manually judging the duplication of products, thereby achieving the effect of product duplicate detection.
[0054] As an implementation method, the first model is a double-tower model, and the second model is a single-tower model. When establishing and maintaining a commodity library, for newly added commodities, a portion of existing commodity library commodities will be first recalled from the commodity library as candidate commodity library commodities that roughly match the newly added commodities, and then these candidate commodity library commodities will be sorted to determine the matching commodity library commodities that best match the newly added commodities. The first model is used in the recall stage, using the double-tower model to recall possible duplicate commodity library commodities from the commodity library. Considering the large number of commodity library commodities in the commodity library, if a single-tower (interactive) model is used, the calculation cost is too high. The second model is used in the sorting stage, using the single-tower (interactive) model to predict the similarity of several (such as dozens) candidate commodity library commodities recalled in the recall stage, and selecting the candidate commodity library commodities that exceed the similarity threshold and are ranked high (such as the first or top five) as matching commodity library commodities.
[0055] As an implementation, both the first model and the second model may be dual-stream or single-stream Transformer models. As an example, the first model may include two multimodal BERT models, and the second model may be a single multimodal BERT model.
[0056] like Figure 2 As shown, according to an embodiment of the present application, the graphic model training method is executed in the order of the above steps, but those skilled in the art will understand that this should not be used as a limitation of the present application. For example, the extraction of named entity information and image information in the training data can be performed simultaneously, or can be performed sequentially and the order of the two can be interchanged.
[0057] According to the embodiments of the present application, by directly pre-fusing the original text and pictures related to the products and the named entity information and picture information extracted therefrom, the fusion strategy is handed over to the model, which improves the overall effect and the robustness of duplicate detection, and achieves a balance between time and effect through the two stages of recall and sorting.
[0058] The following combination Figure 3A and Figure 3B right Figure 2 The graphic training method shown is explained. Figure 3A and Figure 3B They are respectively schematic diagrams of a method for training a graphic-text model according to an embodiment of the present application.
[0059] like Figure 3A As shown, the input 302 received by the left multimodal BERT model 301 in the first model includes the text of the first product (i.e., Text 1), the image (i.e., Image 1), the named entity information (i.e., NER 1), and the image information (i.e., Crop Figure 1 ), while the input 304 received by the multimodal BERT model 303 on the right includes the text (i.e., Text 2), the image (i.e., Image 2), the named entity information (i.e., NER 2), and the image information (i.e., Crop Figure 2 ).
[0060] The multimodal BERT model 301 on the left only extracts and fuses the input information of the first product, and the multimodal BERT model 303 on the right only extracts and fuses the input information of the second product, but there is no information fusion between the two products.
[0061] Using contrastive learning, the first model learns the similarity between two products. For positive examples, the cosine similarity between the fused vectors of the left and right models should be small, while for negative examples, the cosine similarity between the fused vectors of the left and right models should be large.
[0062] Next, through iterative training, the model training is stopped when the loss function value of the first model is less than or equal to the loss threshold, or when the number of iterations reaches the number threshold. After the training is completed, the left and right models share parameters.
[0063] like Figure 3B As shown, the input 306 received by the multimodal BERT model 305 as the second model includes the text (i.e., Text 1), the image (i.e., Image 1), the named entity information (i.e., NER 1), and the image information (i.e., Crop 1) of the first product. Figure 1 ), as well as the text (i.e., Text 2), image (i.e., Image 2), named entity information (i.e., NER 2), and image information (i.e., Crop 2) of the second product. Figure 2 ), where the input information of two products is separated by [SEP].
[0064] The input information of the first model and the second model is the same, but the multimodal BERT model 305 extracts and fuses the input information of the first product and the second product. By fusing the fine-grained features of the two products, the probability of determining whether the products are duplicated can be improved.
[0065] Similarly, using contrastive learning, the second model learns the similarity between two products. For a positive example, the multimodal BERT model 305 outputs 1, which means that the two products are duplicated, and for a negative example, the multimodal BERT model 305 outputs 0, which means that the two products are not duplicated.
[0066] Next, through iterative training, the model training is stopped when the loss function value of the second model is less than or equal to the loss threshold, or the model training is stopped when the number of iterations reaches the number threshold.
[0067] According to an embodiment of the present application, a method for determining the weight of a commodity is also included. Figure 4 It is a flow chart of a method for determining the weight of goods according to an embodiment of the present application.
[0068] like Figure 4 As shown, in S401, data to be processed and commodity library data are received, the data to be processed includes text and pictures related to the commodity to be processed, and the commodity library data includes text and pictures related to commodities in multiple commodity libraries.
[0069] As an implementation method, the unprocessed products in the data to be processed may be newly added products, or products to be added to the product library. Product library products identical to the products may already exist in the product library, so it is necessary to recall possible duplicate product library products from the product library, and then sort the recalled candidate product library products, and select matching product library products.
[0070] In S402, the data to be processed and the commodity library data are used as inputs of the graphic model, and a matching result between the commodity to be processed and each commodity library commodity in the multiple commodity library commodities is output.
[0071] As an implementation method, the first model in the graphic model extracts and fuses the text, pictures, named entity information and image information of the newly added product, and extracts and fuses the text, pictures, named entity information and image information of each product library product in the product library, calculates the similarity between the new product and each product library product, and recalls the top k product library products that roughly match the new product from the product library as candidate product library products.
[0072] The newly added product is combined with each of the k candidate product library products to form k product pairs. The second model in the graphic model extracts and fuses the text, image, named entity information and image information of each product pair, and calculates the similarity between the newly added product and each candidate product library product, thereby selecting a candidate product library product that exceeds the similarity threshold and ranks first as the matching product library product, or selecting multiple candidate product library products that exceed the similarity threshold and rank high as the matching product library products. At this time, it can be considered that there are product library products in the product library that are repeated with the newly added product, so the newly added product can be added to the product library. It is also possible that there are no candidate product library products that exceed the similarity threshold. At this time, it can be considered that there are no product library products in the product library that are repeated with the newly added product, so the newly added product can be added to the product library. In this way, the establishment and maintenance of the product library can be achieved.
[0073] Those skilled in the art will understand that Figure 4 The method shown is based on Figure 2 The specific content can be found in the above Figure 2 The description of FIG. 3 will not be repeated here.
[0074] Figure 5 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present application.
[0075] like Figure 5 As shown, the electronic device may include one or more processors 502, a system control logic 508 connected to at least one of the processors 502, a system memory 504 connected to the system control logic 508, a non-volatile memory (NVM) 506 connected to the system control logic 508, and a communication interface 510 connected to the system control logic 508.
[0076] The processor 502 may include one or more single-core or multi-core processors. The processor 502 may include any combination of a general-purpose processor and a special-purpose processor (e.g., a graphics processor, an application processor, a baseband processor, etc.). In an embodiment of the present application, the processor 502 may be configured to execute the following Figure 2-4 The method shown.
[0077] In some embodiments, system control logic 508 may include any suitable interface controller to provide any suitable interface to at least one of processors 502 and / or any suitable device or component in communication with system control logic 508 .
[0078] In some embodiments, the system control logic 508 may include one or more memory controllers to provide an interface to the system memory 504. The system memory 504 may be used to load and store data and / or instructions. In some embodiments, the system memory 504 of the electronic device may include any suitable volatile memory, such as a suitable dynamic random access memory (DRAM).
[0079] NVM 506 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, NVM 506 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as at least one of a HDD (Hard Disk Drive), a CD (CompactDisc) drive, and a DVD (Digital Versatile Disc) drive.
[0080] NVM 506 may include a portion of storage resources installed on a device of an electronic device, or it may be accessible by the device but not necessarily a portion of the device. For example, NVM 506 may be accessed over a network via communication interface 510 .
[0081] In particular, system memory 504 and NVM 506 may include, respectively, a temporary copy and a permanent copy of instructions 520. Instructions 520 may include instructions that, when executed by at least one of processors 502, cause the electronic device to implement the following: Figure 2-4 In some embodiments, instructions 520, hardware, firmware, and / or software components thereof may additionally / alternatively be placed in system control logic 508, communication interface 510, and / or processor 502.
[0082] The communication interface 510 may include a transceiver for providing a radio interface for the electronic device to communicate with any other suitable device (e.g., a front-end module, an antenna, etc.) through one or more networks. In some embodiments, the communication interface 510 may be integrated with other components of the electronic device. For example, the communication interface 510 may be integrated with at least one of the processor 502, the system memory 504, the NVM 506, and a firmware device (not shown) having instructions. When at least one of the processors 502 executes the instructions, the electronic device implements Figure 2-4 One or more of the various embodiments shown.
[0083] The communication interface 510 may further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, the communication interface 510 may be a network adapter, a wireless network adapter, a telephone modem, and / or a wireless modem.
[0084] In one embodiment, at least one of the processors 502 may be packaged together with one or more controllers for the system control logic 508 to form a system in package (SiP). In one embodiment, at least one of the processors 502 may be integrated on the same die with one or more controllers for the system control logic 508 to form a system on a chip (SoC).
[0085] The electronic device may further include: an input / output (I / O) device 512 connected to the system control logic 508. The I / O device 512 may include a user interface to enable a user to interact with the electronic device; the design of the peripheral component interface enables the peripheral component to interact with the electronic device. In some embodiments, the electronic device also includes a sensor for determining at least one of an environmental condition and location information related to the electronic device.
[0086] In some embodiments, I / O device 512 may include, but is not limited to, a display (e.g., a liquid crystal display, a touch screen display, etc.), a speaker, a microphone, one or more cameras (e.g., a still image camera and / or a video camera), a flashlight (e.g., an LED flash), and a keyboard.
[0087] In some embodiments, the peripheral component interface may include, but is not limited to, a non-volatile memory port, an audio jack, and a power interface.
[0088] It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.
[0089] Program code can be applied to input instructions to perform the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a system for processing instructions including a processor includes any system having a processor such as a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.
[0090] Program code can be implemented with high-level programming language or object-oriented programming language to communicate with the processing system. When necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.
[0091] According to an embodiment of the present application, a computer-readable storage medium is also proposed. The computer-readable storage medium stores one or more programs. The one or more programs can be executed by one or more processors to implement the aforementioned method.
[0092] According to one embodiment of the present application, a computer program product is also provided, and the aforementioned method is implemented when the computer program is executed by a processor.
[0093] The illustrative embodiments of the present application include, but are not limited to, a graphic model training method, a commodity weight determination method, a device, and a medium.
[0094] The various aspects of the illustrative embodiments will be described using terms commonly used by those skilled in the art to convey the essence of their work to other persons skilled in the art. However, it is apparent to those skilled in the art that some alternative embodiments may be implemented using the features described in part. For purposes of explanation, specific numbers and configurations are set forth to provide a more thorough understanding of the illustrative embodiments. However, it is apparent to those skilled in the art that alternative embodiments may be implemented without the specific details. In some other cases, some well-known features are omitted or simplified herein to avoid obscuring the illustrative embodiments of the present application.
[0095] In addition, various operations will be described as multiple separate operations in a manner that is most helpful for understanding the illustrative embodiments; however, the order of description should not be interpreted as implying that these operations must rely on the order of description, and many of the operations can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the operations can also be rearranged. When the described operations are completed, the process can be terminated, but can also have additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.
[0096] References in the specification to "an example", "an example", "an embodiment", "an implementation", etc. indicate that the described embodiment may include a particular feature, structure, or property, but each embodiment may or may not necessarily include the particular feature, structure, or property. Moreover, these phrases are not necessarily directed to the same embodiment. In addition, when particular features are described in conjunction with a specific embodiment, the knowledge of those skilled in the art can affect the combination of these features with other embodiments, whether or not these embodiments are explicitly described.
[0097] Unless the context dictates otherwise, the terms "comprising," "having," and "including" are synonymous. The phrase "A and / or B" means "(A), (B), or (A and B)."
[0098] As used herein, the term "module" may refer to, be part of, or include: memory (shared, dedicated, or group) for running one or more software or firmware programs, application-specific integrated circuits (ASICs), electronic circuits and / or processors (shared, dedicated, or group), combinational logic circuits, and / or other suitable components that provide the described functionality.
[0099] In the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order is not necessary. Instead, in some embodiments, these features may be described in a manner and / or order different from that shown in the illustrative drawings. In addition, the structural or method features included in a specific figure do not mean that all embodiments need to include such features. In some embodiments, these features may not be included or may be combined with other features.
[0100] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.
[0101] Similarly, it should be understood that in order to streamline the present application and aid in understanding one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the following intention: the claimed application requires more features than the features explicitly recited in each claim. More specifically, as reflected in the claims, the inventive aspects lie in less than all the features of the single embodiment disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present application.
[0102] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.
[0103] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present application and form different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.
Claims
1. A method for training a graphic model, used in electronic equipment, characterized in that: The method comprises: Acquire training data, wherein the training data includes text and images related to a plurality of commodities; For each product, extract the named entity information of the product from the text related to the product, and extract the image information of the product from the image related to the product; Using the texts, pictures, named entity information and picture information of the multiple commodities to train a pre-built model to obtain the trained picture-text model, The pre-built model includes a first model and a second model, and the text, the picture, the named entity information and the picture information of two of the multiple products are respectively input into the first model and the second model to calculate the similarity of the two products, and the first model only fuses the text, the picture, the named entity information and the picture information of the same product, and the second model fuses the text, the picture, the named entity information and the picture information of the two products.
2. The method according to claim 1, characterized in that The named entity information includes one or more of a brand, a series, and a category of a product.
3. The method according to claim 2, characterized in that The image information includes a cutout corresponding to the commodity detection frame.
4. The method according to claim 1, characterized in that The first model is a dual tower model, and the second model is a single tower model.
5. A method for determining the weight of a commodity, used in electronic equipment, characterized in that: The method comprises: Receiving data to be processed and commodity library data, wherein the data to be processed includes text and pictures related to the commodity to be processed, and the commodity library data includes text and pictures related to commodities in multiple commodity libraries; The data to be processed and the commodity library data are used as inputs of the graphic model, and a matching result between the commodity to be processed and each commodity library commodity in the plurality of commodity library commodities is output. Wherein, the graphic model is trained according to the method described in any one of claims 1 to 4, and wherein, a first model of the graphic model determines a plurality of candidate product library products that match the product to be processed, and a second model of the graphic model determines a matching product library product that matches the product to be processed from the plurality of candidate product library products.
6. A graphic model training device, characterized in that: The device comprises: an acquisition module configured to acquire training data, wherein the training data includes texts and pictures related to a plurality of commodities; an extraction module configured to extract, for each commodity, named entity information of the commodity from the text related to the commodity, and extract picture information of the commodity from the pictures related to the commodity; a training module configured to train a pre-built model using the text, the image, the named entity information and the image information of the plurality of commodities to obtain the trained image-text model; The pre-built model includes a first model and a second model, and the text, the picture, the named entity information and the picture information of two of the multiple products are respectively input into the first model and the second model to calculate the similarity of the two products, and the first model only fuses the text, the picture, the named entity information and the picture information of the same product, and the second model fuses the text, the picture, the named entity information and the picture information of the two products.
7. A commodity weight determination device, characterized in that: The device comprises: A receiving module, configured to receive data to be processed and commodity library data, wherein the data to be processed includes text and pictures related to the commodity to be processed, and the commodity library data includes text and pictures related to commodities in multiple commodity libraries; a matching module configured to use the data to be processed and the commodity library data as inputs of the graphic model, and output a matching result between the commodity to be processed and each commodity library commodity in the plurality of commodity library commodities, Wherein, the graphic model is trained according to the method described in any one of claims 1 to 4, and wherein, a first model of the graphic model determines a plurality of candidate product library products that match the product to be processed, and a second model of the graphic model determines a matching product library product that matches the product to be processed from the plurality of candidate product library products.
8. An electronic device, characterized in that: The electronic device comprises a memory storing computer executable instructions and a processor; when the computer executable instructions are executed by the processor, the electronic device implements the method according to any one of claims 1 to 4, or implements the method according to claim 5.
9. A computer-readable medium, characterized in that The computer-readable medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the method according to any one of claims 1 to 4, or to implement the method according to claim 5.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented, or the method according to claim 5 is implemented.
