Image-text model training method, commodity main image selection method, equipment and medium

CN119068279APending Publication Date: 2024-12-03XIAOHONGSHU TECH CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202310640895.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-06-01
Publication Date
2024-12-03

Smart Images

  • Figure CN119068279A_ABST
    Figure CN119068279A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of artificial intelligence, in particular to an image-text model training method, a commodity main image selection method, equipment and a medium. The image-text model training method comprises the steps of obtaining training data, wherein the training data comprises texts and pictures related to commodities; for the commodity, extracting named entity information of the commodity from the text related to the commodity, and extracting picture information of the commodity from the picture related to the commodity; the text, the picture, the named entity information and the picture information of the commodity are utilized to train a pre-constructed model, a trained image-text model is obtained, the text, the picture, the named entity information and the picture information of the commodity are input into the pre-constructed model to calculate the prediction probability of the picture information, and the prediction probability of the picture information of the commodity is calculated; the pre-constructed model fuses the text, the picture, the named entity information and the picture information of the commodity. According to the invention, through multi-mode information fusion, the accuracy of commodity main image selection can be further improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method for training a graphic model, a method for selecting a main product image, a device and a medium. Background Art

[0002] Users will publish and share user-generated content (UGC) with other users on the Internet, and the content may be about a certain product.

[0003] In order to identify the products that appear in these user-generated contents, it is necessary to first establish a product library. An important attribute of the product library is the main image of the product, that is, to select the product image that best matches the product from multiple product images that contain background information and / or products.

[0004] Traditional main image selection methods mainly rely on image detection and the size of the detection box, the position in the image, etc., and will extract the content in the larger, off-center detection box as the main image. However, this method has limitations. For smaller products, such as accessories such as bracelets and watches, the picture may show a model wearing a bracelet or watch, and the size of these accessories will be much smaller than the clothes on the model. In this case, the existing technical methods based on simple rules can easily extract the wrong content in the detection box as the main image of the product. Summary of the invention

[0005] In order to solve the above-mentioned problems, the present application provides a picture-text model training method, a product main picture selection method, equipment and medium.

[0006] The first aspect of the present application discloses a method for training a graphic model for an electronic device, the method comprising:

[0007] Acquire training data, where the training data includes text and images related to the product;

[0008] For the product, extracting the named entity information of the product from the text related to the product, and extracting the image information of the product from the image related to the product;

[0009] Using the text, the picture, the named entity information and the picture information of the product to train a pre-built model to obtain the trained picture-text model,

[0010] The text, the picture, the named entity information and the picture information of the product are input into the pre-built model to calculate the prediction probability of the picture information, and the pre-built model integrates the text, the picture, the named entity information and the picture information of the product.

[0011] Optionally, the named entity information includes one or more of a brand, a series, and a category of a product.

[0012] Optionally, the image information includes a cropped image corresponding to the product detection frame.

[0013] Optionally, the image information further includes coordinates of the product detection frame and the size of the product detection frame.

[0014] A second aspect of the present application discloses a method for selecting a main image of a product, which is used in an electronic device. The method comprises:

[0015] Receiving data to be processed, wherein the data to be processed includes text and pictures related to the commodity to be processed;

[0016] The data to be processed is used as the input of the image-text model, and the main image of the product matching the product to be processed is output.

[0017] Wherein, the graphic model is trained according to the method described in the first aspect of the present application.

[0018] The third aspect of the present application discloses a graphic model training device, the device comprising:

[0019] An acquisition module is configured to acquire training data, wherein the training data includes text and pictures related to the product;

[0020] an extraction module, configured to extract, for the commodity, named entity information of the commodity from the text related to the commodity, and extract image information of the commodity from the image related to the commodity;

[0021] A training module is configured to train a pre-built model using the text, the image, the named entity information and the image information of the product to obtain the trained image-text model,

[0022] The text, the picture, the named entity information and the picture information of the product are input into the pre-built model to calculate the prediction probability of the picture information, and the pre-built model integrates the text, the picture, the named entity information and the picture information of the product.

[0023] The fourth aspect of the present application discloses a device for selecting a main image of a product, the device comprising:

[0024] A receiving module, configured to receive data to be processed, wherein the data to be processed includes text and pictures related to the commodity to be processed;

[0025] The selection module is configured to use the data to be processed as the input of the image-text model and output a main image of a product that matches the product to be processed.

[0026] Wherein, the graphic model is trained according to the method described in the first aspect of the present application.

[0027] The fifth aspect of the present application discloses an electronic device, which includes a memory storing computer-executable instructions and a processor; when the computer-executable instructions are executed by the processor, the electronic device implements the method described in the first aspect of the present application, or implements the method described in the second aspect of the present application.

[0028] The sixth aspect of the present application discloses a computer-readable medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the method described in the first aspect of the present application, or to implement the method described in the second aspect of the present application.

[0029] The seventh aspect of the present application discloses a computer program product, including a computer program, which, when executed by a processor, implements the method according to the first aspect of the present application, or implements the method according to the second aspect of the present application.

[0030] Compared with the existing technology, this application can further improve the accuracy of product main image selection through multi-modal information fusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a schematic diagram of an application scenario according to an embodiment of the present application;

[0032] Figure 2 is a flowchart of a method for training a graphic model according to an embodiment of the present application;

[0033] Figure 3 is a schematic diagram of a method for training a graphic model according to an embodiment of the present application;

[0034] Figure 4 is a flowchart of a method for selecting a main image of a product according to an embodiment of the present application;

[0035] Figure 5 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0036] The present application will be further described below in conjunction with specific embodiments and accompanying drawings. It will be appreciated that the specific embodiments described herein are only intended to explain the present application and are not intended to limit the present application. In addition, for ease of description, only some but not all structures or processes related to the present application are shown in the accompanying drawings. It should be noted that in this specification, similar reference numerals and letters represent similar items in the following accompanying drawings.

[0037] It should be understood that although the terms "first", "second", etc. may be used herein to describe various features, these features should not be limited by these terms. These terms are used only to distinguish and should not be understood as indicating or implying relative importance. For example, a first feature may be referred to as a second feature, and similarly a second feature may be referred to as a first feature without departing from the scope of the exemplary embodiments.

[0038] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0039] Figure 1 : is a schematic diagram of an application scenario according to an embodiment of the present application. In this application scenario, the device involved may include a model training device for training a graphic model, and the model training device may be a server or a terminal. Figure 1 Take the model training device as the first server 101 as an example. On the first server 101, the pre-built model can be trained based on the training data containing text and pictures to obtain a trained picture and text model.

[0040] like Figure 1 As shown, the training data may be collected in advance and stored in a database.

[0041] like Figure 1 As shown, the device involved in the application scenario may also include a processing device for processing the data to be processed through the graphic model, and the processing device may be a server or a terminal. Figure 1 For example, the processing device is the second server 102. After the graph model is trained based on the training data on the first server 101, the graph model on the first server 101 is deployed on the second server 102, and the data to be processed is processed on the second server 102 by the graph model.

[0042] like Figure 1 As shown, the device involved in the application scenario may also include a terminal 103 , and the terminal 103 may send the data to be processed to the second server 102 . The second server 102 processes the data to be processed through a graphic model and returns the result to the terminal 103 .

[0043] Among them, the client 103 can be one or more, and the number of the client 103 should not be used as a limiting condition of the present application. The client 103 can be a computer device such as a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., which has computing power, input and output functions, and network connection functions. The servers 101 and 102 can be a single server device, or a server cluster, a cloud server, etc., which can provide application services to the outside.

[0044] The technical solution of the present application and how the technical solution of the present application solves the technical problem are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0045] Figure 2 It is a flowchart of a method for training a graphic model according to an embodiment of the present application.

[0046] like Figure 2 As shown, in S201, training data is obtained, and the training data includes text and pictures related to the product.

[0047] As an implementation method, the training data may be derived from a large amount of user-generated content, wherein each user-generated content includes textual content describing one or more products (such as the title and body of the user-generated content), and includes image content containing background information and / or products. It is worth noting that the acquisition, storage, use, and processing of data in the technical solutions disclosed in the embodiments of this application are in compliance with the relevant provisions of national laws and regulations.

[0048] In S202 , for a product, named entity information of the product is extracted from text related to the product, and image information of the product is extracted from images related to the product.

[0049] As an implementation, based on Name Entity Recognition (NER) technology, named entity information of the product is extracted from the text related to the product, and the named entity information may include one or more of the brand, series, and category of the product. As an example, the text related to the product may be "Brand A, Series B Bracelet", and the named entity information of the product may be "Brand A", "Series B", and "Bracelet".

[0050] As an implementation, the image information may include a cropped image corresponding to the product detection frame. The number of the cropped images may be one or more, and these cropped images are candidate images for the main image of the product, and one or more cropped images will be selected from these cropped images as the main image of the product.

[0051] As an implementation method, the image information may also include the coordinates of the commodity detection frame and the size of the commodity detection frame, etc. For each cropped image, parameters such as the coordinates and size of the corresponding commodity detection frame are introduced.

[0052] Those skilled in the art will appreciate that the extraction of the above-mentioned picture information can be achieved by utilizing the commodity recognition technology based on image detection in the prior art, which will not be elaborated herein.

[0053] In S203, the pre-built model is trained using the text and pictures related to the product obtained in S201 and the named entity information and picture information of the product obtained in S202 to obtain a trained graphic-text model, wherein the text, pictures, named entity information and picture information of the product are input into the pre-built model to calculate the prediction probability of the picture information, and wherein the pre-built model integrates the text, pictures, named entity information and picture information of the product.

[0054] As an implementation method, the image information in the training data can be manually annotated, that is, it is manually determined that one or more cropped images can be used as the main image of the product, and the remaining cropped images cannot be used as the main image of the product. Based on this, the pre-built model can learn the alignment relationship between the original text and image and the cropped image and the main image, and at the same time introduce parameters such as the coordinates and size of the product detection frame, and then learn the information that is relied on when manually judging the main image, so as to achieve the effect of selecting the main image of the product.

[0055] As an implementation, the pre-built model may be a Transformer model. As an example, the pre-built model may be a multimodal BERT model.

[0056] As an implementation method, at the corresponding cropping image position, the last layer of the Transformer model outputs 0 or 1, that is, the predicted probability of each cropping image, where 0 represents that the corresponding cropping image is not the main image, and 1 represents that the corresponding cropping image is the main image. Those skilled in the art will understand that there may be a situation where one or more cropping images can be used as the main image of the product, that is, the last layer of the Transformer model may output multiple 1s.

[0057] like Figure 2 As shown, according to an embodiment of the present application, the graphic model training method is executed in the order of the above steps, but those skilled in the art will understand that this should not be used as a limitation of the present application. For example, the extraction of named entity information and image information in the training data can be performed simultaneously, or can be performed sequentially and the order of the two can be interchanged.

[0058] According to the embodiments of the present application, the model can fuse the original text and pictures related to the product and the named entity information and picture information extracted therefrom, learn the fine-grained information in the original text (such as the mentioned A brand B series bracelet), align the original text and pictures and each cropped picture in the same semantic space, and consider the coordinates and size of the product detection frame and other parameters, and then learn the information relied on when manually judging the main picture, so as to achieve the effect of selecting the main picture of the product. By utilizing the fusion of the above multi-modal information, the trained picture and text model has better recognition accuracy.

[0059] The following combination Figure 3 right Figure 2 The graphic training method shown is explained. Figure 3 It is a schematic diagram of a method for training a graphic model according to an embodiment of the present application.

[0060] like Figure 3 As shown, the input 301 received by the Transformer model 300 includes the text, image, named entity information (i.e., NER) and image information (i.e., Crop Figure 1-3 ).

[0061] After learning the Transformer model 300, at the corresponding Crop graph position, the last layer of the Transformer model 300 outputs 0 or 1, 0 represents that the corresponding Crop graph is not the main graph, and 1 represents that the corresponding Crop graph is the main graph.

[0062] like Figure 3 As shown, the output of the last layer of Transformer model 300 is 0 / 1 / 0, so Crop Figure 2 The main image that is selected or determined to be the most suitable for the product.

[0063] Next, the output result of the Transformer model 300 is compared with the manual annotation result, and the parameters of the Transformer model 300 are updated according to the comparison result. Specifically, through iterative training, the model training is stopped when the difference of the comparison result is less than or equal to the difference threshold, or when the number of iterations reaches the number threshold.

[0064] According to an embodiment of the present application, a method for selecting a main image of a product is also included. Figure 4 It is a flowchart of a method for selecting a main image of a product according to an embodiment of the present application.

[0065] like Figure 4 As shown, in S401, data to be processed is received, and the data to be processed includes text and pictures related to the commodity to be processed.

[0066] As an implementation method, the product to be processed in the data to be processed may be a product to be added to the product library, and the original image related to the product is not necessarily the main image that best matches the product, so the corresponding main image needs to be cropped from the original image.

[0067] In S402, the data to be processed is used as the input of the image-text model, and a main image of the product matching the product to be processed is output.

[0068] As an implementation mode, the text and image of the commodity to be processed and the named entity information and image information extracted from the text and image respectively are integrated by the image-text model, and the prediction probability of the image information is calculated.

[0069] Those skilled in the art will understand that Figure 4 The method shown is based on Figure 2 and Figure 3 The trained graph-text model shown in the figure is implemented. For details, please refer to the above Figure 2 and Figure 3 The description will not be repeated here.

[0070] Figure 5 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present application.

[0071] like Figure 5 As shown, the electronic device may include one or more processors 502, a system control logic 508 connected to at least one of the processors 502, a system memory 504 connected to the system control logic 508, a non-volatile memory (NVM) 506 connected to the system control logic 508, and a communication interface 510 connected to the system control logic 508.

[0072] The processor 502 may include one or more single-core or multi-core processors. The processor 502 may include any combination of a general-purpose processor and a special-purpose processor (e.g., a graphics processor, an application processor, a baseband processor, etc.). In an embodiment of the present application, the processor 502 may be configured to execute the following Figure 2-4 The method shown.

[0073] In some embodiments, system control logic 508 may include any suitable interface controller to provide any suitable interface to at least one of processors 502 and / or any suitable device or component in communication with system control logic 508 .

[0074] In some embodiments, the system control logic 508 may include one or more memory controllers to provide an interface to the system memory 504. The system memory 504 may be used to load and store data and / or instructions. In some embodiments, the system memory 504 of the electronic device may include any suitable volatile memory, such as a suitable dynamic random access memory (DRAM).

[0075] NVM 506 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, NVM 506 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as at least one of a HDD (Hard Disk Drive), a CD (CompactDisc) drive, and a DVD (Digital Versatile Disc) drive.

[0076] NVM 506 may include a portion of storage resources installed on a device of an electronic device, or it may be accessible by the device but not necessarily a portion of the device. For example, NVM 506 may be accessed over a network via communication interface 510 .

[0077] In particular, system memory 504 and NVM 506 may include, respectively, a temporary copy and a permanent copy of instructions 520. Instructions 520 may include instructions that, when executed by at least one of processors 502, cause the electronic device to implement the following: Figure 2-4 In some embodiments, instructions 520, hardware, firmware, and / or software components thereof may additionally / alternatively be placed in system control logic 508, communication interface 510, and / or processor 502.

[0078] The communication interface 510 may include a transceiver for providing a radio interface for the electronic device to communicate with any other suitable device (e.g., a front-end module, an antenna, etc.) through one or more networks. In some embodiments, the communication interface 510 may be integrated with other components of the electronic device. For example, the communication interface 510 may be integrated with at least one of the processor 502, the system memory 504, the NVM 506, and a firmware device (not shown) having instructions. When at least one of the processors 502 executes the instructions, the electronic device implements Figure 2-4 One or more of the various embodiments shown.

[0079] The communication interface 510 may further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, the communication interface 510 may be a network adapter, a wireless network adapter, a telephone modem, and / or a wireless modem.

[0080] In one embodiment, at least one of the processors 502 may be packaged together with one or more controllers for the system control logic 508 to form a system in package (SiP). In one embodiment, at least one of the processors 502 may be integrated on the same die with one or more controllers for the system control logic 508 to form a system on a chip (SoC).

[0081] The electronic device may further include: an input / output (I / O) device 512 connected to the system control logic 508. The I / O device 512 may include a user interface to enable a user to interact with the electronic device; the design of the peripheral component interface enables the peripheral component to interact with the electronic device. In some embodiments, the electronic device also includes a sensor for determining at least one of an environmental condition and location information related to the electronic device.

[0082] In some embodiments, I / O device 512 may include, but is not limited to, a display (e.g., a liquid crystal display, a touch screen display, etc.), speakers, a microphone, one or more cameras (e.g., a still image camera and / or a video camera), a flashlight (e.g., an LED flash), and a keyboard.

[0083] In some embodiments, the peripheral component interface may include, but is not limited to, a non-volatile memory port, an audio jack, and a power interface.

[0084] It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0085] Program code can be applied to input instructions to perform the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a system for processing instructions including a processor includes any system having a processor such as a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0086] Program code can be implemented with high-level programming language or object-oriented programming language to communicate with the processing system. When necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.

[0087] According to an embodiment of the present application, a computer-readable storage medium is also proposed. The computer-readable storage medium stores one or more programs. The one or more programs can be executed by one or more processors to implement the aforementioned method.

[0088] According to one embodiment of the present application, a computer program product is also provided, and the aforementioned method is implemented when the computer program is executed by a processor.

[0089] The illustrative embodiments of the present application include, but are not limited to, a method for training a graphic model, a method for selecting a main product image, a device, and a medium.

[0090] The various aspects of the illustrative embodiments will be described using terms commonly used by those skilled in the art to convey the essence of their work to other persons skilled in the art. However, it is apparent to those skilled in the art that some alternative embodiments may be implemented using the features described in part. For purposes of explanation, specific numbers and configurations are set forth to provide a more thorough understanding of the illustrative embodiments. However, it is apparent to those skilled in the art that alternative embodiments may be implemented without the specific details. In some other cases, some well-known features are omitted or simplified herein to avoid obscuring the illustrative embodiments of the present application.

[0091] In addition, various operations will be described as multiple separate operations in a manner that is most helpful for understanding the illustrative embodiments; however, the order of description should not be interpreted as implying that these operations must rely on the order of description, and many of the operations can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the operations can also be rearranged. When the described operations are completed, the process can be terminated, but can also have additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0092] References in the specification to "an example", "an example", "an embodiment", "an implementation", etc. indicate that the described embodiment may include a particular feature, structure, or property, but each embodiment may or may not necessarily include the particular feature, structure, or property. Moreover, these phrases are not necessarily directed to the same embodiment. In addition, when particular features are described in conjunction with a specific embodiment, the knowledge of those skilled in the art can affect the combination of these features with other embodiments, whether or not these embodiments are explicitly described.

[0093] Unless the context dictates otherwise, the terms "comprising," "having," and "including" are synonymous. The phrase "A and / or B" means "(A), (B), or (A and B)."

[0094] As used herein, the term "module" may refer to, be part of, or include: memory (shared, dedicated, or group) for running one or more software or firmware programs, application-specific integrated circuits (ASICs), electronic circuits and / or processors (shared, dedicated, or group), combinational logic circuits, and / or other suitable components that provide the described functionality.

[0095] In the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order is not necessary. Instead, in some embodiments, these features may be described in a manner and / or order different from that shown in the illustrative drawings. In addition, the structural or method features included in a specific figure do not mean that all embodiments need to include such features. In some embodiments, these features may not be included or may be combined with other features.

[0096] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0097] Similarly, it should be understood that in order to streamline the present application and aid in understanding one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the following intention: the claimed application requires more features than the features explicitly recited in each claim. More specifically, as reflected in the claims, the inventive aspects lie in less than all the features of the single embodiment disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present application.

[0098] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0099] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present application and form different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.

Claims

1. A method for training a graphic model for electronic equipment, It is characterized in that The method comprises: Acquire training data, where the training data includes text and images related to the product; For the product, extracting the named entity information of the product from the text related to the product, and extracting the image information of the product from the image related to the product; Using the text, the picture, the named entity information and the picture information of the product to train a pre-built model to obtain the trained picture-text model, The text, the picture, the named entity information and the picture information of the product are input into the pre-built model to calculate the prediction probability of the picture information, and the pre-built model integrates the text, the picture, the named entity information and the picture information of the product.

2. The method according to claim 1, It is characterized in that The named entity information includes one or more of a brand, a series, and a category of a product.

3. The method according to claim 2, It is characterized in that The image information includes a cutout corresponding to the commodity detection frame.

4. The method according to claim 3, It is characterized in that The image information also includes the coordinates of the commodity detection frame and the size of the commodity detection frame.

5. A method for selecting a main product image for electronic equipment. It is characterized in that The method comprises: Receiving data to be processed, wherein the data to be processed includes text and pictures related to the commodity to be processed; The data to be processed is used as the input of the image-text model, and the main image of the product matching the product to be processed is output. The graphic model is trained according to any one of claims 1 to 4.

6. A graphic model training device, It is characterized in that The device comprises: An acquisition module is configured to acquire training data, wherein the training data includes text and pictures related to the product; an extraction module, configured to extract, for the commodity, named entity information of the commodity from the text related to the commodity, and extract image information of the commodity from the image related to the commodity; A training module is configured to train a pre-built model using the text, the image, the named entity information and the image information of the product to obtain the trained image-text model, The text, the picture, the named entity information and the picture information of the product are input into the pre-built model to calculate the prediction probability of the picture information, and the pre-built model integrates the text, the picture, the named entity information and the picture information of the product.

7. A device for selecting a main picture of a product, It is characterized in that The device comprises: A receiving module, configured to receive data to be processed, wherein the data to be processed includes text and pictures related to the commodity to be processed; The selection module is configured to use the data to be processed as the input of the image-text model and output a main image of a product that matches the product to be processed. The graphic model is trained according to any one of claims 1 to 4.

8. An electronic device, It is characterized in that The electronic device comprises a memory storing computer executable instructions and a processor; when the computer executable instructions are executed by the processor, the electronic device implements the method according to any one of claims 1 to 4, or implements the method according to claim 5.

9. A computer readable medium, It is characterized in that The computer-readable medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the method according to any one of claims 1 to 4, or to implement the method according to claim 5.

10. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented, or the method according to claim 5 is implemented.

Citation Information

Patent Citations

  • Target commodity identification method and device, equipment, medium and product

    CN114898349A

  • Commodity map determination method and electronic equipment

    CN115796991A

  • Method and apparatus for extracting structured data from image, and device and storage medium

    WO2021151270A1