Picture matching processing method and device and storage medium

Abstract

The invention provides a picture matching processing method and device and a storage medium, and the method comprises the steps: carrying out the semantic recognition of a to-be-processed article picture, and obtaining first article description information corresponding to the to-be-processed article picture; acquiring second article description information of the at least one article based on article classification information in the first article description information; determining a matching degree between the first article description information and second article description information of any article in the at least one article; and based on the matching degree, determining the associated article matched with the to-be-processed article picture, and determining the to-be-processed article picture as the article picture of the associated article, so that the to-be-processed article picture and the existing article in the database can be automatically associated, the personnel participation degree is reduced, and the consumption of human resources is avoided.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer technology and image processing, and particularly relates to a picture matching processing method and device and a storage medium. BACKGROUND

[0002] In many fields, such as the home decoration field and the e-commerce shopping guide field, the completeness and accuracy of the information of an item are crucial. However, due to the fact that a supplier does not upload pictures in a timely manner or the format of the pictures is unqualified, many items in the system lack pictures, which causes trouble for users to select items. For example, in the home decoration field, when a user (such as a designer or a homeowner) selects an item and determines a decoration scheme, the user usually needs to determine the decoration effect through an item picture. When an item (such as a ceramic tile or a cabinet) lacks a picture, the selection operation of the user is disturbed.

[0003] In the related art, when an item picture provided by a supplier is received, the received item picture needs to be manually entered into a database corresponding to an item SKU (Stock Keeping Unit) by a human being. When the quantity of the item SKU is large, the efficiency is low, and errors and omissions are prone to occur. SUMMARY

[0004] To solve the above technical problems, the present disclosure is proposed. Embodiments of the present disclosure provide a picture matching processing method, device and storage medium.

[0005] According to an aspect of an embodiment of the present disclosure, a picture matching processing method is provided, including: performing semantic recognition on a to-be-processed item picture to obtain first item description information corresponding to the to-be-processed item picture; obtaining second item description information of at least one item based on item classification information in the first item description information; determining a matching degree of the first item description information and the second item description information of any one of the at least one item; based on the matching degree, determining an associated item matched with the to-be-processed item picture, and determining the to-be-processed item picture as an item picture of the associated item.

[0006] In some optional embodiments, the performing semantic recognition on the to-be-processed item picture to obtain the first item description information corresponding to the to-be-processed item picture includes: generating a model prompt word based on the to-be-processed item picture; performing semantic recognition processing on the to-be-processed item picture based on the model prompt word by using a multi-modal language model to obtain the first item description information.

[0007] In some optional embodiments, the obtaining, based on the item classification information in the first item description information, the second item description information of the at least one item comprises: performing word segmentation processing on the first item description information to obtain item keywords including the item classification information; obtaining, by using the item classification information, the second item description information of the at least one item.

[0008] In some optional embodiments, the determining the matching degree between the first item description information and the second item description information of any of the at least one item comprises: for any of the item keywords, obtaining first item attribute information corresponding to the item keyword in the first item description information, and second item attribute information corresponding to the item keyword in the second item description information; determining a matching degree between the first item attribute information and the second item attribute information to obtain an attribute matching degree; determining the matching degree between the first item description information and the second item description information based on the attribute matching degree corresponding to any of the item keywords.

[0009] In some optional embodiments, the determining, based on the matching degree, the associated item matching the to-be-processed item picture comprises: determining, based on the matching degree, a reference item with the highest matching degree from the at least one item and the to-be-processed item picture; in response to the matching degree corresponding to the reference item being greater than a first preset value, determining the reference item as the associated item.

[0010] In some optional embodiments, after the determining, from the at least one item, the reference item with the highest matching degree and the to-be-processed item picture, the method further comprises: in response to the matching degree corresponding to the reference item being greater than a second preset value and not greater than the first preset value, determining the reference item as a recommendable item; generating associated prompt information, the associated prompt information including the second item description information of the recommendable item and the to-be-processed item picture; receiving a response operation triggered by a user based on the associated prompt information, and determining, based on the response operation, whether the recommendable item is the associated item matching the to-be-processed item picture.

[0011] According to still another aspect of the embodiments of the present disclosure, a picture matching processing apparatus is provided, comprising: An identification module is configured to perform semantic identification on the to-be-processed item picture to obtain first item description information corresponding to the to-be-processed item picture. An information acquisition module is configured to acquire second item description information of at least one item based on item classification information in the first item description information. A matching degree determination module is configured to determine a matching degree between the first item description information and the second item description information of any one of the at least one item. An item determination module is configured to determine an associated item matching the to-be-processed item picture based on the matching degree, and determine the to-be-processed item picture as an item picture of the associated item.

[0012] In some optional embodiments, the identification module comprises: A classification sub-module is configured to generate a model prompt word based on the to-be-processed item picture. An identification sub-module is configured to perform semantic identification on the to-be-processed item picture based on the model prompt word by using a multi-modal language model to obtain the first item description information.

[0013] In some optional embodiments, the information acquisition module comprises: A word segmentation sub-module is configured to perform word segmentation on the first item description information to obtain an item keyword comprising the item classification information. A first acquisition sub-module is configured to acquire second item description information of the at least one item by using the item classification information.

[0014] In some optional embodiments, the matching degree determination module comprises: A second acquisition sub-module is configured to acquire, for any one of the item keywords, first item attribute information corresponding to the item keyword in the first item description information, and second item attribute information corresponding to the item keyword in the second item description information. A first determination sub-module is configured to determine a matching degree between the first item attribute information and the second item attribute information to obtain an attribute matching degree. A second determination sub-module is configured to determine the matching degree between the first item description information and the second item description information based on the attribute matching degree corresponding to any one of the item keywords.

[0015] In some optional embodiments, the item determination module comprises: A third determination sub-module is configured to determine, from the at least one item, a reference item having the highest matching degree with the to-be-processed item picture based on the matching degree. A fourth determining sub-module is configured to determine the reference item as the associated item in response to the matching degree corresponding to the reference item being greater than a first preset value.

[0016] In some optional embodiments, the item determining module comprises: A fifth determining sub-module is configured to determine the reference item as the recommendable item in response to the matching degree corresponding to the reference item being greater than a second preset value and not greater than the first preset value. A prompting sub-module is configured to generate associated prompting information, wherein the associated prompting information comprises second item description information of the recommendable item and the to-be-processed item picture. A receiving sub-module is configured to receive a response operation triggered by a user based on the associated prompting information. A sixth determining sub-module is configured to determine whether the recommendable item is the associated item matching the to-be-processed item picture based on the response operation.

[0017] According to still another aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, which stores computer program instructions. The computer program instructions are executed to implement the picture matching processing method.

[0018] According to still another aspect of the embodiments of the present disclosure, an electronic device is provided, which comprises: a memory configured to store a computer program product; a processor configured to execute the computer program product stored in the memory, and the computer program product is executed to implement the picture matching processing method.

[0019] According to still another aspect of the embodiments of the present disclosure, a computer program product is provided, which comprises computer program instructions. The computer program instructions are executed by a processor to implement the picture matching processing method.

[0020] Based on the embodiments of the present disclosure, for the scene that needs to associate the picture of the to-be-processed item with the item SKU in the database, the picture of the to-be-processed item can be subjected to semantic recognition to obtain the first item description information corresponding to the picture of the to-be-processed item, and based on the item classification information in the first item description information, the second item description information of at least one item is obtained, then the matching degree of the first item description information and the second item description information of any one of the at least one item is determined, and based on the matching degree, the associated item matched with the picture of the to-be-processed item is determined, and the picture of the to-be-processed item is determined as the item picture of the associated item. Thus, the present technical solution can automatically realize the association between the picture of the to-be-processed item and the existing item in the database, reduce the participation of personnel, avoid the consumption of human resources, and improve the efficiency and accuracy of item picture association. Moreover, by screening the second item description information of the same item category and then determining the matching degree, the calculation amount of the matching degree can be reduced, and the calculation efficiency is further improved. In addition, by adding the item picture to the database as the item picture of the associated item, the user experience is improved, and the sales conversion rate of the item is increased.

[0021] The technical solutions of the present disclosure will be described in further detail below by means of the accompanying drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0022] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. The drawings provided below for the purpose of explanation only and are construed to be a part of the specification and not limiting of the present disclosure. In the drawings, like reference numerals refer to corresponding parts throughout the several views.

[0023] The present disclosure can be understood more clearly with reference to the accompanying drawings in conjunction with the following detailed description. In the drawings: Figure 1 Flowchart of one embodiment of the picture matching processing method of the present disclosure; Figure 2 Flowchart of step 101 in the picture matching processing method of the present disclosure; Figure 3 Flowchart of step 102 in the picture matching processing method of the present disclosure; Figure 4 Flowchart of step 103 in the picture matching processing method of the present disclosure; Figure 5 Flowchart of step 104 in the picture matching processing method of the present disclosure; Figure 6 Structural schematic diagram of another embodiment of the picture matching processing device of the present disclosure; Figure 7Structure diagram of another embodiment of the picture matching processing device of the present disclosure; Figure 8 Structure diagram of an electronic device provided by an illustrative embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] In the related art, in order to be able to associate the picture of the to-be-processed item with the item SKU, so as to make the item information in the system more complete, the operation personnel of the system searches for the associated item matching the picture of the to-be-processed item in the database according to the item description information embodied on the picture of the to-be-processed item, and adds the picture of the to-be-processed item to the item description information of the associated item. When there are more items missing the item pictures, the cost of adding the item pictures by manual operation is high and the efficiency is low.

[0025] In order to solve the technical problems in the related art, the embodiments of the present disclosure provide a picture matching processing method, device and storage medium. The technical scheme of the present disclosure can realize the automation of item picture association and addition, reduce the cost, and improve the efficiency.

[0026] The picture matching processing method of the embodiments of the present disclosure can be implemented by an agent. The agent refers to an agent that can perform semantic recognition on the picture of the to-be-processed item to obtain the first item description information corresponding to the picture of the to-be-processed item, and match the first item description information with the second item description information of the items of the same category in the database to determine the associated item. The agent designed in the technical scheme of the present disclosure can implement the above picture matching processing method and add the picture of the to-be-processed item to the item description information of the associated item.

[0027] The technical scheme of the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0028] Exemplary method Figure 1 Flowchart of one embodiment of the picture matching processing method of the present disclosure; the picture matching processing method can be applied on an electronic device (such as a computer system, a server), as shown in Figure 1 The picture matching processing method includes the following steps: In step 101, the picture of the to-be-processed item is subjected to semantic recognition to obtain the first item description information corresponding to the picture of the to-be-processed item.

[0029] In the embodiments of the present disclosure, the picture of the to-be-processed item can be an item picture with key item description information, for example, an item picture embedded with text information such as item brand, specification, size, model, etc. The picture of the to-be-processed item can include a background picture area and a foreground text area, the background picture area is used to show the visual effect of the item, and the foreground text area is the item description information.

[0030] The first article description information is article description information identified based on the to-be-processed article picture, and can include article name, article model, article size, article brand, article category, and the like.

[0031] For example, the first article description information is "A brand, ceramic tile, size , model 1123541".

[0032] In the embodiment of the disclosure, before performing semantic recognition on the to-be-processed article picture, the to-be-processed article picture can be preprocessed, such as denoising and contrast adjustment, to avoid interference features from affecting picture recognition.

[0033] In the embodiment of the disclosure, there can be multiple to-be-processed article pictures, such as 5000 pictures, and each to-be-processed article picture can be sequentially subjected to semantic recognition to obtain corresponding first article description information.

[0034] In step 102, based on the article classification information in the first article description information, second article description information of at least one article is obtained.

[0035] In the embodiment of the disclosure, the article classification information is information used to classify articles into different types. The article classification information can be an article category, an article brand, or an article category and an article brand.

[0036] For example, the article category can include a cabinet category, a floor category, a hanging cabinet category, and a ceramic tile category; and the article brand can include an A brand, a B brand, a C brand, and the like.

[0037] In the embodiment of the disclosure, the article classification information can be used as a screening condition to screen second article description information of articles consistent with the article classification information in the first article description information from a database.

[0038] The database is a data structure used to store description information of articles.

[0039] For example, the article classification information in the first article description information indicates that the to-be-processed article picture is a ceramic tile, and is a ceramic tile of supplier A, and then "ceramic tile, supplier A" can be used as a screening condition to obtain second article description information of articles belonging to supplier A from the database.

[0040] The second article description information is text description information of an article that has been input into the database, and can include article identification information, article name, article specification, article model, article brand, article category, and the like.

[0041] In step 103, the matching degree of the first item description information and the second item description information of any one of the at least one item is determined.

[0042] The matching degree is used to indicate the content similarity of the semantic, keyword, label and other features of the first item description information and the second item description information, and is used to determine the association strength of the to-be-processed item picture and the second item description information.

[0043] In some embodiments, the first item description information and the second item description information can be respectively converted into string vectors, and the matching degree is determined by calculating the cosine value of the vector angle.

[0044] In a specific implementation, the item attributes (item keywords) included in the first item description information can be determined first, such as that the first item description information only contains item name, item size, item brand, item category and the like, and the second item description information contains item identification information, item name, item specification, item model, item brand, item category and the like. Then, the information corresponding to the item name, item size, item brand and item category in the second item description information can be obtained, and then the item name, item size, item brand and item category in the first item description information are converted into a first vector, and the item name, item size, item brand and item category in the second item description information are converted into a second vector. Then, the cosine value of the angle between the first vector and the second vector is calculated to obtain the matching degree.

[0045] In another embodiment, the first item description information and the second item description information can also be respectively converted into semantic vectors by using a pre-trained model (such as BERT and Word2Vec), and the vector distance (i.e., the matching degree) is determined by calculating the cosine value of the angle between the vectors.

[0046] In step 104, based on the matching degree, the associated item matched with the to-be-processed item picture is determined, and the to-be-processed item picture is determined as the item picture of the associated item.

[0047] In the embodiment, the to-be-processed item picture is determined as the item picture of the associated item, which means that the to-be-processed item picture is added to the item information of the associated item, and the picture can be displayed to the user when the user searches for the item.

[0048] Through the steps 101-104, for the scenario that the picture of the to-be-processed item needs to be associated with the item SKU in the database, the picture of the to-be-processed item can be subjected to semantic recognition to obtain the first item description information corresponding to the picture of the to-be-processed item, and based on the item classification information in the first item description information, the second item description information of at least one item is obtained, then the matching degree of the first item description information and the second item description information of any one of the at least one item is determined, and then based on the matching degree, the associated item matched with the picture of the to-be-processed item is determined, and the picture of the to-be-processed item is determined as the item picture of the associated item. Thus, the technical scheme of the present disclosure can automatically realize the association between the picture of the to-be-processed item and the existing item in the database, reduce the participation of personnel, avoid the consumption of human resources, and improve the efficiency and accuracy of item picture association. Moreover, by screening the second item description information of the same item category and then determining the matching degree, the calculation amount of the matching degree can be reduced, and the calculation efficiency is further improved. In addition, by adding the item picture to the database as the item picture of the associated item, the user experience is improved, and the sales conversion rate of the item is increased.

[0049] Figure 2 The flowchart of step 101 in the picture matching processing method of the present disclosure. As shown in the above Figure 2 , on the basis of the embodiment Figure 1 , step 101 includes steps 111-112. Each step will be described below.

[0050] In step 111, based on the picture of the to-be-processed item, a model prompt word is generated.

[0051] The model prompt word (Prompt) is an instruction or information input by the user when interacting with the intelligent agent (such as a multi-modal language model in the intelligent agent), which is used to guide the intelligent agent to perform semantic recognition processing on the picture of the to-be-processed item.

[0052] For example, the model prompt word is as follows: "I will provide a picture, please help identify the item description information contained in the picture".

[0053] In step 112, a multi-modal language model is used to perform semantic recognition processing on the picture of the to-be-processed item based on the model prompt word to obtain the first item description information.

[0054] In this embodiment of the disclosure, the multimodal language model can be a large-scale artificial intelligence model capable of processing multiple types of data (such as text, images, audio, video, etc.), such as GPT-4o (Generative Pre-trained Transformer 4 Omni), Flamingo, and GPT-4V (Generative Pre-trained Transformer 4 with Vision).

[0055] In some implementations, the text in the image can be accurately extracted first using an OCR (Optical Character Recognition) module, and then semantic understanding can be performed using a multimodal language model to reduce the error of directly processing the image.

[0056] In other implementations, the OCR module can be jointly trained with a multimodal language model, so that the trained multimodal language model can both recognize the text in the image of the item to be processed and understand the semantics of the text, thereby obtaining the description information of the first item.

[0057] Based on the embodiments of this disclosure, a multimodal language model can be used to automatically identify the first item description information corresponding to the item image to be processed, which helps to determine the associated items based on the first item description information, improves the accuracy of matching item images with items, reduces manual intervention, and improves processing efficiency.

[0058] Figure 3 This is a flowchart of step 102 in the image matching processing method of this disclosure. For example... Figure 3 As shown above, in the above Figure 1 Based on the illustrated embodiment, step 102 includes steps 121-122. Each step is described below.

[0059] In step 121, the first item description information is segmented to obtain item keywords that include item classification information.

[0060] In this embodiment of the disclosure, a data analysis engine can be used to perform word segmentation processing on the first item description information. The data analysis engine is an engine used to perform word segmentation processing on the first item description information. It can be configured with a finite lexical automaton. The finite lexical automaton can recursively match the keywords in the first item description information, and then extract the item keywords and the item attributes corresponding to the item keywords in the text string.

[0061] Among them, the item keywords can include words such as brand, specifications, name, size, etc., and the item attributes corresponding to the item keywords can be the attribute values ​​corresponding to the item keywords. For example, the brand is brand A.

[0062] In step 122, the second item description information of at least one item is obtained using the item classification information.

[0063] Among them, item classification information is used to categorize items into different types. Item classification information can be item category, item brand, or both.

[0064] In this embodiment of the disclosure, the item classification information can be used as a filtering condition to filter out the second item description information of items that are consistent with the item classification information in the first item description information from the database.

[0065] Based on the embodiments of this disclosure, word segmentation can further subdivide the first item description information into corresponding item keywords and item attributes, which helps to accurately match the item keywords and item attributes when determining the associated items based on the first item description information, thereby further improving the accuracy of matching item images with items.

[0066] Figure 4 This is a flowchart of step 103 in the image matching processing method of this disclosure. For example... Figure 4 As shown above, in the above Figure 1 Based on the illustrated embodiment, step 131 includes steps 131-133. Each step is described below.

[0067] In step 131, for any item keyword, the first item attribute information corresponding to the item keyword in the first item description information and the second item attribute information corresponding to the item keyword in the second item description information are obtained.

[0068] In this embodiment, the second item description information is the text description information of items already entered into the database. When storing text description information in the database, the second attribute information corresponding to each item keyword is typically stored in the form of fields. For example, the second item description information includes an item identifier field, an item name field, an item specification field, an item model field, an item brand field, and an item category field, respectively used to store the item identifier information, item name, item specification, item model, item brand, and item category. After obtaining the first item attribute information through word segmentation, the corresponding second item attribute information can be retrieved from the second item description information based on the item keywords.

[0069] In step 132, the matching degree between the first item attribute information and the second item attribute information is determined to obtain the attribute matching degree.

[0070] In this embodiment of the disclosure, the corresponding attribute matching degree can be determined for each item attribute in the first item attribute information and each item attribute in the second item attribute information.

[0071] For example, if the first item attribute information and the second item attribute information include item name, item model, item brand, and item category, then the matching degree of the item name in the first item attribute information and the item name in the second item attribute information can be calculated to obtain the name matching degree; the matching degree of the item brand in the first item attribute information and the item brand in the second item attribute information can be calculated to obtain the brand matching degree; the matching degree of the item category in the first item attribute information and the item category in the second item attribute information can be calculated to obtain the category matching degree; and the matching degree of the item model in the first item attribute information and the item model in the second item attribute information can be calculated to obtain the model matching degree.

[0072] In step 133, the matching degree between the first item description information and the second item description information is determined based on the attribute matching degree corresponding to any item keyword.

[0073] In this embodiment of the disclosure, the weighted sum of the matching degree of each attribute can be calculated to obtain the matching degree between the first item description information and the second item description information.

[0074] In this embodiment of the disclosure, when determining the matching degree, the matching degree between the second item description information of each item with the same item classification information and the first item description information of the image of the item to be processed can be calculated separately, and multiple matching degrees can be obtained accordingly.

[0075] Based on the embodiments of this disclosure, the attribute information of the corresponding item keywords can be obtained from the second item description information through the item keywords contained in the first item description information. This enables the determination of the matching degree based on the specific attribute information contained in the image of the item to be processed, which helps to improve the effectiveness and accuracy of the determined matching degree.

[0076] Figure 5 This is a flowchart of step 104 in the image matching processing method of this disclosure. For example... Figure 5 As shown above, in the above Figure 1 Based on the illustrated embodiment, step 104 includes steps 141-145. Each step is described below.

[0077] In step 141, based on the matching degree, a reference item with the highest matching degree to the image of the item to be processed is determined from at least one item.

[0078] Among them, the higher the matching degree, the better the representation matches the image of the item to be processed. Therefore, the item with the highest matching degree can be determined from the multiple matching degrees corresponding to multiple items as the reference item.

[0079] Furthermore, if the matching degree of the reference item is greater than the first preset value, step 142 can be executed; if the matching degree of the reference item is greater than the second preset value but not greater than the first preset value, step 143 can be executed.

[0080] In step 142, in response to the matching degree of the reference item being greater than a first preset value, the reference item is determined to be an associated item.

[0081] The first preset value is a relatively high matching degree set in advance, such as 99% or 100%. The first preset value can be used to identify items that completely or nearly completely match the image of the item to be processed. Therefore, the item with the highest matching degree and a matching degree greater than the first preset value can be identified as the associated item.

[0082] In step 143, in response to the matching degree of the reference item being greater than the second preset value and not greater than the first preset value, the reference item is determined as a recommended item.

[0083] The second preset value can be a pre-set matching degree that is smaller than the first preset value, such as 85%. This second preset value can be used to identify items that are more compatible with the image of the item to be processed.

[0084] In this embodiment of the disclosure, in addition to determining the item with the highest matching degree and a matching degree greater than the second preset value but not greater than the first preset value as a recommended item, a set number of items with a matching degree greater than the second preset value but not greater than the first preset value can also be determined as recommended items.

[0085] In step 144, associated prompt information is generated.

[0086] The associated prompts include a second item description of the recommended items and an image of the item to be processed.

[0087] In this embodiment of the disclosure, after generating the association prompt information, it can be displayed on the interface in the form of a pop-up window or text, so that the user can determine whether the recommended item is an associated item of the item image to be processed by manual verification based on the association prompt information.

[0088] In step 145, a response operation triggered by the user based on the associated prompt information is received, and the recommended item is determined based on the response operation to determine whether it is an associated item that matches the image of the item to be processed.

[0089] In this embodiment of the disclosure, after the association prompt information is displayed on the interface, the user (system operator) can manually verify whether the recommended item in the association prompt information is an associated item of the image of the item to be processed. If it is determined that the recommended item is an associated item of the image of the item to be processed, the association operation is performed. If it is determined that the recommended item is not an associated item of the image of the item to be processed, the association is rejected.

[0090] Based on the embodiments of this disclosure, a specific implementation method for determining associated items based on matching degree is disclosed, and a method of determining associated items by combining manual verification when the matching degree cannot be used is disclosed, which effectively makes up for the lack of judgment of intelligent agents under fuzzy boundaries and improves the accuracy and reliability of image association.

[0091] Corresponding to the aforementioned embodiments of the image matching processing method, this disclosure also provides embodiments of the image matching processing apparatus.

[0092] Exemplary apparatus Figure 6 This is a schematic diagram illustrating the structure of one embodiment of the image matching processing apparatus of this disclosure. The image matching processing apparatus is applied to electronic devices (such as computer systems, servers), such as... Figure 6 As shown, the device includes: The recognition module 61 is used to perform semantic recognition on the image of the item to be processed, and obtain the first item description information corresponding to the image of the item to be processed; Information acquisition module 62 is used to acquire second item description information of at least one item based on item classification information in the first item description information; The matching degree determination module 63 is used to determine the matching degree between the first item description information and the second item description information of any one of the at least one items; The item identification module 64 is used to identify associated items that match the image of the item to be processed based on the matching degree, and to identify the image of the item to be processed as the image of the associated items.

[0093] Figure 7 This is a schematic diagram of another embodiment of the image matching processing apparatus of this disclosure. Figure 7 As shown, in Figure 6 Based on the illustrated embodiment, in some optional implementations, the identification module 61 may include: The classification submodule 611 is used to generate model prompts based on the images of the items to be processed; The recognition submodule 612 is used to perform semantic recognition processing on the image of the item to be processed based on the model prompt words using a multimodal language model, so as to obtain the first item description information.

[0094] In some alternative implementations, the information acquisition module 62 may include: The word segmentation submodule 621 is used to segment the first item description information into words to obtain item keywords that include item classification information. The first acquisition submodule 622 is used to acquire second item description information of at least one item using item classification information.

[0095] In some alternative implementations, the matching degree determination module 63 may include: The second acquisition submodule 631 is used to acquire, for any item keyword, the first item attribute information corresponding to the item keyword in the first item description information, and the second item attribute information corresponding to the item keyword in the second item description information; The first determining submodule 632 is used to determine the matching degree between the first item attribute information and the second item attribute information, and to obtain the attribute matching degree. The second determining submodule 633 is used to determine the matching degree between the first item description information and the second item description information based on the attribute matching degree corresponding to any item keyword.

[0096] In some alternative implementations, the item determination module 64 may include: The third determination submodule 641 is used to determine, based on the matching degree, the reference item with the highest matching degree with the image of the item to be processed from at least one item; The fourth determination submodule 642 is used to determine the reference item as an associated item in response to the matching degree of the reference item being greater than the first preset value.

[0097] In some alternative implementations, the item determination module 64 may include: The fifth determination submodule 643 is used to determine the reference item as a recommended item in response to the matching degree of the reference item being greater than the second preset value and not greater than the first preset value; The prompt submodule 644 is used to generate associated prompt information, which includes a second item description of the recommended item and an image of the item to be processed. The receiving submodule 645 is used to receive response operations triggered by the user based on associated prompt information; The sixth determination submodule 646 is used to determine, based on the response operation, whether the recommended item is an associated item that matches the image of the item to be processed.

[0098] The modules and units in this disclosed device can be further divided into finer-grained units according to actual needs, and the specific configuration can be set according to actual needs.

[0099] The apparatus of this disclosure embodiment can be used to implement the methods of the above embodiments of this disclosure. The two correspond to each other in specific implementation, and the specific implementation of related parts can be referred to each other, which will not be repeated here.

[0100] Exemplary electronic device, computer program product, and computer-readable storage medium This disclosure also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program stored in the memory, wherein when the computer program is executed, it implements the image matching processing method of any of the above embodiments of this disclosure.

[0101] Below, for reference Figure 8 This describes an electronic device according to embodiments of the present disclosure, wherein apparatus for implementing methods according to embodiments of the present disclosure may be integrated. Figure 8 This is a structural diagram of an electronic device provided in an illustrative embodiment of the present disclosure, such as... Figure 8 As shown, the electronic device includes one or more processors 81, one or more memory 82s of computer-readable storage media, and a computer program stored in the memory and executable on the processor. When the program in the memory 82 is executed, the image matching processing method described above can be implemented.

[0102] Specifically, in practical applications, the electronic device may also include components such as an input device 83 and an output device 84, which are interconnected via a bus system and / or other forms of connection mechanisms (not shown). Those skilled in the art will understand that... Figure 8 The structure of the electronic device shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or certain components, or different component arrangements. Wherein: The processor 81 may be a central processing unit (CPU) or other processing unit with image matching processing capability and / or instruction execution capability. It performs various functions and processes data by running or executing software programs and / or modules stored in memory 82 and calling data stored in memory 82, thereby performing overall monitoring of the electronic device.

[0103] The memory 82 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 81 may execute the program instructions to implement the image matching processing methods of the various embodiments of this disclosure described above and / or other desired functions. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.

[0104] The input device 83 can be used to receive input digital or character information, and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0105] The output device 84 can output various information to the outside, including determined distance information, direction information, etc. The output device 84 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0106] Electronic devices may also include a power supply for powering various components, which can be logically connected to the processor 81 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply may also include one or more DC or AC power sources, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and any other components.

[0107] Of course, for the sake of simplicity, Figure 8 Only some of the components of the electronic device relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device may include any other suitable components depending on the specific application.

[0108] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products, including computer program instructions that, when executed by a processor, cause the processor to perform the steps in the image matching processing methods according to various embodiments of this disclosure as described in the "Exemplary Methods" section of this specification.

[0109] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0110] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the image matching processing methods according to various embodiments of this disclosure as described in the "Exemplary Methods" section above.

[0111] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0112] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0113] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0114] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as ROM, RAM, magnetic disk, or optical disk.

[0115] The methods and apparatus of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the method is for illustrative purposes only, and the steps of the method of this disclosure are not limited to the order specifically described above, unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the method according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the method according to this disclosure.

[0116] The description in this disclosure is provided for illustrative and descriptive purposes only and is not intended to be exhaustive or to limit the disclosure to its forms. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described in order to better illustrate the principles and practical application of this disclosure and to enable those skilled in the art to understand this disclosure and to design various embodiments with various modifications suitable for a particular purpose.

Claims

1. An image matching processing method, characterized in that, include: Semantic recognition is performed on the image of the item to be processed to obtain the first item description information corresponding to the image of the item to be processed. Based on the item classification information in the first item description information, obtain the second item description information of at least one item; Determine the degree of matching between the first item description information and the second item description information of any one of the at least one items; Based on the matching degree, associated items that match the image of the item to be processed are determined, and the image of the item to be processed is identified as the image of the associated items.

2. The method according to claim 1, characterized in that, The semantic recognition of the image of the item to be processed is performed to obtain the first item description information corresponding to the image of the item to be processed, including: Based on the image of the item to be processed, generate model prompts; Using a multimodal language model, semantic recognition processing is performed on the image of the item to be processed based on the model prompt words to obtain the description information of the first item.

3. The method according to any one of claims 1-2, characterized in that, The step of obtaining second item description information for at least one item based on item classification information in the first item description information includes: The first item description information is segmented into words to obtain item keywords that include the item classification information; Using the item classification information, obtain the second item description information of the at least one item.

4. The method according to claim 3, characterized in that, Determining the matching degree between the first item description information and the second item description information of any one of the at least one items includes: For any of the item keywords, obtain the first item attribute information corresponding to the item keyword in the first item description information, and the second item attribute information corresponding to the item keyword in the second item description information; Determine the matching degree between the first item attribute information and the second item attribute information to obtain the attribute matching degree; Based on the attribute matching degree corresponding to any of the item keywords, the matching degree between the first item description information and the second item description information is determined.

5. The method according to any one of claims 1-4, characterized in that, The step of determining associated items that match the image of the item to be processed based on the matching degree includes: Based on the matching degree, a reference item with the highest matching degree with the image of the item to be processed is determined from the at least one item; In response to the fact that the matching degree of the reference item is greater than a first preset value, the reference item is determined to be an associated item.

6. The method according to claim 5, characterized in that, After determining the reference item with the highest matching degree to the image of the item to be processed from the at least one item, the process includes: In response to the fact that the matching degree of the reference item is greater than the second preset value and not greater than the first preset value, the reference item is determined to be a recommended item. Generate associated prompt information, which includes a second item description of the recommendable item and an image of the item to be processed; Receive a response operation triggered by the user based on the associated prompt information, and determine whether the recommended item is an associated item that matches the image of the item to be processed based on the response operation.

7. An image matching processing device, characterized in that, include: The recognition module is used to perform semantic recognition on the image of the item to be processed, and obtain the first item description information corresponding to the image of the item to be processed. The information acquisition module is used to acquire second item description information of at least one item based on the item classification information in the first item description information; A matching degree determination module is used to determine the matching degree between the first item description information and the second item description information of any one of the at least one items; The item identification module is used to identify associated items that match the image of the item to be processed based on the matching degree, and to identify the image of the item to be processed as the image of the associated items.

8. A computer-readable storage medium storing computer program instructions that, when executed, implement the method described in any one of claims 1-6.

9. An electronic device, the electronic device comprising: Memory, used to store computer program products; A processor is configured to execute a computer program product stored in the memory, wherein, when the computer program product is executed, it implements the method described in any one of claims 1-6.

10. A computer program product comprising computer program instructions, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1-6.

Images