Material retrieval method and device, computing equipment and computer storage medium

Abstract

The invention discloses a material retrieval method and device, computing equipment and a computer storage medium. The method comprises the following steps: receiving retrieval input data provided by a user; performing entity recognition processing on the retrieval input data to obtain an entity retrieval tag; searching a target entity node matched with the entity retrieval tag in a material knowledge graph; wherein the material knowledge graph is constructed according to materials in a material library; and screening materials contained in the target entity node to obtain a material retrieval result. Through the mode, the semantic association between the user input content and the material can be deeply understood, the accuracy and efficiency of material retrieval can be improved, meanwhile, a multi-modal input mode can be supported, the user does not need to provide complex and accurate keywords for material retrieval, the retrieval operation of the user can be simplified, and the user experience is improved. A wider material matching capability is realized, and cross-modal material retrieval can also be realized.

Description

Technical Field

[0001] This application relates to the field of retrieval technologies, and in particular, to a method and apparatus for retrieving materials, a computing device, a computer storage medium, and a computer program product. Background Art

[0002] Retrieval refers to the process of searching for and extracting specific content from a large amount of information or data. The application scenarios of retrieval technologies are extensive, including traditional search engines and database queries, as well as emerging fields such as personalized recommendations and intelligent customer service. For example, in a multimedia content platform, when a user creates multimedia content, the user can retrieve the required materials from a material library for content creation.

[0003] Traditional material retrieval methods mainly rely on keyword search, and determine retrieval results by calculating the matching degree between keywords and materials. This method cannot deeply understand the semantic relationship between the user input and the materials, and cannot guarantee the accuracy of the retrieval results. At the same time, retrieval results with a high matching degree rely on accurate retrieval terms, which requires a relatively high retrieval ability from users. Summary of the Invention

[0004] In view of the above problems, this application is proposed to provide a method and apparatus for retrieving materials, a computing device, a computer storage medium, and a computer program product that overcome the above problems or at least partially solve the above problems.

[0005] According to one aspect of this application, there is provided a method for retrieving materials, including:

[0006] Receiving retrieval input data provided by a user;

[0007] Performing entity recognition processing on the retrieval input data to obtain an entity retrieval label;

[0008] Searching for a target entity node in a material knowledge graph that matches the entity retrieval label; wherein, the material knowledge graph is constructed based on materials in a material library;

[0009] Screening the materials included in the target entity node to obtain a material retrieval result.

[0010] Optionally, the entity nodes of the material knowledge graph include materials and material labels, and the material knowledge graph is used to record the association relationships between materials and the association relationships between materials and material labels.

[0011] Optionally, further including, when searching for a target entity node in a material knowledge graph that matches the entity retrieval label:

[0012] Matching the entity retrieval label with the entity nodes in the material knowledge graph to determine a first entity node that is the same as and / or similar to the entity retrieval label;

[0013] Determine a second entity node that is associated with the first entity node in the material knowledge graph and whose distance is within a preset distance range;

[0014] Determine the first entity node and the second entity node as target entity nodes.

[0015] Optionally, screening the materials included in the target entity nodes to obtain a material retrieval result further includes:

[0016] Perform sentiment analysis processing on the retrieval input data to obtain a sentiment retrieval label;

[0017] According to the sentiment retrieval label, screen out target materials from the materials included in the target entity nodes to obtain a material retrieval result.

[0018] Optionally, after obtaining the material retrieval result, the method further includes:

[0019] Perform feature extraction processing on the retrieval input data to obtain retrieval input features;

[0020] Calculate the matching degree between the retrieval input features and the material features of each target material included in the material retrieval result;

[0021] Sort each target material according to the matching degree.

[0022] Optionally, the retrieval input features include input key features in multiple dimensions, and the material features include material key features in multiple dimensions; calculating the matching degree between the retrieval input features and the material features of each target material included in the material retrieval result further includes:

[0023] For each dimension, calculate the similarity between the input key feature of this dimension and the material key feature of this dimension of the target material;

[0024] Perform weighted fusion according to the similarities and weights corresponding to multiple dimensions to obtain the matching degree between the retrieval input features and the material features of the target material.

[0025] Optionally, the multiple dimensions include multiple of the following dimensions: semantic dimension, sentiment dimension, visual dimension.

[0026] According to another aspect of the present application, there is provided a material retrieval device, including:

[0027] A receiving module, adapted to receive retrieval input data provided by a user;

[0028] An identification module, adapted to perform entity identification processing on the retrieval input data to obtain an entity retrieval label;

[0029] A retrieval module, adapted to find target entity nodes in a material knowledge graph that match entity retrieval tags; wherein, the material knowledge graph is constructed based on materials in a material library.

[0030] A screening module, adapted to screen the materials included in the target entity nodes to obtain a material retrieval result.

[0031] Optionally, the entity nodes of the material knowledge graph include materials and material tags, and the material knowledge graph is used to record the association relationships between materials and the association relationships between materials and material tags.

[0032] Optionally, the retrieval module is further adapted to:

[0033] Match the entity retrieval tags with the entity nodes in the material knowledge graph to determine first entity nodes that are the same as and / or similar to the entity retrieval tags;

[0034] Determine second entity nodes that are associated with the first entity nodes in the material knowledge graph and whose distance is within a preset distance range;

[0035] Determine the first entity nodes and the second entity nodes as the target entity nodes.

[0036] Optionally, the screening module is further adapted to:

[0037] Perform sentiment analysis processing on the retrieval input data to obtain sentiment retrieval tags;

[0038] According to the sentiment retrieval tags, screen out target materials from the materials included in the target entity nodes to obtain a material retrieval result.

[0039] Optionally, the device further includes:

[0040] A sorting module, adapted to perform feature extraction processing on the retrieval input data to obtain retrieval input features; calculate the matching degrees between the retrieval input features and the material features of each target material included in the material retrieval result; sort the target materials according to the matching degrees.

[0041] Optionally, the retrieval input features include input key features in multiple dimensions, and the material features include material key features in multiple dimensions;

[0042] The sorting module is further adapted to: for each dimension, calculate the similarity between the input key feature of this dimension and the material key feature of this dimension of the target material; perform weighted fusion according to the similarities and weights corresponding to multiple dimensions to obtain the matching degree between the retrieval input features and the material features of the target material.

[0043] Optionally, the multiple dimensions include multiple of the following dimensions: semantic dimension, sentiment dimension, visual dimension.

[0044] According to another aspect of the present application, a computing device is provided, including: a processor, a memory, a communication interface, and a communication bus, and the processor, the memory, and the communication interface complete communication with each other through the communication bus;

[0045] The memory is used to store at least one executable instruction, and the executable instruction causes the processor to execute the operations corresponding to the above-mentioned material retrieval method.

[0046] According to yet another aspect of the present application, a computer storage medium is provided, in which at least one executable instruction is stored, and the executable instruction causes the processor to execute the operations corresponding to the above-mentioned material retrieval method.

[0047] According to still another aspect of the present application, a computer program product is provided, including at least one executable instruction, and the executable instruction causes the processor to execute the operations corresponding to the above-mentioned material retrieval method.

[0048] According to the material retrieval method, device, computing device, computer storage medium, and computer program product provided by the embodiments of the present application, receive the retrieval input data provided by the user; perform entity recognition processing on the retrieval input data to obtain an entity retrieval tag; search for a target entity node in the material knowledge graph that matches the entity retrieval tag; wherein, the material knowledge graph is constructed according to the materials in the material library; screen the materials included in the target entity node to obtain a material retrieval result. By the above method, it is possible to deeply understand the semantic association between the user input content and the materials, improve the accuracy and efficiency of material retrieval, and at the same time, support a multi-modal input mode, without the user providing complex and accurate keywords for material retrieval, which can simplify the user's retrieval operation, achieve a wider material matching ability, and also achieve cross-modal material retrieval.

[0049] The above description is only an overview of the technical solution of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features, and advantages of the present application more obvious and understandable, the specific embodiments of the present application are specifically described below. Description of the Drawings

[0050] By reading the detailed description of the preferred embodiments below, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present application. And throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0051] Figure 1The flowchart of the material retrieval method provided by an embodiment of the present application is shown;

[0052] Figure 2 The flowchart of the material retrieval method provided by another embodiment of the present application is shown;

[0053] Figure 3 The functional structure diagram of the material retrieval device provided by an embodiment of the present application is shown;

[0054] Figure 4 The structural diagram of the computing device provided by an embodiment of the present application is shown. Detailed implementation manners

[0055] Hereinafter, the exemplary embodiments of the present application will be described in more detail with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present application can be more thoroughly understood and the scope of the present application can be fully conveyed to those skilled in the art.

[0056] First, the noun terms related to one or more embodiments of the present application are explained.

[0057] Multimodal input: capable of receiving and processing input signals from different types of data sources, including but not limited to information such as text, audio, video, and pictures.

[0058] Material library: a database storing various digital materials, including videos, audios, pictures, special effects, filters, transitions, stickers, etc., providing resources for users during the creation process.

[0059] Feature extraction: extracting representative features from input data through technologies such as computer vision, speech recognition, and natural language processing.

[0060] Entity recognition: a core technology in the field of natural language processing (NLP), aiming to automatically identify entities with specific meanings from unstructured text and classify them. These entities can include various types such as personal names, place names, organizations, date and time, currency amounts, etc.

[0061] Knowledge graph: a graphical data structure used to represent and organize knowledge, which describes concepts in the real world and their associations through nodes (corresponding to entities) and edges (corresponding to relationships).

[0062] Figure 1 The flowchart of the material retrieval method provided by an embodiment of the present application is shown. As Figure 1 shown, the method includes the following steps:

[0063] Step S110: Receive the retrieval input data provided by the user.

[0064] In the method of the embodiments of the present application, material retrieval is achieved by semantically matching the entity tags of the retrieval input data with the material knowledge graph. Therefore, there is no restriction on the type of the retrieval input data, and the user is supported to input multimodal data for material retrieval. That is, the user can provide any type of data such as text, video, audio, and pictures for material retrieval, or can also provide multiple types of data for material retrieval at the same time.

[0065] Step S120: Perform entity recognition processing on the retrieval input data to obtain entity retrieval tags.

[0066] Perform entity recognition on the retrieval input data to identify entities with specific meanings therein, such as people, places, times, emotions, scenes, actions, etc. The identified entities are used as retrieval tags, and the entity retrieval tags are used to retrieve matching materials.

[0067] Among them, for different types of retrieval input data, the entity recognition technologies adopted are inconsistent. For text, natural language processing technology can be used to identify entities; for audio, an audio analysis model can be used to identify entities; for video, image recognition technology can be used to identify entities for each video frame image included therein; for pictures, image recognition technology can be used to identify entities.

[0068] Step S130: Search for target entity nodes in the material knowledge graph that match the entity retrieval tags.

[0069] Among them, the material knowledge graph is constructed based on the materials in the material library. The material knowledge graph is a semantic network representing the materials in the material library and their features, classifications, applicable scenarios, etc., and is constructed through the structure of "entity-relationship-entity" or "entity-attribute-attribute value", and can support efficient semantic retrieval and recommendation. Specifically, the material knowledge graph is constructed by analyzing the material tags in multiple dimensions of the materials and analyzing the relationships between the materials.

[0070] Match the entity retrieval tags with the entity nodes in the material knowledge graph to determine the target entity nodes that match the entity retrieval tags. The target entity nodes may be materials or may also be material tags.

[0071] Step S140: Screen the materials included in the target entity nodes to obtain the material retrieval results.

[0072] For the materials matched from the material knowledge graph according to the entity retrieval tags, a preset screening algorithm is further adopted for fine screening, such as further introducing factors such as feature similarity and emotion for fine screening to obtain the final material retrieval results.

[0073] In the existing keyword-based material retrieval method, users need to input keywords for retrieval. However, this method has many limitations. First, the keyword retrieval method requires users to input accurate descriptive words. At the same time, it requires users to accurately understand the materials and precisely select keywords, which has relatively high requirements for users. The material retrieval results usually fail to meet the needs of users. Second, the search efficiency is low. As the resources in the material library gradually become large, the method of users retrieving by trying keywords one by one is not only time-consuming but also difficult to obtain accurate retrieval results. Third, the matching degree of the retrieval results is not high. It cannot deeply and accurately understand the specific needs of users and cannot perform multi-level and detailed matching based on the keywords input by users.

[0074] In summary, according to the material retrieval method provided in this embodiment, receive the retrieval input data provided by the user; perform entity recognition processing on the retrieval input data to obtain an entity retrieval label; search for a target entity node in the material knowledge graph that matches the entity retrieval label, where the material knowledge graph is constructed based on the materials in the material library; screen the materials included in the target entity node to obtain the material retrieval result. Through the above method, a material retrieval method based on a knowledge graph is realized, which can deeply understand the semantic relationship between the content input by the user and the materials, can improve the accuracy and efficiency of material retrieval. At the same time, the retrieval method based on semantic understanding does not limit the type of retrieval input data, can support multi-modal input modes, does not require users to provide complex and accurate keywords for material retrieval, can simplify the user's retrieval operation, realizes a wider material matching ability, and can also realize cross-modal material retrieval.

[0075] Figure 2 The flowchart of the material retrieval method provided in another embodiment of the present application is shown. As Figure 2 shown, the method includes the following steps:

[0076] Step S210, receive the retrieval input data provided by the user.

[0077] Provide a retrieval entry for the user, and the user uploads the retrieval input data through the retrieval entry.

[0078] In the method of the embodiment of the present application, material retrieval is realized by semantically matching the entity label of the retrieval input data with the material knowledge graph. Therefore, the type of the retrieval input data is not limited, and it supports users to input multi-modal data for material retrieval. That is, users can provide any type of data such as text, video, audio, and pictures for material retrieval, or can also provide multiple types of data for material retrieval at the same time.

[0079] Specifically, the ways for users to provide retrieval input data of text type include: uploading pure text data and text files; the ways for users to provide retrieval input data of audio type include: uploading audio files and inputting real-time voice; the ways for users to provide retrieval input data of video type include: uploading video files (such as MP4 files, AVI files) and uploading real-time streaming media.

[0080] Step S220: Perform entity recognition processing on the retrieval input data to obtain entity retrieval tags.

[0081] Perform entity recognition on the retrieval input data to identify entities with specific meanings therein, such as people, locations, times, emotions, scenes, actions, etc., and use the identified entities as retrieval tags for retrieving materials.

[0082] In the case where the user provides multiple retrieval input data, entity recognition processing is performed on each retrieval input data, and the obtained entity retrieval tags are all used for retrieving materials.

[0083] Step S230: Match the entity retrieval tags with the entity nodes in the material knowledge graph to determine the first entity nodes that are the same as and / or similar to the entity retrieval tags; determine the second entity nodes in the material knowledge graph that are associated with the first entity nodes and whose distance is within a preset distance range; and determine the first entity nodes and the second entity nodes as target entity nodes.

[0084] All materials and their characteristics in the material library are pre-modeled into a material knowledge graph. That is, the entity nodes of the material knowledge graph include materials and material tags. The material knowledge graph is used to record the association relationships between materials and the association relationships between materials and material tags, and the material tags are determined according to the characteristics of the materials.

[0085] The elements included in the material knowledge graph are: entity nodes, relationships, attributes, and attribute values; the entity nodes include materials and material tags, such as: video materials, audio materials, picture materials, scene tags, style tags, emotion tags, etc. The relationships include the relationships between materials and the relationships between materials and material tags, such as: similar to, belonging to a category, suitable for matching, scene association, etc. The attributes and attribute values refer to the attribute information of the materials themselves, such as: resolution - 4K, style - natural, duration - 30 seconds, etc. For example, the association relationships recorded by the material knowledge graph between entity nodes are: Video A is similar to Video B, the category to which Video A belongs is natural scenery, Audio C is suitable for matching Video D, and the scene association of Audio A is a forest.

[0086] Match the physical retrieval tag with the entity nodes in the material knowledge graph to determine the entity nodes in the material knowledge graph that are similar and / or identical to the physical retrieval tag, that is, obtain the first entity nodes; then, based on the material knowledge graph, search for the second entity nodes that are associated with the first entity nodes and whose distance is within the preset distance range. Since the second entity nodes are associated with the first entity nodes, several materials can be determined as alternatives according to the second entity nodes to expand the scope of the retrieval results. For example, select the entity nodes that are "one-hop" away from the first entity node and have an association relationship with the first entity node as the second entity nodes; the first entity nodes and the second entity nodes together constitute the target entity nodes that match the physical retrieval tag, and the material retrieval results are selected according to the target entity nodes in the subsequent steps.

[0087] Step S240, screen the materials included in the target entity nodes to obtain the material retrieval results.

[0088] For the materials matched from the material knowledge graph, a preset screening algorithm is further adopted for screening. In the method of the embodiment of the present application, emotional factors are introduced for fine screening to obtain the final material retrieval results.

[0089] Specifically, perform sentiment analysis processing on the retrieval input data to obtain a sentiment retrieval tag; according to the sentiment retrieval tag, screen out the target materials from the materials included in the target entity nodes to obtain the material retrieval results. Perform sentiment analysis on the retrieval input data to obtain a sentiment retrieval tag, and then compare the sentiment tags of the materials included in the target entity nodes with the sentiment retrieval tag, and retain the target materials whose sentiment tags match the sentiment retrieval tag, and filter out the materials whose sentiment tags do not match the sentiment retrieval tag. Specifically, read the sentiment tags of the materials through the material knowledge graph.

[0090] Among them, the sentiment tag of the material matching the sentiment retrieval tag means that the two are the same, and it can also mean that the sentiment retrieval tag is the lower-level sentiment tag or the upper-level sentiment tag of the sentiment tag of the material. For example, the sentiment tag - "angry" is a lower-level sentiment tag of the sentiment tag - "negative", and the sentiment tag - "negative" is an upper-level sentiment tag of the sentiment tag - "angry". Then, when the sentiment retrieval tag is "angry" or "negative", and the sentiment tag of the material is "negative" or "angry", it is considered that the sentiment retrieval tag matches the sentiment tag of the material.

[0091] Through the above method, the sentiment tags of the retrieval input data are extracted, and the semantic matching results are further optimized by using the sentiment tags, so that the sentiment contained in the retrieved materials better meets the user's retrieval requirements.

[0092] Step S250: Perform feature extraction processing on the retrieved input data to obtain retrieved input features; calculate the matching degrees between the retrieved input features and the material features of each target material included in the material retrieval results; and sort the various target materials according to the matching degrees.

[0093] After semantic retrieval based on the material knowledge graph and matching optimization based on sentiment tags, further determine the matching degree between the retrieved input data and the retrieval results as a sorting factor for sorting the various target materials included in the material retrieval results.

[0094] In an alternative approach, perform feature extraction processing on the retrieved input data from multiple dimensions, that is, the retrieved input features include input key features in multiple dimensions. Correspondingly, the material features of the materials also include material key features in multiple dimensions. The calculation method for the matching degree between the retrieved input features and the material features is as follows: for each dimension, calculate the similarity between the input key feature of this dimension and the material key feature of the target material in this dimension; perform weighted fusion according to the similarities and weights corresponding to multiple dimensions to obtain the matching degree between the retrieved input features and the material features of the target material. That is, calculate the similarity between the input key feature of each dimension and the material key feature respectively, and then calculate the weighted sum according to the similarities and weight values of each dimension to obtain the sorting factor, that is, the matching degree.

[0095] In an alternative approach, multiple dimensions include multiple of the following dimensions: semantic dimension, sentiment dimension, and visual dimension. Among them, for all modal data, use a deep learning model to determine the sentiment tendency, such as positive, negative, and neutral, etc., and then determine the features of the sentiment dimension. For example, for audio data, features such as rhythm, pitch, and timbre can be extracted to determine the sentiment tendency. The features of the semantic dimension can include: keywords, entities (including people, places, times, etc.), themes, semantic scenes, context information, etc. The features of the semantic dimension can be extracted using natural language techniques. For audio data, the speech content can be extracted first through speech recognition technology, and then the features of the semantic dimension can be extracted using natural language techniques. The features of the visual dimension can include: visual scenes, objects, colors, textures, etc. Extract the visual features in videos and images through image recognition technology. For the retrieved input data and the materials in the material library, the same technology can be used to extract the features of each dimension respectively.

[0096] Among them, the calculation method of the matching degree is expressed as follows:

[0097] S = ω 1 × S semantic + ω 2 × S emotional + ω 3 × S visual ;

[0098] Among them, S is the matching degree between the retrieval input feature and the material feature, ω 1 is the weight of the semantic dimension, S smantic is the similarity of the semantic dimension, ω 2 is the weight of the emotion dimension, S emotional is the similarity of the emotion dimension, ω 3 is the weight of the visual dimension, S visual is the similarity of the visual dimension.

[0099] It should be noted that when the retrieval input data is a video or a picture, the retrieval input feature of the visual dimension can be extracted, and when the target material is a video or a picture, the matching degree corresponding to the visual dimension can be calculated.

[0100] In an alternative manner, for the input video data and the video materials in the material library, they are first split into video frame images, and the feature extraction of the semantic dimension, the feature extraction of the emotion dimension, and the feature extraction of the visual dimension are performed on all or part of the video frame images (such as key video frame images), forming a time-series feature data body, which contains features of multiple dimensions corresponding to each video frame image.

[0101] Specifically, the retrieval input feature is converted into a vector representation to obtain a retrieval input feature vector; for the materials in the material library, their material features are converted into vector representations in the same way to obtain material feature vectors; then, the matching degree between the retrieval input feature vector and the material feature vector is calculated.

[0102] For the semantic dimension, the extracted semantic information is converted into a semantic vector through a language model, and the language model can be a BERT model or a Word2Vec model; for the emotion dimension, the extracted emotion information is encoded into an emotion vector. For example, the encoding 1 represents a positive emotion, the encoding "0.5" represents a negative emotion, and the encoding "0.2" represents a neutral emotion; for the visual dimension, visual features are extracted from the image using a convolutional neural network (CNN), and the visual features are converted into a vector representation.

[0103] In an alternative manner, the cosine similarity between the semantic vector of the retrieval input data and the semantic vector of the target material is calculated to obtain the similarity of the semantic dimension; the Euclidean distance between the emotion encoding of the retrieval input data and the emotion encoding of the target material is calculated to obtain the similarity of the emotion dimension; the dot product result or the embedding space distance between the visual vector of the retrieval input data and the visual vector of the target material is calculated to obtain the similarity of the visual dimension.

[0104] Specifically, sort each target material in descending order of matching degree, so as to arrange the materials with high matching degree to the user's needs in a more forward position, and then preferentially display the materials with high matching degree to the user.

[0105] In another alternative manner, receive a weight configuration instruction for a target dimension among multiple dimensions and adjust the weight of the target dimension. Among them, the weight configuration instruction can be triggered in response to a user operation, that is, the user can manually adjust the dimension weight to meet the user's personalized needs; the weight configuration instruction can also be dynamically adjusted according to the type and needs of the user input. For example, if the user mainly retrieves by inputting videos, the weight of the visual dimension can be increased so that the weight configuration conforms to the user's retrieval habit.

[0106] In another alternative manner, for the retrieved input data and target materials, if the feature similarity of a certain dimension reaches a preset similarity threshold, then increase the sorting priority of the target material; optionally, multiply by a boosting factor greater than 1 on the basis of the matching degree, and determine the calculation result as the sorting factor.

[0107] Step S260, return each sorted target material to the user.

[0108] Send each sorted target material to the client, and the client displays each sorted target material in the form of a list.

[0109] In another alternative manner, select the top N (N>1) target materials among the sorted target materials and return them to the user.

[0110] In an alternative manner, return each sorted target material and the associated information of each target material to the client, and the client displays each target material and its associated information in association. The associated information of the target material includes: material preview information (thumbnail, video clip), semantic overview, and / or sentiment description, etc., to facilitate the user to select the required material from the retrieval result list.

[0111] In summary, according to the material retrieval method provided in this embodiment, by modeling materials, information such as their characteristics, and the associations between materials as a material knowledge graph, and performing semantic matching in the material knowledge graph based on the entity retrieval tags of the retrieval input data, it is possible to deeply understand the semantic association between the user input content and the materials, thereby achieving accurate retrieval based on semantics and improving the accuracy of material retrieval; further, the retrieval method based on semantic understanding does not limit the type of retrieval input data, supports multi-modal input modes, does not require users to provide complex and accurate keywords for material retrieval, simplifies the user retrieval operation, and realizes a wider material matching ability; further, by extracting entities from different modal data and performing matching in combination with the semantic relationships in the material knowledge graph, it is possible to identify the semantic associations between different modal data, thereby realizing cross-modal material retrieval; further, in addition to semantic matching based on entity tags, sentiment analysis is also performed on the retrieval input data, and the semantic matching results are filtered and screened in combination with sentiment tags to ensure that the sentiment tendency of the material retrieval results matches the user needs, which can further improve the accuracy of the material retrieval results; further, for the obtained material retrieval results, the matching degrees are comprehensively calculated and sorted by integrating features from multiple dimensions, and the sorted material retrieval results are returned to the user to help the user quickly find the required materials and improve the user retrieval experience.

[0112] Figure 3 FIG. shows a functional structure diagram of a material retrieval device provided in an embodiment of the present application. As Figure 3 shown, the device includes:

[0113] A receiving module 31, adapted to receive retrieval input data provided by a user;

[0114] An identification module 32, adapted to perform entity identification processing on the retrieval input data to obtain entity retrieval tags;

[0115] A retrieval module 33, adapted to find target entity nodes in the material knowledge graph that match the entity retrieval tags; wherein, the material knowledge graph is constructed according to the materials in the material library;

[0116] A screening module 34, adapted to screen the materials included in the target entity nodes to obtain material retrieval results.

[0117] In an optional manner, the entity nodes of the material knowledge graph include materials and material tags, and the material knowledge graph is used to record the association relationships between materials and the association relationships between materials and material tags.

[0118] In an optional manner, the retrieval module 33 is further adapted to:

[0119] Match the physical entity retrieval tag with the entity nodes in the material knowledge graph to determine the first entity nodes that are the same as and / or similar to the physical entity retrieval tag;

[0120] Determine the second entity nodes in the material knowledge graph that are associated with the first entity nodes and whose distance is within a preset distance range;

[0121] Determine the first entity nodes and the second entity nodes as the target entity nodes.

[0122] In an alternative manner, the screening module 34 is further adapted to:

[0123] Perform sentiment analysis processing on the retrieval input data to obtain a sentiment retrieval tag;

[0124] According to the sentiment retrieval tag, screen out the target materials from the materials included in the target entity nodes to obtain a material retrieval result.

[0125] In an alternative manner, the device further includes:

[0126] A sorting module, adapted to perform feature extraction processing on the retrieval input data to obtain retrieval input features; calculate the matching degree between the retrieval input features and the material features of each target material included in the material retrieval result; sort the target materials according to the matching degree.

[0127] In an alternative manner, the retrieval input features include input key features in multiple dimensions, and the material features include material key features in multiple dimensions;

[0128] The sorting module is further adapted to: for each dimension, calculate the similarity between the input key feature of this dimension and the material key feature of this dimension of the target material; perform weighted fusion according to the similarities and weights corresponding to multiple dimensions to obtain the matching degree between the retrieval input features and the material features of the target material.

[0129] In an alternative manner, the multiple dimensions include multiple of the following dimensions: semantic dimension, sentiment dimension, visual dimension.

[0130] In summary, according to the material retrieval device provided in this embodiment, a material retrieval method based on a knowledge graph is realized, which can deeply understand the semantic association between the user input content and the materials, can improve the accuracy and efficiency of material retrieval. At the same time, the retrieval method based on semantic understanding does not limit the type of retrieval input data, can support a multi-modal input mode, does not require the user to provide complex and accurate keywords for material retrieval, can simplify the user retrieval operation, realizes a wider material matching ability, and can also realize cross-modal material retrieval.

[0131] An embodiment of the present application provides a non-volatile computer storage medium, which stores at least one executable instruction or computer program, and the executable instruction or computer program can cause a processor to perform operations corresponding to the material retrieval method in any of the above method embodiments.

[0132] An embodiment of the present application provides a computer program product, which includes at least one executable instruction or computer program, and the executable instruction or computer program can cause a processor to perform operations corresponding to the material retrieval method in any of the above method embodiments.

[0133] Figure 4 The structure diagram of a computing device provided by an embodiment of the present application is shown. The specific implementation of the computing device is not limited in the specific embodiments of the present application.

[0134] As Figure 4 shown, the computing device may include: a processor 402, a communication interface 404, a memory 406, and a communication bus 408.

[0135] Among them: the processor 402, the communication interface 404, and the memory 406 communicate with each other through the communication bus 408. The communication interface 404 is used to communicate with network elements of other devices such as clients or other servers. The processor 402 is used to execute the program 410, and specifically can execute relevant steps in the above embodiment of the material retrieval method for the computing device.

[0136] Specifically, the program 410 may include program code, and the program code includes computer operation instructions.

[0137] The processor 402 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application. One or more processors included in the computing device may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0138] The memory 406 is used to store the program 410. The memory 406 may include a high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

[0139] The program 410 can be specifically used to cause the processor 402 to execute the material retrieval method in any of the above method embodiments. For the specific implementation of each step in the program 410, reference can be made to the corresponding steps and descriptions in the material retrieval embodiments, which will not be elaborated here. Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the above-described devices and modules can refer to the corresponding process descriptions in the foregoing method embodiments, which will not be repeated here.

[0140] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings provided herein. The structure required to construct such systems will be apparent from the above description. In addition, the embodiments of the present application are not directed to any specific programming language. It should be understood that the content of the present application described herein can be implemented using various programming languages, and the description of the specific language above is to disclose the best mode of the present application.

[0141] In the specification provided herein, a large number of specific details are set forth. However, it can be understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures, and technologies have not been shown in detail so as not to obscure the understanding of this specification.

[0142] Similarly, it should be understood that in order to streamline the present application and assist in understanding one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, the various features of the embodiments of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be construed as reflecting the intention that the claimed subject matter of the present application requires more features than are expressly recited in each claim. Rather, as the claims reflect, the inventive aspects lie in less than all of the features of the single foregoing embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate embodiment of the present application.

[0143] Those skilled in the art can understand that the modules in the devices in the embodiments can be adaptively changed and set in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition, they can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive, any combination can be adopted to combine all the features disclosed in this specification (including the accompanying claims, abstract and drawings) and all the processes or units of any method or device so disclosed. Unless otherwise explicitly stated, each feature disclosed in this specification (including the accompanying claims, abstract and drawings) can be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0144] In addition, those skilled in the art can understand that although some of the embodiments herein include certain features included in other embodiments rather than other features, the combination of the features of different embodiments means that it is within the scope of this application and forms different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.

[0145] Each component embodiment of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that in practice, a microprocessor or a digital signal processor (DSP) can be used to implement some or all of the functions of some or all of the components according to the embodiments of the present application. The present application can also be implemented as a device or device program (for example, a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for implementing the present application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0146] It should be noted that the above embodiments illustrate the present application rather than limit the present application, and those skilled in the art can design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in the claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In the unit claims listing several devices, several of these devices may be embodied by the same item of hardware. The use of the words first, second, and third, etc. does not denote any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

Claims

1. A material retrieval method, comprising: receiving search input data provided by a user; Performing entity recognition processing on the search input data to obtain an entity search tag; Searching for a target entity node matching the entity search tag in a material knowledge graph; wherein the material knowledge graph is constructed based on materials in a material library; The materials contained in the target entity node are screened to obtain a material search result.

2. The method according to claim 1, wherein: The entity nodes of the material knowledge graph include materials and material tags, and the material knowledge graph is used to record the association relationship between materials and the association relationship between materials and material tags.

3. The method according to claim 1 or 2, wherein: The step of searching the material knowledge graph for a target entity node that matches the entity search tag further includes: Matching the entity search tag with the entity node in the material knowledge graph to determine a first entity node that is identical and / or similar to the entity search tag; Determine a second entity node in the material knowledge graph that is associated with the first entity node and whose distance is within a preset distance range; The first entity node and the second entity node are determined as target entity nodes.

4. The method according to claim 1 or 2, wherein: The screening of the materials contained in the target entity node to obtain the material search result further includes: Performing sentiment analysis on the retrieval input data to obtain sentiment retrieval labels; According to the emotion retrieval tag, the target material is screened out from the materials contained in the target entity node to obtain a material retrieval result.

5. The method according to any one of claims 1 to 4, wherein: After obtaining the material search result, the method further includes: Performing feature extraction processing on the retrieval input data to obtain retrieval input features; Calculating the matching degree between the search input feature and the material feature of each target material included in the material search result; Sort each target material according to the matching degree.

6. The method according to claim 5, wherein: The retrieval input feature includes input key features of multiple dimensions, and the material feature includes material key features of multiple dimensions; and the calculating of the matching degree between the retrieval input feature and the material feature of each target material included in the material retrieval result further includes: For each dimension, calculate the similarity between the input key features of the dimension and the material key features of the target material in the dimension; A weighted fusion is performed according to the similarities and weights corresponding to the multiple dimensions to obtain a matching degree between the retrieval input feature and the material feature of the target material.

7. The method according to claim 6, wherein: The multiple dimensions include multiple dimensions of the following dimensions: semantic dimension, emotional dimension, and visual dimension.

8. A material retrieval device, comprising: A receiving module, adapted to receive search input data provided by a user; An identification module, adapted to perform entity identification processing on the search input data to obtain an entity search tag; A retrieval module, adapted to search for a target entity node matching the entity retrieval tag in a material knowledge graph; wherein the material knowledge graph is constructed based on materials in a material library; The screening module is adapted to screen the materials contained in the target entity node to obtain a material search result.

9. A computing device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the material retrieval method according to any one of claims 1 to 7.

10. A computer storage medium, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to execute an operation corresponding to the material retrieval method according to any one of claims 1 to 7.

11. A computer program product, comprising at least one executable instruction, wherein the executable instruction enables a processor to execute operations corresponding to the material retrieval method according to any one of claims 1 to 7.

Images