Knowledge graph construction

Abstract

Disclosed in the embodiments of the present disclosure are a knowledge graph construction method and apparatus, a storage medium and a terminal. The method comprises: generating at least one user entity on the basis of user data in a target scenario, and generating at least one data entity on the basis of unstructured data in the target scenario; for the data entities of each type, separately extracting a data association relationship for every two data entities of the type; and on the basis of a corresponding relationship between each user entity and each data entity and the data association relationships between the data entities, constructing a knowledge graph corresponding to the target scenario. The present application generates the user entity and the data entities of the unstructured data on the basis of original data in scenarios; and by means of analysis of the unstructured data, the present application extracts the association relationship between any two entities among data entities of each type, and uses same as edges for connecting entity nodes, thus effectively integrating data of various types in the scenarios, and constructing knowledge graphs with richer and more comprehensive knowledge in the target scenarios.

Description

Knowledge Graph Construction Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to knowledge graph construction. Background Technology

[0002] Generally, various types of user-related data can be obtained from different data sources, such as personal information, behavioral data, and uploaded text and image materials. This data can reflect users' needs and preferences for services. By aggregating and analyzing multiple types of data, the platform can better understand users, improve the matching degree between user needs and services, and thus enhance the user experience. However, data obtained from different data sources often differ in type and structure. Therefore, a knowledge graph construction method that can integrate the characteristics of multiple types of data is needed to accurately describe user preferences. Summary of the Invention

[0003] This disclosure provides a knowledge graph construction method, apparatus, storage medium, and terminal, which can solve the technical problem in related technologies that it is impossible to construct knowledge graphs based on unstructured data.

[0004] In a first aspect, embodiments of this disclosure provide a knowledge graph construction method, the method comprising: acquiring user data and at least one type of unstructured data in a target scenario; generating at least one user entity based on the user data; generating at least one data entity based on the unstructured data; for each type of data entity, extracting data association relationships between every two data entities in each type; constructing a knowledge graph corresponding to the target scenario based on the correspondence between each user entity and each data entity and the data association relationships between each data entity, wherein each entity is used as a node in the knowledge graph and the relationships between each entity are used as edges connecting the nodes.

[0005] Secondly, embodiments of this disclosure provide a knowledge graph construction apparatus, which includes: an entity preparation module, used to acquire user data in a target scenario and at least one type of unstructured data, generate at least one user entity based on the user data, and generate at least one data entity based on the unstructured data; a relationship extraction module, used to extract data association relationships between every two data entities of each type for each type of data entity; and a graph construction module, used to construct a knowledge graph corresponding to the target scenario based on the correspondence between each user entity and each data entity and the data association relationships between each data entity, wherein each entity is used as a node and the relationship between each entity is used as an edge connecting the nodes.

[0006] Thirdly, embodiments of this disclosure provide a computer program product containing instructions that, when run on a computer or processor, cause the computer or processor to perform the steps of the method described above.

[0007] Fourthly, embodiments of this disclosure provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the steps of the method described above.

[0008] Fifthly, embodiments of this disclosure provide a terminal including a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being adapted to be loaded by the processor and to execute the steps of the methods described above.

[0009] The beneficial effects of the technical solutions provided by some embodiments of this disclosure include at least the following: This disclosure provides a knowledge graph construction method, which acquires user data and at least one type of unstructured data in a target scenario; generates at least one user entity based on the user data; and generates at least one data entity based on the unstructured data. For each type of data entity, data relationships are extracted between every two data entities of each type. A knowledge graph corresponding to the target scenario is constructed based on the correspondence between each user entity and each data entity, and the data relationships between each data entity. In the knowledge graph, each entity is used as a node, and the relationships between each entity are used as edges connecting the nodes. Since user entities and data entities of unstructured data are generated based on the original data in the scenario, and by analyzing the unstructured data, the relationships between any two entities in each type of data entity are extracted and used as edges connecting the entity nodes, thereby linking all unstructured data, effectively integrating various types of data in the scenario, and constructing a knowledge graph with richer and more comprehensive knowledge in the target scenario that is easier to analyze. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments or related technologies of this disclosure, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 is an exemplary system architecture diagram of a knowledge graph construction method provided in an embodiment of this disclosure;

[0012] Figure 2 is a flowchart illustrating a knowledge graph construction method provided in an embodiment of this disclosure;

[0013] Figure 3 is a flowchart illustrating a knowledge graph construction method according to another embodiment of this disclosure;

[0014] Figure 4 is an example diagram of the logical modules of a knowledge graph construction method provided in an embodiment of this disclosure;

[0015] Figure 5 is a structural block diagram of a knowledge graph construction device provided in an embodiment of this disclosure;

[0016] Figure 6 is a schematic diagram of the structure of a terminal provided in an embodiment of this disclosure. Detailed Implementation

[0017] To make the features and advantages of the embodiments of this disclosure more apparent and understandable, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the embodiments of this disclosure.

[0018] In the following description, when referring to the accompanying drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this disclosure as detailed in the appended claims.

[0019] In the era of big data, the sources of data on a platform are diverse. For example, in addition to the personal information a user fills in, a user's data may also come from various data modalities such as text data, image data, and audio data. Different modalities of data can describe the characteristics of information from different aspects. Therefore, multimodal data can more comprehensively reflect users' needs and preferences for services. By summarizing and analyzing multiple types of data, it is beneficial to build a knowledge base related to the platform, services, and users. This knowledge base can be used to analyze user needs and locate target markets, so as to understand market and user needs and thus better provide targeted services to users and improve user experience.

[0020] However, compared to structured data, which is easier to process, data obtained from different data sources often consists of unstructured data. This data often differs in type and characteristics, contains richer raw information, and is relatively difficult to process. Therefore, traditional data processing methods struggle to effectively integrate this data and cannot easily link different types of data together. This results in a weak connection between data and users, preventing data from being used as it should and limiting the platform's ability to provide personalized and high-quality services to users.

[0021] Therefore, this disclosure provides a knowledge graph construction method, which acquires user data and at least one type of unstructured data in a target scenario, generates at least one user entity based on the user data, and generates at least one data entity based on the unstructured data; for each type of data entity, extracts data association relationships between every two data entities in each type; and constructs a knowledge graph corresponding to the target scenario based on the correspondence between each user entity and each data entity and the data association relationships between each data entity. In the knowledge graph, each entity is used as a node, and the relationship between each entity is used as an edge connecting the nodes, so as to solve the above-mentioned technical problem that it is impossible to construct a knowledge graph based on unstructured data.

[0022] Please refer to Figure 1, which is an exemplary system architecture diagram of a knowledge graph construction method provided in this disclosure embodiment.

[0023] As shown in Figure 1, the system architecture may include a terminal 101, a network 102, and a server 103. The network 102 serves as the medium for providing a communication link between the terminal 101 and the server 103. The network 102 may include various types of wired or wireless communication links, such as wired communication links including fiber optic cables, twisted-pair cables, or coaxial cables, and wireless communication links including Bluetooth, Wireless-Fidelity (Wi-Fi), or microwave communication links.

[0024] Terminal 101 can interact with server 103 via network 102 to receive messages from or send messages to server 103. Alternatively, terminal 101 can interact with server 103 via network 102 to receive messages or data sent to server 103 by other users. Terminal 101 can be hardware or software. When terminal 101 is hardware, it can be various electronic devices, including but not limited to smartwatches, smartphones, tablets, laptops, and desktop computers. When terminal 101 is software, it can be installed in the aforementioned electronic devices and can be implemented as multiple software programs or software modules (e.g., to provide distributed services) or as a single software program or software module; no specific limitation is made here.

[0025] In this embodiment of the disclosure, terminal 101 acquires user data and at least one type of unstructured data in the target scenario, generates at least one user entity based on the user data, and generates at least one data entity based on the unstructured data; further, for each type of data entity, terminal 101 extracts the data association relationship between every two data entities in each type; at this time, terminal 101 can construct a knowledge graph corresponding to the target scenario based on the correspondence between each user entity and each data entity and the data association relationship between each data entity, wherein each entity is used as a node and the relationship between each entity is used as an edge connecting the nodes.

[0026] Server 103 can be a business server providing various services. It should be noted that server 103 can be either hardware or software. When server 103 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When server 103 is software, it can be implemented as multiple software programs or software modules (e.g., used to provide distributed services), or as a single software program or software module; no specific limitations are made here.

[0027] Alternatively, the system architecture may not include server 103. In other words, server 103 may be an optional device in the embodiments of this disclosure. That is, the method provided in the embodiments of this disclosure can be applied to a system structure that only includes terminal 101. The embodiments of this disclosure do not limit this.

[0028] It should be understood that the number of terminals, networks, and servers in Figure 1 is only illustrative, and can be any number of terminals, networks, and servers depending on the implementation needs.

[0029] Please refer to Figure 2, which is a flowchart illustrating a knowledge graph construction method provided in this embodiment of the present disclosure. The execution entity in this embodiment can be a terminal executing knowledge graph construction, a processor within the terminal executing the knowledge graph construction method, or a knowledge graph construction service within the terminal executing the knowledge graph construction method. For ease of description, the following uses the processor within the terminal as an example to describe the specific execution process of the knowledge graph construction method.

[0030] As shown in Figure 2, a knowledge graph construction method can include at least the following steps.

[0031] S202. Obtain user data and at least one type of unstructured data in the target scenario, generate at least one user entity based on the user data, and generate at least one data entity based on the unstructured data.

[0032] Optionally, when dealing with large volumes of data on a platform, and analyzing this data for user-related decision-making, a knowledge graph can be constructed to represent the connections between users and data. As a crucial component of artificial intelligence technology, a knowledge graph possesses powerful semantic processing, interconnection, information retrieval, and knowledge reasoning capabilities. A knowledge graph is a semantic network graph that uses nodes (or vertices) to represent entities or concepts and edges to represent relationships, describing various entities or concepts existing in the real world and their relationships. A knowledge graph, also known as a knowledge domain visualization or knowledge domain mapping map, is a series of various graphs displaying the development process and structural relationships of knowledge. It uses visualization techniques to describe knowledge resources and their carriers, mining, analyzing, constructing, drawing, and displaying knowledge and their interrelationships. A knowledge graph can describe entities and their relationships existing in the real world, typically represented using triples. Each triple includes a head entity, a tail entity, and a relationship. Entities are interconnected through relationships, forming a network-like knowledge structure.

[0033] Optionally, most current data maps are constructed based on structured data, which is typically stored in databases and can be logically expressed and implemented using a two-dimensional table structure. Each column has a specific meaning and can be formally stored. Because structured data has a fixed format and structure, it is relatively easy for computers to process. Unstructured data, such as text, images, and audio, is not easily represented using two-dimensional logical tables in databases. While it contains rich raw information, its diverse formats and lack of a unified logical structure make it more complex to process. Traditional data processing methods cannot effectively integrate unstructured data, limiting the platform's ability to provide personalized and high-quality services to users. On the other hand, most current user data analysis methods rely on personal profiles and basic user behavior to understand users, without considering the behavioral correlations between different users or the correlations between different behaviors of the same user, making it difficult to deeply understand user needs and preferences.

[0034] Optionally, to construct a richer and more comprehensive knowledge graph, unstructured data can be used. This first requires building the entities that serve as nodes in the knowledge graph. Therefore, the first step is to acquire user data from the target scenario and at least one type of unstructured data. Each user and each piece of unstructured data is then transformed into an entity node in the graph. Further analysis of the relationships between the data is then conducted, connecting the entities in the knowledge graph through relational edges to obtain a knowledge graph linking users and unstructured data. Specifically, the types of unstructured data can include at least one of text data, image data, and audio data. The specific content of the data is related to the user and the services of the target scenario. For example, in a financial institution platform, text data could be text records of consultations between users and account managers, text descriptions of services selected by users, etc.; image data could be images of supporting documents uploaded by users, etc.; and audio data could be audio clips uploaded by users, etc.

[0035] Optionally, a user entity is generated for each user based on user data. The identifier of the user entity is the unique identifier of the user in the target scenario, and the attributes are basic attribute information such as the user's identity information and interests. Similarly, a data entity is generated for each data point based on unstructured data. The identifier of the data entity is the unique identifier of the data in the target scenario, and the attributes are key information and descriptive information of the data itself.

[0036] Furthermore, we can first clarify the relationship between users and their corresponding unstructured data. This is equivalent to first constructing a knowledge subgraph centered on the user entity based on the user's own data and the text, image, and voice data belonging to that user. Then, we can extract the correlation between various unstructured data, thereby including different users and different data in the platform into the knowledge graph.

[0037] S204. For each type of data entity, extract the data association relationship between every two data entities in each type.

[0038] Optionally, when constructing relationships between unstructured data, considering that different types of data may have different characteristics and structures, relationships can be extracted from data of the same category according to categories. That is, for each type of data entity, data relationships are extracted from every two data entities in each category. The data relationships are used to describe the degree of similarity between different data entities. This can clarify the relationship between any two data of the same type, thereby connecting a large amount of data and users in the target scenario, and effectively using unstructured data to analyze user needs and preferences and provide users with better services.

[0039] S206. Construct a knowledge graph corresponding to the target scenario based on the correspondence between each user entity and each data entity and the data association between each data entity. In the knowledge graph, each entity is used as a node and the relationship between each entity is used as an edge connecting the nodes.

[0040] Optionally, after extracting the relationships between the data, entities are used as nodes in the knowledge graph, and the relationships between entities are used as edges connecting the nodes. That is, a knowledge graph corresponding to the target scenario is constructed based on user entities and data entities, as well as the correspondence between user entities and data entities and the data association between data entities. The resulting knowledge graph links all unstructured data, effectively integrates various types of data in the scenario, and helps the platform to better understand users, improve the matching degree between user needs and services, and thus improve the user experience.

[0041] This disclosure provides a knowledge graph construction method. The method involves acquiring user data and at least one type of unstructured data from a target scenario. At least one user entity is generated based on the user data, and at least one data entity is generated based on the unstructured data. For each type of data entity, data relationships are extracted between every two data entities of that type. A knowledge graph corresponding to the target scenario is constructed based on the correspondence between user entities and data entities, as well as the data relationships between data entities. In this knowledge graph, entities are used as nodes, and relationships between entities are used as edges connecting the nodes. By generating user entities and unstructured data entities from the original data in the scenario, and by analyzing the unstructured data and extracting the relationships between any two entities in each type as edges connecting entity nodes, all unstructured data is linked, effectively integrating various types of data in the scenario and constructing a more comprehensive and easier-to-analyze knowledge graph for the target scenario.

[0042] Please refer to Figure 3, which is a flowchart illustrating a knowledge graph construction method provided in another embodiment of this disclosure.

[0043] As shown in Figure 3, a knowledge graph construction method can include at least the following steps.

[0044] S302. Obtain user data in the target scenario and at least one type of unstructured data, and generate at least one user entity based on the user data.

[0045] Optionally, as described in the above embodiments, in order to construct a more comprehensive and richer knowledge graph, user data and unstructured data can be used to construct entities as nodes in the knowledge graph. Then, the relationships between the data are further analyzed, thereby connecting the entities in the knowledge graph through relational edges to obtain a knowledge graph of associated users and unstructured data. Therefore, the first step is to acquire user data from the target scenario and at least one type of unstructured data, transforming each user and each piece of unstructured data into entity nodes in the graph. Please refer to Figure 4, which is an example diagram of the logical modules of a knowledge graph construction method provided in this embodiment of the present disclosure. As shown in Figure 4, in the entity preparation logical module, corresponding user entities and data entities are generated based on user data and other unstructured data. Data entities and data entities are two different types of entities. User entities are used to represent real users in the target scenario, while data entities are entities corresponding to unstructured data in the target scenario. In order to distinguish between user entities and data entities, data entities and user entities can be represented by different node colors in the knowledge graph representation. For example, in the graph construction logical module, data entities are represented by black spheres, and user entities are represented by light gray spheres.

[0046] S304. When unstructured data includes text data, at least one text entity shall be generated based on the text data. The text data includes at least one text, each text corresponds to one text entity and each text entity has a unique text identifier.

[0047] Optionally, in the target scenario, text messages sent by users during consultations, product descriptions purchased by users, etc., may contain information about users' needs and interests. This text information can serve as user-related text data, which is beneficial for building a knowledge graph based on this text data to gain a deeper understanding of users. Therefore, when unstructured data includes text data, generating data entities involves generating at least one text entity based on the text data. The text data includes at least one text, and each text corresponds to one text entity. In the knowledge graph, the unique text identifier of a text entity is also the unique identifier of its original text.

[0048] S306. Input the text data into the text relation extraction model, calculate the text similarity between each pair of texts through the text relation extraction model, and determine the text association relationship between each pair of text entities based on the text similarity.

[0049] Optionally, referring to Figure 4, in the relation extraction logic module, after preparing the text entities, it is necessary to extract the relationships between every two text entities, deeply analyze the connections and relevance between different texts, and use the relevance between texts as relational knowledge to construct a knowledge graph, making the knowledge graph richer and more accurate in knowledge. Since the data structure of text is complex, containing rich semantic, lexical, and emotional information, a neural network model capable of processing text information can be used for text feature representation and relevance calculation when extracting relationships between text entities. Specifically, a pre-trained BERT model can be used to complete Natural Language Processing (NLP) tasks. Based on the BERT model, the text relation extraction model in this embodiment is obtained after targeted training using sample text data in the target scenario. The text data is input into the text relation extraction model, and the text similarity between every two texts is calculated. The text relationship between every two text entities is determined based on the text similarity. It should be noted that, in the embodiments of this disclosure, the text relationship extraction model can be used only to perform feature representation on the text, and then output the text representation, and other calculation modules can calculate the similarity between the texts based on the text representation output by the model; the text relationship extraction model can also not only perform feature representation, but also directly calculate the text similarity and output the similarity calculation result.

[0050] Optionally, after obtaining the raw text data, it is necessary to clean the data and remove numbers, symbols, etc., that are irrelevant to the target scene. Considering that different texts may have different lengths and be long, and that the model usually requires the text length to not exceed a certain threshold when extracting text representation vectors, a sliding window slicing method can be used when comparing the similarity between texts. Each text in the text data is divided into at least one text slice of a preset length. The text data is then input into the text relation extraction model, which calculates the slice similarity between each pair of text slices for every two texts. Each text slice pair consists of one text slice from each of the two texts being compared. The slice similarity between each pair of texts that meets the preset conditions is selected as the text similarity. For example, if text 'a' is segmented into fragment 1 and fragment 2, and text 'b' is segmented into fragment 3 and fragment 4, when calculating the text similarity between text 'a' and text 'b', we first calculate the slice similarity between each pair of fragments, that is, the slice similarity between fragment 1 and fragment 3, fragment 1 and fragment 4, fragment 2 and fragment 3, and fragment 2 and fragment 4. The highest slice similarity is taken as the text similarity between text 'a' and text 'b'. This solves the problem that when the text is long, a complete calculation cannot be performed in one step in the model.

[0051] Optionally, besides sliding window slicing of the text, text can also be simplified by extracting text summaries. This involves extracting a text summary for each text in the text data, then inputting all text summaries into a text relation extraction model. The model calculates the summary similarity between any two text summaries, and this similarity is then used to determine the text similarity. Specifically, a Large Language Model (LLM) can be introduced to extract text summaries. LLM excels at understanding long texts, extracting accurate summaries without losing rich semantic meaning. Once text summaries that meet the input requirements of the text relation extraction model are obtained, the model outputs the summary similarity as the text similarity between the original texts.

[0052] Furthermore, Large Language Models (LLMs) can also be used for attribute extraction of text entities. Prompts can be provided to LLMs to inform them of the analysis to be performed, the content to be output, the required output format, and so on. Based on these prompts, LLMs can extract points of interest from the text. For example, in a financial context, the points of interest in the text might be lending products, investment recommendations, etc. These points of interest represent the key points of the text and can be directly used as attribute information for text entities.

[0053] S308. When unstructured data includes image data, at least one image entity is generated based on the image data. The image data includes at least one image, each image corresponds to one image entity, and each image entity has a unique image identifier.

[0054] Optionally, in the target scenario, users may upload images of their identity information, qualification certificates, etc., when obtaining services or purchasing products. This contains a large amount of user-related information. Building a knowledge graph based on this image data is beneficial for a deeper understanding of users. Therefore, when unstructured data includes image data, generating data entities involves generating at least one image entity based on the image data. The image data includes at least one image, and each image corresponds to one image entity. In the knowledge graph, the unique image identifier of an image entity is also the unique identifier of its original image.

[0055] S310. Extract the image content association relationship for every two image entities, and extract the image ontology association relationship for every two image entities.

[0056] Optionally, image information typically includes two aspects: textual information within the image and pixel information of the image itself. Therefore, when extracting the correlation between images, the correlation can be determined from both aspects. That is, please refer to Figure 4. When extracting the data association between any two image entities, it is necessary to extract the image content association between each pair of image entities, and also to extract the image body association between each pair of image entities.

[0057] Optionally, when extracting the image content association between any two image entities, it is necessary to extract and compare the text content in the images. This involves extracting the target text content from each image using image text recognition algorithms such as OCR, and then calculating the content similarity of the target text content between each pair of images using methods such as Manhattan distance, cosine similarity, and simhash. The image content association between each pair of image entities is then determined based on the content similarity. When extracting the target text content, key information in the images can be selectively extracted. Different types of key information are pre-classified by technicians. For example, all images can be divided into images related to loan repayment and images related to investment. Images belonging to different key information types are identified using algorithms with different focus areas.

[0058] Optionally, when extracting image ontology relationships between two image entities, analyzing and comparing the pixel features of the images themselves can focus on the positions of different elements in the image, such as the location of a stamp or signature, thereby more accurately inferring the correlation between different images. In this embodiment, image data is input into an image relationship extraction model, which calculates the image similarity between two images and determines the image ontology relationships between two image entities based on the image similarity. The image relationship extraction model can use the Swin Transformer model to extract image feature representations. The advantage of the Swin Transformer is that it is based on a hierarchical Transformer architecture and uses a hierarchical attention mechanism to process features at different scales, making it suitable for image representation tasks.

[0059] S312. When unstructured data includes audio data, at least one audio entity is generated based on the audio data. The audio data includes at least one audio, each audio corresponds to one audio entity, and each audio entity has a unique audio identifier.

[0060] Optionally, in the target scenario, the audio information related to the user also includes user-related information. Building a knowledge graph based on this audio data helps to supplement the user information in the graph. Therefore, when unstructured data includes audio data, generating data entities involves generating at least one audio entity based on the audio data. The audio data includes at least one audio file, and each audio file corresponds to one audio entity. In the knowledge graph, the unique audio identifier of the audio entity is also the unique identifier of its original audio file.

[0061] S314. Obtain the voiceprint information corresponding to each audio in the audio data based on the voiceprint extraction algorithm; calculate the voiceprint similarity of the voiceprint information corresponding to each two audios, and determine the audio association relationship between each two audio entities based on the voiceprint similarity.

[0062] Optionally, referring to Figure 4, when extracting the data association between each pair of audio files, voiceprint extraction algorithms such as x-vector and comfomer are used to obtain the voiceprint information corresponding to each audio file in the audio data, constructing a voiceprint database. The comfomer algorithm combines a convolutional neural network (CNN) and a Transformer neural network architecture, utilizing the spatial awareness of convolutional layers and the global dependency modeling capability of Transformers to extract more robust and discriminative voiceprint representations. Then, the voiceprint similarity of the voiceprint information corresponding to each pair of audio files is calculated, and the audio association between each pair of audio entities is determined based on the voiceprint similarity. When new audio data appears subsequently, voiceprint information is extracted from the new audio data, and voiceprint retrieval capabilities are used to search and compare the voiceprint database to see if there are voiceprints with similarity that meet the conditions, thereby determining the position of the new audio in the knowledge graph and its connection relationships with other audio entities.

[0063] S316. Determine the target data association relationships that meet the similarity conditions of each type, and construct the knowledge graph corresponding to the target scene based on the correspondence between each user entity and each data entity and the target data association relationships between each data entity.

[0064] Optionally, as can be seen from the above embodiments, when extracting the data association relationship between various types of data entities, the similarity between every two data in each type of unstructured data can be calculated first, and then the data association relationship between every two data entities in each type can be determined based on the similarity.

[0065] Specifically, considering that a certain level of similarity between data indicates a high degree of correlation, while low similarity indicates a low correlation, it is necessary to filter data relationships based on their similarity levels when constructing a knowledge graph. This prevents the graph from containing excessive redundant data and ensures its accuracy. Therefore, in this embodiment, target data relationships that meet the similarity criteria for each type are determined, and a knowledge graph corresponding to the target scenario is constructed based on the correspondence between user entities and data entities, as well as the target data relationships between data entities.

[0066] It should be noted that different types of data have different characteristics, and their relationship characteristics will also be different. Therefore, the similarity conditions for each type can be obtained based on the data characteristics and relationship characteristics of each type. Combined with the actual data requirements of the target scenario, the corresponding similarity threshold for each type can be determined as the filtering condition. During the filtering, data associations with similarity greater than the similarity threshold are selected to enter the graph, while other relationships that do not meet the requirements are discarded.

[0067] S318. Conduct personalized user analysis and / or product recommendation based on knowledge graphs in target scenarios.

[0068] Optionally, after the knowledge graph for the target scenario is built, personalized user analysis and / or product recommendations can be performed based on the knowledge graph in the target scenario. Applying the knowledge graph to personalized user services and product recommendations can provide users with services and products that better meet their needs by leveraging the user information and implicit information of unstructured data in the knowledge graph, thereby improving the user experience.

[0069] In this embodiment, a knowledge graph construction method is provided. When the unstructured data includes text data, at least one text entity is generated based on the text data. The text data is input into a text relation extraction model, and the text similarity between each pair of texts is calculated using the text relation extraction model. The text association between each pair of text entities is determined based on the text similarity. Applying a large model to long text understanding can obtain more supplementary information and enrich semantic information. With the powerful knowledge insight capability of the model, more accurate similarity calculation results are obtained. When the unstructured data includes image data, at least one image entity is generated based on the image data. Image content association and image ontology association are extracted between each pair of image entities. The correlation between images is analyzed from multiple aspects, including image content and image ontology. This allows attention to both document content and image ontology. The system identifies the relationships between key elements in an image and their locations. When unstructured data includes audio data, at least one audio entity is generated based on the audio data. Voiceprint information corresponding to each audio element is obtained using a voiceprint extraction algorithm. The voiceprint similarity between each pair of audio elements is calculated, and the audio relationship between each pair of audio entities is determined based on the voiceprint similarity. The problem of audio relationship extraction is solved by constructing a voiceprint database and developing retrieval capabilities. Target data relationships that meet the similarity conditions for each type are identified. A knowledge graph is constructed based on these target data relationships. Data relationships are filtered according to their similarity levels to prevent redundant data in the graph and ensure its accuracy. Personalized user analysis and / or product recommendations are performed based on the knowledge graph in target scenarios, leveraging user information and implicit information from unstructured data to improve user experience.

[0070] Please refer to Figure 5, which is a structural block diagram of a knowledge graph construction device provided in an embodiment of this disclosure. As shown in Figure 5, the knowledge graph construction device 500 includes: an entity preparation module 510, used to acquire user data and at least one type of unstructured data in a target scenario, generate at least one user entity based on the user data, and generate at least one data entity based on the unstructured data; a relationship extraction module 520, used to extract data association relationships for each type of data entity from every two data entities in each type; and a graph construction module 530, used to construct a knowledge graph corresponding to the target scenario based on the correspondence between each user entity and each data entity and the data association relationships between each data entity, wherein each entity is used as a node and the relationship between each entity is used as an edge connecting the nodes.

[0071] Optionally, the types of unstructured data include at least one of text data, image data, and audio data.

[0072] Optionally, when the unstructured data includes text data, the entity preparation module 510 is further configured to generate at least one text entity based on the text data, wherein the text data includes at least one text, each text corresponds to one text entity and each text entity has a unique text identifier; the relation extraction module 520 is further configured to input the text data into the text relation extraction model, calculate the text similarity between each pair of texts through the text relation extraction model, and determine the text association relationship between each pair of text entities based on the text similarity.

[0073] Optionally, the relation extraction module 520 is further configured to divide each text in the text data into at least one text slice of a preset length; input the text data into the text relation extraction model, calculate the slice similarity between each pair of text slices between each pair of texts through the text relation extraction model, wherein each text slice pair consists of one text slice from each of the two texts being compared; and select the slice similarity between each pair of texts that meets the preset conditions as the text similarity.

[0074] Optionally, the relation extraction module 520 is also used to extract a text summary for each text in the text data, input all text summaries into the text relation extraction model, calculate the summary similarity between every two text summaries through the text relation extraction model, and determine the summary similarity between every two texts as the text similarity.

[0075] Optionally, when the unstructured data includes image data, the entity preparation module 510 is further configured to generate at least one image entity based on the image data, wherein the image data includes at least one image, each image corresponds to one image entity and each image entity has a unique image identifier; the relationship extraction module 520 is further configured to extract the image content association relationship for every two image entities and extract the image ontology association relationship for every two image entities.

[0076] Optionally, the relation extraction module 520 is also used to extract the target text content in each image based on the image text recognition algorithm; calculate the content similarity of the target text content corresponding to each two images; and determine the image content association relationship between each two image entities based on the content similarity.

[0077] Optionally, the relation extraction module 520 is also used to input image data into the image relation extraction model, calculate the image similarity between each pair of images through the image relation extraction model, and determine the image ontology association relationship between each pair of image entities based on the image similarity.

[0078] Optionally, when the unstructured data includes audio data, the entity preparation module 510 is further configured to generate at least one audio entity based on the audio data, wherein the audio data includes at least one audio, each audio corresponds to one audio entity and each audio entity has a unique audio identifier; the relation extraction module 520 is further configured to obtain the voiceprint information corresponding to each audio in the audio data based on the voiceprint extraction algorithm; calculate the voiceprint similarity of the voiceprint information corresponding to each two audios, and determine the audio association relationship between each two audio entities based on the voiceprint similarity.

[0079] Optionally, the relation extraction module 520 is also used to calculate the similarity between every two data in each type of unstructured data, and determine the data association relationship between every two data entities in each type based on the similarity; the graph construction module 530 is also used to determine the target data association relationship that satisfies the similarity condition of each type, and construct the knowledge graph corresponding to the target scene based on the correspondence between each user entity and each data entity and the target data association relationship between each data entity; wherein, the similarity condition of each type is obtained based on the data characteristics and relation characteristics of each type itself.

[0080] Optionally, the knowledge graph construction device 500 also includes a graph application module for performing personalized user analysis and / or product recommendation based on the knowledge graph in a target scenario.

[0081] In this embodiment, a knowledge graph construction apparatus is provided, comprising: an entity preparation module for acquiring user data and at least one type of unstructured data in a target scenario, generating at least one user entity based on the user data, and generating at least one data entity based on the unstructured data; a relation extraction module for extracting data relationships between every two data entities of each type; and a graph construction module for constructing a knowledge graph corresponding to the target scenario based on the correspondence between each user entity and each data entity, and the data relationships between each data entity. The knowledge graph uses each entity as a node and the relationships between entities as edges connecting the nodes. By generating user entities and unstructured data entities from the original data in the scenario, and by analyzing the unstructured data and extracting the relationships between any two entities of each type as edges connecting entity nodes, all unstructured data are linked, effectively integrating various types of data in the scenario, and constructing a more comprehensive and easier-to-analyze knowledge graph for the target scenario.

[0082] This disclosure provides a computer program product containing instructions that, when run on a computer or processor, cause the computer or processor to perform the steps of any of the methods described above.

[0083] This disclosure also provides a computer storage medium that can store multiple instructions adapted for loading by a processor and executing the steps of any of the methods described in the above embodiments.

[0084] Please refer to Figure 6, which is a schematic diagram of the structure of a terminal provided in an embodiment of this disclosure. As shown in Figure 6, the terminal 600 may include: at least one terminal processor 601, at least one network interface 604, a user interface 603, a memory 605, and at least one communication bus 602.

[0085] The communication bus 602 is used to enable communication between these components.

[0086] The user interface 603 may include a display screen and a camera. Optionally, the user interface 603 may also include a standard wired interface and a wireless interface.

[0087] The network interface 604 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0088] The terminal processor 601 may include one or more processing cores. The terminal processor 601 connects to various parts within the terminal 600 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 605, and by calling data stored in the memory 605. Optionally, the terminal processor 601 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The terminal processor 601 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed on the screen; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the terminal processor 601 and may be implemented as a separate chip.

[0089] The memory 605 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 605 may include a non-transitory computer-readable storage medium. The memory 605 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 605 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. Optionally, the memory 605 may also be at least one storage device located remotely from the aforementioned terminal processor 601. As shown in FIG6, the memory 605, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a knowledge graph construction program.

[0090] In the terminal 600 shown in Figure 6, the user interface 603 is mainly used to provide an input interface for the user and obtain the user input data; while the terminal processor 601 can be used to call the knowledge graph construction program stored in the memory 605 and specifically perform the following operations: obtain user data in the target scene and at least one type of unstructured data; generate at least one user entity based on the user data and at least one data entity based on the unstructured data; for each type of data entity, extract the data association relationship between every two data entities in each type; construct the knowledge graph corresponding to the target scene based on the correspondence between each user entity and each data entity and the data association relationship between each data entity, in which each entity is used as a node and the relationship between each entity is used as an edge connecting the nodes.

[0091] In some embodiments, the type of unstructured data includes at least one of text data, image data, and audio data.

[0092] In some embodiments, when the unstructured data includes text data, the terminal processor 601, when generating at least one data entity based on the unstructured data, specifically performs the following steps: generating at least one text entity based on the text data, wherein the text data includes at least one text, each text corresponds to one text entity and each text entity has a unique text identifier; the terminal processor 601, when extracting data association relationships for each type of data entity, specifically performs the following steps: inputting the text data into the text relationship extraction model, calculating the text similarity between each pair of texts through the text relationship extraction model, and determining the text association relationship between each pair of text entities based on the text similarity.

[0093] In some embodiments, when the terminal processor 601 performs the following steps when inputting text data into a text relation extraction model and calculating the text similarity between each pair of texts using the text relation extraction model: dividing each text in the text data into at least one text slice of a preset length; inputting the text data into the text relation extraction model and calculating the slice similarity between each pair of text slices between each pair of texts using the text relation extraction model, wherein each text slice pair consists of one text slice from each of the two texts being compared; and selecting the slice similarity between each pair of texts that satisfies a preset condition as the text similarity.

[0094] In some embodiments, when the terminal processor 601 performs the following steps when it inputs text data into a text relation extraction model and calculates the text similarity between each pair of texts using the text relation extraction model: extracting a text summary for each text in the text data, inputting all text summaries into the text relation extraction model, calculating the summary similarity between each pair of text summaries using the text relation extraction model, and determining the summary similarity between each pair of texts as the text similarity.

[0095] In some embodiments, when the unstructured data includes image data, the terminal processor 601, when generating at least one data entity based on the unstructured data, specifically performs the following steps: generating at least one image entity based on the image data, wherein the image data includes at least one image, each image corresponds to one image entity and each image entity has a unique image identifier; and when the terminal processor 601, for each type of data entity, extracts the data association relationship for each pair of data entities in each type, specifically performs the following steps: extracting the image content association relationship for each pair of image entities, and extracting the image ontology association relationship for each pair of image entities.

[0096] In some embodiments, when the terminal processor 601 extracts the image content association relationship between every two image entities, it specifically performs the following steps: extracting the target text content in each image based on an image text recognition algorithm; calculating the content similarity of the target text content corresponding to every two images; and determining the image content association relationship between every two image entities based on the content similarity.

[0097] In some embodiments, when the terminal processor 601 extracts the image ontology association relationship between every two image entities, it specifically performs the following steps: inputting image data into the image relationship extraction model, calculating the image similarity between every two images through the image relationship extraction model, and determining the image ontology association relationship between every two image entities based on the image similarity.

[0098] In some embodiments, when the unstructured data includes audio data, the terminal processor 601, when generating at least one data entity based on the unstructured data, specifically performs the following steps: generating at least one audio entity based on the audio data, wherein the audio data includes at least one audio, each audio corresponds to one audio entity and each audio entity has a unique audio identifier; the terminal processor 601, when extracting the data association relationship for each type of data entity, specifically performs the following steps: obtaining the voiceprint information corresponding to each audio in the audio data based on the voiceprint extraction algorithm; calculating the voiceprint similarity of the voiceprint information corresponding to each two audios, and determining the audio association relationship between each two audio entities based on the voiceprint similarity.

[0099] In some embodiments, when the terminal processor 601 extracts data association relationships for each type of data entity, it specifically performs the following steps: calculating the similarity between each pair of unstructured data in each type, and determining the data association relationship between each pair of data entities in each type based on the similarity; when the terminal processor 601 constructs a knowledge graph corresponding to the target scene based on the correspondence between each user entity and each data entity and the data association relationship between each data entity, it specifically performs the following steps: determining the target data association relationship that satisfies the similarity condition of each type, and constructing the knowledge graph corresponding to the target scene based on the correspondence between each user entity and each data entity and the target data association relationship between each data entity; wherein, the similarity condition of each type is obtained based on the data characteristics and relationship characteristics of each type itself.

[0100] In some embodiments, after the terminal processor 601 constructs a knowledge graph corresponding to the target scenario based on the correspondence between each user entity and each data entity and the data association between each data entity, it further performs the following steps: performing personalized user analysis and / or product recommendation in the target scenario based on the knowledge graph.

[0101] In the several embodiments provided in this disclosure, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0102] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0103] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this disclosure are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0104] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this disclosure are not limited to the described order of actions, because according to the embodiments of this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this disclosure.

[0105] Furthermore, it should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in the embodiments of this disclosure are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the user data, text data, image data, audio data, etc. involved in this disclosure were all obtained under full authorization.

[0106] The foregoing has described specific embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0107] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0108] The above is a description of a knowledge graph construction method, apparatus, storage medium, and terminal provided in the embodiments of this disclosure. For those skilled in the art, based on the ideas of the embodiments of this disclosure, there will be changes in the specific implementation methods and application scope. Therefore, the content of this disclosure should not be construed as a limitation on the embodiments of this disclosure.

Claims

1. A knowledge graph construction method, the method comprising: Acquire user data in the target scenario and at least one type of unstructured data, generate at least one user entity based on the user data, and generate at least one data entity based on the unstructured data; For each type of data entity, extract the data relationship between every two data entities in each type; A knowledge graph corresponding to the target scenario is constructed based on the correspondence between each user entity and each data entity, as well as the data association between each data entity. In the knowledge graph, each entity is used as a node, and the relationship between each entity is used as an edge connecting the nodes.

2. The method according to claim 1, wherein the type of unstructured data includes at least one of text data, image data, and audio data.

3. The method according to claim 2, wherein when the unstructured data includes text data, generating at least one data entity based on the unstructured data includes: At least one text entity is generated based on the text data, wherein the text data includes at least one text, each text corresponds to one text entity and each text entity has a unique text identifier; For each type of data entity, the data association relationship is extracted for every two data entities in each type, including: The text data is input into a text relationship extraction model, and the text similarity between each pair of texts is calculated using the text relationship extraction model. Based on the text similarity, the text association relationship between each pair of text entities is determined.

4. The method according to claim 3, wherein inputting the text data into a text relation extraction model and calculating the text similarity between every two texts using the text relation extraction model includes: Each text in the text data is divided into at least one text slice of a preset length; The text data is input into the text relation extraction model, and the slice similarity between each pair of text slices between each pair of texts is calculated by the text relation extraction model. The text slice pair consists of one text slice from each of the two texts being compared. The slice similarity between any two texts that meets the preset conditions is selected as the text similarity.

5. The method according to claim 3, wherein inputting the text data into a text relation extraction model and calculating the text similarity between every two texts using the text relation extraction model includes: For each text in the text data, extract a text summary, input all text summaries into the text relation extraction model, and calculate the summary similarity between every two text summaries using the text relation extraction model; The summary similarity between any two texts is defined as the text similarity.

6. The method according to claim 2, wherein when the unstructured data includes image data, generating at least one data entity based on the unstructured data includes: At least one image entity is generated based on the image data, wherein the image data includes at least one image, each image corresponds to one image entity and each image entity has a unique image identifier; For each type of data entity, the data association relationship is extracted for every two data entities in each type, including: Extract the image content association relationship for every two image entities, and extract the image ontology association relationship for every two image entities.

7. The method according to claim 6, wherein extracting the image content association relationship between every two image entities includes: Extracting target text content from each image based on image text recognition algorithms; Calculate the content similarity of the target text content corresponding to each pair of images, and determine the image content association relationship between each pair of image entities based on the content similarity.

8. The method according to claim 6, wherein extracting the image ontology association relationship for every two image entities includes: The image data is input into the image relationship extraction model, and the image similarity between each pair of images is calculated through the image relationship extraction model. Based on the image similarity, the image ontology association relationship between each pair of image entities is determined.

9. The method according to claim 2, wherein when the unstructured data includes audio data, generating at least one data entity based on the unstructured data includes: At least one audio entity is generated based on the audio data, wherein the audio data includes at least one audio, each audio corresponds to one audio entity and each audio entity has a unique audio identifier; For each type of data entity, the data association relationship is extracted for every two data entities in each type, including: The voiceprint information corresponding to each audio in the audio data is obtained based on the voiceprint extraction algorithm; Calculate the voiceprint similarity of voiceprint information corresponding to each pair of audios, and determine the audio association relationship between each pair of audio entities based on the voiceprint similarity.

10. The method according to claim 1, wherein extracting the data association relationship for every two data entities in each type of data entity includes: Calculate the similarity between every two data points in each type of unstructured data, and determine the data association relationship between every two data entities in each type based on the similarity. The construction of the knowledge graph corresponding to the target scenario based on the correspondence between each user entity and each data entity, as well as the data association between each data entity, includes: Determine the target data association relationships that meet the similarity conditions of each type, and construct the knowledge graph corresponding to the target scene based on the correspondence between each user entity and each data entity and the target data association relationships between each data entity. The similarity conditions for each type are derived based on the data characteristics and relational characteristics of each type.

11. The method according to claim 1, wherein after constructing the knowledge graph corresponding to the target scene based on the correspondence between each user entity and each data entity and the data association relationship between each data entity, the method further includes: Based on the knowledge graph, personalized user analysis and / or product recommendations are performed in the target scenario.

12. A knowledge graph construction apparatus, the apparatus comprising: An entity preparation module is used to acquire user data in a target scenario and at least one type of unstructured data, generate at least one user entity based on the user data, and generate at least one data entity based on the unstructured data. The relation extraction module is used to extract the data relationship between every two data entities in each type of data entity. The knowledge graph construction module is used to construct a knowledge graph corresponding to the target scenario based on the correspondence between each user entity and each data entity and the data association between each data entity. In the knowledge graph, each entity is used as a node and the relationship between each entity is used as an edge connecting the nodes.

13. A computer program product comprising instructions that, when run on a computer or processor, cause the computer or processor to perform the steps of the method as claimed in any one of claims 1 to 11.

14. A computer storage medium storing a plurality of instructions adapted for loading by a processor and performing the steps of the method as claimed in any one of claims 1 to 11.

15. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method as claimed in any one of claims 1 to 11.

Images