Data processing method

Abstract

The embodiment of the invention provides a data processing method, and the method comprises the steps: obtaining an initial to-be-processed problem for a target industry, and extracting at least one piece of initial entity information in the initial to-be-processed problem; expanding the at least one piece of initial entity information based on an industry knowledge graph corresponding to the target industry to obtain at least one piece of to-be-processed entity information; generating a problem expansion prompt text according to the initial to-be-processed problem and the to-be-processed entity information, and inputting the problem expansion prompt text into an industry large model corresponding to the target industry to obtain at least one expansion to-be-processed problem; and querying in an industry database corresponding to the target industry based on each expanded to-be-processed problem to obtain a data query result. Through the method, the correlation and diversity of retrieval are improved, the universality and richness of query results are ensured, and diversified requirements of users can be met.

Description

Technical Field

[0001] The embodiments of the present specification relate to the field of computer technology, and in particular, to a data processing method. Background Art

[0002] In the medical field, the generation of massive amounts of literature and clinical data has made efficient and accurate retrieval of relevant information a major challenge. Traditional text retrieval solutions are usually based on keyword matching. However, the keyword matching method has a low matching degree in the medical field because medical terms are complex and diverse, with synonyms and polysemy. Using keyword matching is difficult to deal with complex medical terms and their synonyms and polysemy, which often leads to the omission of key information, thus affecting clinical decision-making and in-depth research. Summary of the invention

[0003] In view of this, an embodiment of this specification provides a data processing method. One or more embodiments of this specification also relate to a data processing device, a computing device, a computer-readable storage medium and a computer program product to solve the technical defects existing in the prior art.

[0004] According to a first aspect of an embodiment of this specification, a data processing method is provided, including:

[0005] Acquire initial pending issues for a target industry, and extract at least one initial entity information from the initial pending issues;

[0006] Expanding the at least one initial entity information based on the industry knowledge graph corresponding to the target industry to obtain at least one entity information to be processed;

[0007] Generate a problem expansion prompt text according to the initial problem to be processed and each entity information to be processed, and input the problem expansion prompt text into the industry macro model corresponding to the target industry to obtain at least one expanded problem to be processed;

[0008] Based on each expanded problem to be processed, the industry database corresponding to the target industry is queried to obtain data query results.

[0009] According to a second aspect of an embodiment of this specification, a data processing method is provided, including:

[0010] Acquire an initial problem to be processed in the medical field, and extract at least one initial entity information in the initial problem to be processed;

[0011] Expanding the at least one initial entity information based on the medical knowledge graph corresponding to the medical field to obtain at least one entity information to be processed;

[0012] Generate a problem expansion prompt text according to the initial problem to be processed and each entity information to be processed, and input the problem expansion prompt text into the medical big model corresponding to the medical field to obtain at least one expanded problem to be processed;

[0013] Based on each expanded problem to be processed, a medical database corresponding to the medical field is queried to obtain a data query result.

[0014] According to a third aspect of an embodiment of this specification, a data processing method is provided, which is applied to a cloud-side device, including:

[0015] receiving an initial pending problem for a target industry sent by a terminal-side device, and extracting at least one initial entity information in the initial pending problem;

[0016] Expanding the at least one initial entity information based on the industry knowledge graph corresponding to the target industry to obtain at least one entity information to be processed;

[0017] Generate a problem expansion prompt text according to the initial problem to be processed and each entity information to be processed, and input the problem expansion prompt text into the industry macro model corresponding to the target industry to obtain at least one expanded problem to be processed;

[0018] Based on each expanded problem to be processed, query the industry database corresponding to the target industry to obtain data query results;

[0019] The data query result is sent to the terminal side device.

[0020] According to a fourth aspect of an embodiment of this specification, there is provided a data processing device, including:

[0021] An acquisition module is configured to acquire initial pending issues for a target industry and extract at least one initial entity information from the initial pending issues;

[0022] An expansion module is configured to expand the at least one initial entity information based on the industry knowledge graph corresponding to the target industry to obtain at least one entity information to be processed;

[0023] A generating module is configured to generate a problem expansion prompt text according to the initial problem to be processed and each entity information to be processed, and input the problem expansion prompt text into the industry macro model corresponding to the target industry to obtain at least one expanded problem to be processed;

[0024] The query module is configured to query the industry database corresponding to the target industry based on each expanded problem to be processed to obtain a data query result.

[0025] According to a fifth aspect of an embodiment of this specification, a computing device is provided, including:

[0026] Memory and processor;

[0027] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, the steps of the above-mentioned data processing method are implemented.

[0028] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores a computer program / instruction, and the computer program / instruction implements the steps of the above-mentioned data processing method when executed by a processor.

[0029] According to a seventh aspect of the embodiments of this specification, there is provided a computer program product, comprising a computer program / instruction, which implements the steps of the above-mentioned data processing method when executed by a processor.

[0030] Through the method provided in the embodiments of this specification, it is possible to intelligently identify the initial entity information in the initial pending questions input by the user, ensuring that the content related to the user's intention can be processed first. The initial entity information is expanded in combination with the industry knowledge graph to obtain multiple entity information to be processed, and the various expressions corresponding to the initial pending questions can be understood, thereby improving the relevance and diversity of subsequent retrieval. The industry big model is then used to perform a fusion analysis of the entity information to be processed and the initial pending questions to generate expanded pending questions that are more in line with the query intent, thereby providing more accurate search results. Finally, the expanded pending questions are queried in the industry database, thereby ensuring the breadth and richness of the query results, which can meet the diverse needs of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a flow chart of a data processing method provided by an embodiment of this specification;

[0032] Figure 2 is a flow chart of a model training method of an industry vector matching model provided by an embodiment of this specification;

[0033] Figure 3 is a processing flow chart of a data processing method applied to the medical field provided by an embodiment of this specification;

[0034] Figure 4 is a schematic diagram of generating and expanding a problem to be processed provided by an embodiment of this specification;

[0035] Figure 5 It is a flowchart of a data processing method of a cloud-side device provided by an embodiment of this specification;

[0036] Figure 6 is a structural schematic diagram of a data processing device provided by an embodiment of this specification;

[0037] Figure 7 is an architecture diagram of a data processing system provided by one embodiment of this specification;

[0038] Figure 8 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0039] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0040] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0041] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0042] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards in the relevant regions, and corresponding operation entrances shall be provided for users to choose to authorize or refuse.

[0043] In one or more embodiments of this specification, a large model refers to a deep learning model with large-scale model parameters, which usually contains hundreds of millions, tens of billions, hundreds of billions, trillions, or even more than 10 trillion model parameters. A large model can also be called a foundation model / foundation model. The large model is pre-trained with large-scale unlabeled corpus to produce a pre-trained model with more than 100 million parameters. This model can adapt to a wide range of downstream tasks, and the model has good generalization ability, such as a large-scale language model (LLM), a multi-modal pre-training model, etc.

[0044] When the big model is used in practice, only a small number of samples are needed to fine-tune the pre-trained model and it can be applied to different tasks. The big model can be widely used in natural language processing (NLP), computer vision and other fields. Specifically, it can be applied to computer vision tasks such as visual question answering (VQA), image caption (IC), image generation, as well as natural language processing tasks such as text-based sentiment classification, text summary generation, and machine translation. The main application scenarios of the big model include digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc.

[0045] First, the terms involved in one or more embodiments of this specification are explained.

[0046] Knowledge graph: A knowledge graph is a knowledge base in the form of a graph, where nodes represent entities (such as people, places, organizations, etc.) and edges represent relationships between entities. The knowledge graph aims to provide rich contextual information so that computers can perform more complex reasoning and understanding.

[0047] Recall algorithm: Recall algorithm is a key part of information retrieval and recommendation system, which is used to quickly filter out information related to user query from a large amount of candidate data. Its purpose is to find the information most likely to meet user needs from the database based on certain similarity measures or rules.

[0048] With the development of the information age, more and more information is exploding, and massive amounts of data are continuously generated. How to efficiently and accurately retrieve and recall relevant industry information in certain specific technical fields (such as medicine, finance, military, etc.) has become a major challenge. Traditional text retrieval methods are usually based on keyword matching, but they are particularly difficult in specific fields, especially in the medical field. The professions in specific fields are complex and diverse, and there are synonyms and polysemous words. When using these methods, they often cannot meet the user's sophisticated needs for information, resulting in the omission of key information, which in turn affects clinical decision-making and the deepening of research.

[0049] In this specification, a data processing method is provided. This specification also relates to a data processing device, a computing device, a computer-readable storage medium and a computer program product, which are described in detail one by one in the following embodiments.

[0050] See also Figure 1 , Figure 1 A flow chart of a data processing method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0051] Step 102: Acquire initial pending issues for the target industry, and extract at least one initial entity information from the initial pending issues.

[0052] The target industry can be understood as a specific field, for example, the target industry can be the medical industry, the catering industry, the financial industry, etc. The initial problem to be solved can be understood as the industry-related problem for the target industry. For example, taking the medical industry as the target industry, the initial problem to be solved can be "the treatment method of **disease"; taking the catering industry as the target industry, the initial problem to be solved can be "how to make **ingredients", etc.

[0053] After obtaining the initial problem to be processed, entity extraction can be performed on the initial problem to extract the initial entity information in the initial problem to be processed. Entity extraction refers to a method of identifying entity tags with specific meanings from unstructured text. Entity extraction, also known as entity recognition or named entity learning, is an important task in the field of natural language processing. Entity extraction includes entity recognition and entity classification. Entity recognition refers to locating entity boundaries in text, and entity classification refers to assigning recognized entities to corresponding categories.

[0054] The method provided in the embodiments of this specification is a data processing method for a specific target task. Specifically, the method is applied to a terminal, which can be a smart terminal, a personal computer, a server, a cloud server, a smart wearable device, etc. The initial pending problem for the target industry is received, and at least one initial entity information in the initial pending problem is extracted, providing a data basis for subsequent data processing.

[0055] For example, if the target industry is the medical industry, the initial problem to be processed is "natural treatment of tumors". Entity extraction is performed on the initial problem to be processed to obtain two initial entity information, namely "tumor" and "natural treatment".

[0056] Step 104: Expand the at least one initial entity information based on the industry knowledge graph corresponding to the target industry to obtain at least one entity information to be processed.

[0057] Among them, the industry knowledge graph can be understood as a knowledge graph created based on the target industry. For example, when the target industry is the medical industry, the industry knowledge graph is the medical knowledge graph; when the target industry is the financial industry, the industry knowledge graph is the financial knowledge graph, and so on.

[0058] A knowledge graph is a data structure that uses graphics to represent and store knowledge. It organizes information through nodes (representing entities or concepts) and edges (representing the relationships between entities or concepts). Knowledge graphs have a wide range of uses in multiple fields, such as search engines, natural language processing, intelligent question and answer, recommendation systems, and so on.

[0059] After the above processing, at least one initial entity information can be determined and mapped to the industry knowledge graph corresponding to the target industry. Each initial entity information can be normalized and the initial entity information belonging to the same entity meaning can be normalized to unify their names.

[0060] After mapping the initial entity information to the industry knowledge graph, the initial entity information can also be expanded through the industry knowledge graph. The initial entity information is expanded based on the industry knowledge graph to obtain at least one entity information to be processed related to the initial entity information. In the implementation mode provided in this specification, the entity information to be processed can be understood as entity information that has a high degree of relevance to the initial entity information in the industry knowledge graph. The purpose of expanding the initial entity information and obtaining the entity information to be processed is to enrich the initial problem to be processed, so that there can be richer recall content in the subsequent knowledge recall process.

[0061] In a specific implementation provided in this specification, the at least one initial entity information is expanded based on the industry knowledge graph corresponding to the target industry to obtain at least one entity information to be processed, including:

[0062] Determine the initial entity information to be processed, wherein the initial entity information to be processed is any one of the at least one initial entity information;

[0063] Determine a target entity node corresponding to the initial entity information to be processed in the industry knowledge graph corresponding to the target industry;

[0064] Determine at least one reference entity node corresponding to the target entity node based on the industry knowledge graph, wherein the reference entity node is a synonym and / or hyponym corresponding to the target entity node;

[0065] The entity information corresponding to each reference entity node is determined as the entity information to be processed.

[0066] In this embodiment, an initial entity information is taken as an example for further explanation, and one initial entity information to be processed is selected from at least one initial entity information for processing. It should be noted that the initial entity information to be processed can be any one of the at least one initial entity information. In other words, each initial entity information in the at least one initial entity information needs to be processed in the same manner as the initial entity information to be processed.

[0067] After determining the initial entity information to be processed, the initial entity information to be processed can be mapped to the industry knowledge graph. Specifically, the target entity node corresponding to the initial entity information to be processed can be found in the industry knowledge graph. For example, when the initial entity information to be processed is "tumor", the target entity node of "tumor" is determined in the industry knowledge graph for subsequent data processing.

[0068] After determining the target entity node, knowledge walking can be performed in the industry knowledge graph based on the target entity node. Based on the industry knowledge graph, a reference entity node corresponding to the target entity node is determined. The reference entity node can be understood as a synonym and / or hyponym of the target entity node.

[0069] In the knowledge graph, synonyms and hyponyms are two important semantic relationships. Synonyms refer to words with the same or similar meanings that can be used interchangeably in certain contexts. In the knowledge graph, synonym relationships are used to indicate the equivalence or similarity between different words. Hyponies refer to more specific and detailed instances or subclasses of a concept (called hypernyms) in a specific classification or conceptual system. Hyponies embody the hierarchical structure and inheritance relationship between concepts.

[0070] After determining the reference entity node, the entity information corresponding to the reference entity node can be determined as the entity information to be processed. For example, taking the target entity node as "tumor", its corresponding reference entity node can be a synonym "cancer" or its hyponym "liver cancer", "lung cancer", "gastric cancer", etc., and then the entity information corresponding to the reference entity node is determined as the entity information to be processed.

[0071] The synonyms and / or hyponyms corresponding to the target entity node can enrich the content of the expanded question, making the subsequent data recall more abundant and accurate. In the process of entity expansion, the hypernyms of the target entity node will not be used, because the hypernyms will lead to a wider range of information in the subsequent recall, which cannot meet the actual query needs of users.

[0072] In another specific implementation provided in this specification, the initial entity information includes industry entity information and industry-defined information;

[0073] Expanding the at least one initial entity information based on the industry knowledge graph corresponding to the target industry to obtain at least one entity information to be processed includes:

[0074] Based on the industry knowledge graph corresponding to the target industry, the industry entity information and the industry limitation information are expanded to obtain the industry entity information to be processed corresponding to the industry entity information and the industry limitation information to be processed corresponding to the industry limitation information.

[0075] In another specific implementation provided in the embodiments of this specification, the initial entity information also includes two types in actual applications, namely industry entity information and industry-limited information. Industry entity information can be understood as entity information related to the industry, such as "cancer", "tumor", "liver cancer", etc. in the medical field; industry-limited information can be understood as limited information related to industry entity information. For example, minimally invasive surgery refers to a treatment method for a certain disease.

[0076] Taking the initial problem to be solved as “Natural treatment for tumors”, “tumor” is the industry entity information and “natural treatment” is the industry-limited information.

[0077] In the method provided in the embodiments of this specification, the initial entity information is expanded based on the industry knowledge graph, which can be understood as expanding the industry entity information and the industry-restricted information respectively, so as to obtain the to-be-processed industry entity information corresponding to the industry entity information, and the to-be-processed industry-restricted information corresponding to the industry-restricted information. It should be noted that in actual applications, the initial entity information includes at least one of the industry entity information and the industry-restricted information. If the initial entity information only includes the industry entity information or the industry-restricted information, the to-be-processed entity information obtained after data expansion is also the to-be-processed entity information corresponding to the industry entity information or the industry-restricted information. Finally, the to-be-processed industry entity information and / or the to-be-processed industry-restricted information are used together as the to-be-processed entity information.

[0078] In a specific implementation provided in this specification, the industry knowledge graph is created through the following steps:

[0079] Obtain at least one industry data corresponding to the target industry;

[0080] Extract industry entities in various industry data and entity relationships between industry entities;

[0081] An industry knowledge graph corresponding to the target industry is created based on the industry entities and the entity relationships between the industry entities.

[0082] In a specific implementation mode provided in this specification, the industry knowledge graph is pre-built and generated. Specifically, by purchasing and collecting the literature, materials and other information of the target industry in advance, the industry knowledge data set of the target industry, such as guidelines, papers, documents, blogs, patents, etc., is obtained. Information is extracted from each industry knowledge data set, and natural language processing technology (such as named entity recognition and relationship extraction) is used to extract entities, relationships and attributes from each industry data. By mining the relationship between individual entities, knowledge from different knowledge sources is integrated and disambiguated to solve problems such as entity conflicts and relationship conflicts. Then, knowledge reasoning is performed, and new knowledge is derived from existing knowledge. Based on the appropriate data structure, the corresponding industry knowledge is stored to improve efficient query, update and indexing mechanisms, and create an industry knowledge graph corresponding to the target industry.

[0083] Mining the relationships between entities includes:

[0084] If certain entities are mentioned in a document, they are considered to be possibly related, so that the correlation count of the relationship can be increased. For example, in the medical field, if certain drug entities and disease entities appear multiple times in a document, it is considered that the drug can treat the disease; if certain treatment method entities and certain disease entities appear multiple times in a document, it is considered that the treatment method is effective for the disease, and so on.

[0085] Step 106: Generate a problem expansion prompt text based on the initial problem to be processed and each entity information to be processed, and input the problem expansion prompt text into the industry big model corresponding to the target industry to obtain at least one expanded problem to be processed.

[0086] Among them, the question expansion prompt text can be understood as a prompt that needs to be input into the industry big model (also called AI big model) later in the embodiment of this specification. In the AI ​​big model, the role of the prompt is to prompt the AI ​​big model with the context of the input information and the parameter information of the input model. The prompt can help the big model better understand the intention of the input and make corresponding responses, which can improve the interpretability and accessibility of the big model. The question expansion prompt text is used to input into the industry big model, so that the industry big model generates richer questions based on the question expansion prompt text, that is, to obtain at least one expanded problem to be processed output by the industry big model.

[0087] The industry big model can be understood as the big language model corresponding to the target industry. In practical applications, the general big language model is more adaptable and can provide corresponding services in various industries. However, the general big language model does not have a deep understanding of the knowledge in certain specific fields and cannot obtain deeper knowledge in specific fields. Therefore, in the method provided in the embodiment of this specification, the corresponding industry big model is pre-trained for the target industry. The industry big model is based on the general big model and uses the industry data in the target industry for further in-depth training, so that the industry big model has a deeper application in the target industry.

[0088] The expanded questions to be processed can be understood as questions that are richer than the initial questions to be processed based on the information of the entities to be processed. They can be similar questions or subordinate questions to the initial questions to be processed. The purpose is to provide questions with more angles in the subsequent knowledge recall, so as to make the recall content richer.

[0089] In this embodiment, the initial pending issues and the pending entity information are combined into a problem expansion prompt text and input into the industry big model. The model analysis capability of the industry big model is used to enrich the issues from the input problem expansion prompt text to generate at least one expanded pending issue.

[0090] In a specific implementation manner provided in this specification, generating a question expansion prompt text according to the initial question to be processed and each entity information to be processed includes:

[0091] Get the preset extended prompt text template;

[0092] The initial problem to be processed and each entity information to be processed are added to the preset extended prompt text template to generate a problem extended prompt text.

[0093] In the embodiment of the present specification, a preset expansion prompt text template is pre-set, and the filling position of the initial pending question and the entity information to be processed is reserved in the preset expansion prompt text template. After obtaining the initial pending question and each entity information to be processed, it can be added to the preset expansion prompt text template to generate the problem expansion prompt text.

[0094] Through the expanded prompt text template of this embodiment, the format of the problem expansion prompt text is specified. In the process of generating the problem expansion prompt text, there is no need to consider some general and formatted content. It is only necessary to add the obtained initial problem to be processed and each entity information to be processed to the template, thereby improving data processing efficiency.

[0095] Step 108: Based on each expanded problem to be processed, query the industry database corresponding to the target industry to obtain data query results.

[0096] After the above processing, the initial problem to be processed can be expanded through the industry model to obtain multiple expanded problems to be processed. Then, data query can be performed in the industry database corresponding to the target industry through each expanded problem to obtain data query results.

[0097] In actual applications, each extended problem to be processed is queried in the industry database, and each extended problem to be processed will correspond to multiple results, and then the multiple results are sorted to generate the final data query result. The industry database can be understood as a knowledge base that stores the target industry.

[0098] In the above processing, the entity is first expanded to find the entity information related to each entity in the initial problem to be processed, and then the expanded problem to be processed is generated based on the entity information to be processed. This enriches the query dimension and accuracy of data query. The data query results obtained will also be richer and more detailed.

[0099] In a specific implementation provided in this specification, based on each expanded problem to be processed, a query is performed in the industry database corresponding to the target industry to obtain a data query result, including:

[0100] Determine a target expanded problem to be processed, wherein the target expanded problem to be processed is any one of the expanded problems to be processed;

[0101] Based on the target expanded problem to be processed, query the industry database corresponding to the target industry to obtain the problem query result corresponding to the target expanded problem to be processed;

[0102] The data query result is determined according to the question query results corresponding to each expanded question to be processed.

[0103] In this implementation manner, an expanded problem to be processed is still taken as an example for explanation, that is, one of the expanded problems to be processed is randomly selected as the target expanded problem to be processed.

[0104] Based on the target expanded problem to be processed, a query is performed in the industry database to obtain a problem query result corresponding to the target expanded problem to be processed. Specifically, the problem query result includes multiple industry data and the relevance of each industry data to the target expanded problem to be processed.

[0105] After performing the same processing operation on each expanded problem to be processed, the problem query results corresponding to each expanded problem to be processed are obtained, and then the problem query results are combined and displayed to obtain the data query results.

[0106] In actual applications, the combination of multiple query results can be based on a pre-set combination rule. For example, the combination rule is to display the two most relevant industry data in each query result on the homepage. For another example, the industry data in each query result is sorted according to the corresponding relevance, and combined in descending order of relevance to generate a data query result.

[0107] In order to further improve the query efficiency, data query can be performed using vector query. Specifically, based on the target expanded problem to be processed, query in the industry database corresponding to the target industry to obtain the query result corresponding to the target expanded problem to be processed, including:

[0108] S1082: Obtain a target problem vector corresponding to the target expanded problem to be processed, and obtain an industry data vector of each industry data in an industry database.

[0109] First, the target expansion problem to be processed is converted into its corresponding target problem vector, and each industry data in the industry database is converted into its corresponding industry data vector. In practical applications, the text can be converted into the corresponding vector through the existing embedding model. The specific conversion method of the vector is not limited here, and it is subject to the actual application.

[0110] Specifically, obtaining the target problem vector corresponding to the target expanded problem to be processed, and obtaining the industry data vector of each industry data in the industry database include:

[0111] Normalizing the target expansion problem to be processed through the industry knowledge graph to obtain a normalized target expansion problem, and vectorizing the normalized target expansion problem to obtain a target problem vector;

[0112] The industry data in the industry database are normalized according to the industry knowledge graph to obtain normalized industry data corresponding to each industry data, and each normalized industry data is vectorized to obtain industry data vectors corresponding to each industry data.

[0113] Furthermore, since the entity nouns of the data in various industries in the user or industry knowledge base may be called differently, in order to better perform vector matching in the subsequent process, in the specific implementation provided in this specification, the target expansion problem to be processed can be first normalized through the industry knowledge graph to obtain a normalized target expansion problem. The normalized target expansion problem is then vectorized to obtain a target problem vector.

[0114] Similarly, for each industry data in the industry database, it can also be normalized in advance through the industry knowledge graph to obtain the normalized industry data corresponding to each industry data, and then the normalized industry data can be vectorized to obtain the industry data vector corresponding to each industry data.

[0115] At this point, both the target problem vector and the industry data vector are normalized and then embedded in the industry knowledge graph. In the subsequent calculation process, the calculation results will be more accurate.

[0116] In a specific implementation provided in the present specification, in order to further improve data processing efficiency, for each industry data, after vectorization processing and obtaining the industry data vector, the industry data and the industry data vector can be saved correspondingly in an industry database. In subsequent data processing, the industry data vector in the industry database can be directly extracted for calculation without converting the industry data into corresponding industry data vectors each time, thereby improving data processing efficiency.

[0117] S1084. Calculate the vector similarity between the target problem vector and the data vectors of each industry to obtain a vector similarity result.

[0118] Once the target problem vector and the data vectors of each industry are determined, the vector similarities between the target problem vector and the data vectors of each industry can be calculated respectively, thereby obtaining the vector similarity results. It should be noted that the vector similarity results here can be understood as the set of vector similarities between the target problem vector and the data vectors of each industry.

[0119] There are many ways to calculate vector similarity, such as cosine similarity, Euclidean distance, Manhattan distance, etc. In the implementation provided in this specification, the specific form of calculating vector similarity is not limited, and it shall be subject to actual application.

[0120] In practical applications, in order to further improve data processing efficiency, a vector matching model can be pre-trained to match two vectors. Specifically, the vector similarity between the target problem vector and the data vectors of each industry is calculated to obtain the vector similarity result, including:

[0121] Selecting a target industry data vector, wherein the target industry data vector is any one of the industry data vectors;

[0122] Inputting the target problem vector and the target industry data vector into an industry vector matching model corresponding to the target industry, and obtaining a vector similarity between the target problem vector and the target industry data vector output by the industry vector matching model;

[0123] The vector similarity result is formed according to the similarity of each vector.

[0124] In this embodiment, one of the multiple industry data is used as an example for explanation, that is, a target industry data vector is selected from the multiple industry data vectors. The target problem vector and the target industry data vector are input into the industry vector matching model trained based on the target industry for processing. The industry vector matching model is pre-trained based on the industry data corresponding to the target industry and has the ability to calculate the similarity (matching degree) of two vectors. The output of the industry vector matching model is the similarity of the two vectors.

[0125] After the data vectors of each industry and the target problem vector are respectively input into the industry vector matching model for processing, the vector similarity between the data vectors of each industry and the target problem vector is obtained, thereby determining the vector similarity result.

[0126] S1086. Sort the data vectors of each industry and the target problem vector according to the vector similarity results, and select a preset number of industry data vectors corresponding to the vector similarities as the target industry data vectors.

[0127] The vector similarity results record the vector similarities between the data vectors of each industry and the target problem vector. The vector similarity results are sorted according to the similarities of each vector, and the industry data vectors corresponding to a preset number of vector similarities are selected as the target industry data vectors.

[0128] S1088. Determine that the target industry data corresponding to each target industry data vector is the problem query result corresponding to the target expanded problem to be processed.

[0129] After the target industry data vector is determined, the target industry data corresponding to the target industry data vector can be determined as the problem query result corresponding to the target expanded problem to be processed.

[0130] The target industry data vector can be understood as a vector with a high similarity to the target problem vector. Accordingly, the target industry data corresponding to the target industry data vector is the industry data with a higher correlation with the target expanded problem to be processed, and is used as the problem query result corresponding to the target expanded problem to be processed.

[0131] Through the method provided in the embodiments of this specification, it is possible to intelligently identify the initial entity information in the initial pending questions input by the user, ensuring that the content related to the user's intention can be processed first. The initial entity information is expanded in combination with the industry knowledge graph to obtain multiple entity information to be processed, and the various expressions corresponding to the initial pending questions can be understood, thereby improving the relevance and diversity of subsequent retrieval. The industry big model is then used to perform a fusion analysis of the entity information to be processed and the initial pending questions to generate expanded pending questions that are more in line with the query intent, thereby providing more accurate search results. Finally, the expanded pending questions are queried in the industry database, thereby ensuring the breadth and richness of the query results, which can meet the diverse needs of users.

[0132] In addition, the industry knowledge graph is used to normalize entities and mine relationships to provide support for subsequent calculations. The industry data vectors corresponding to the industry data are kept in the industry database. The use of pre-trained industry vector matching models can greatly improve data processing efficiency and speed up data processing. The required data query results can be quickly fed back to users.

[0133] See also Figure 2 , Figure 2 A flowchart of a model training method for an industry vector matching model provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0134] Step 202: Calculate the data similarity between the data of each industry in the industry database based on the industry knowledge graph.

[0135] In the above embodiment, a method for using an industry vector matching model is disclosed. In this implementation, a model training method for the industry vector matching model is further explained.

[0136] First, obtain the data of each industry in the industry database, and calculate the data similarity between the data of each industry through the industry knowledge graph.

[0137] Specifically, calculating the data similarity between the data of each industry in the industry database according to the industry knowledge graph includes:

[0138] Selecting to-be-processed industry data and reference industry data from the industry database;

[0139] Extracting a set of entity information to be processed in the industry data to be processed and a set of reference entity information in the reference industry data based on the industry knowledge graph;

[0140] The data similarity between the to-be-processed entity information set and the reference entity information set is calculated.

[0141] In this embodiment, the data similarity between two industry data in the industry database is used as an example for explanation. That is, the industry data to be processed and the reference industry data are selected in the industry database. Among them, the industry data to be processed can be understood as the industry data that needs to calculate the similarity with other industry data, and the reference industry data can be understood as the industry data that calculates the similarity with the industry data to be processed.

[0142] For example, an industry database with three industry data is used for explanation, and the industry database includes industry data 1, industry data 2, and industry data 3. To calculate the similarity between industry data 1 and industry data 2 and industry data 3, industry data 1 is the industry data to be processed, and industry data 2 and industry data 3 are the reference industry data.

[0143] After determining the industry data to be processed and the reference industry data, the industry knowledge graph can be used to extract the entity information set to be processed in the industry data to be processed and the reference entity information set in the reference industry data. The entity information set to be processed is the entity information set in the industry data to be processed, and the reference entity information set is the reference information set in the reference industry data.

[0144] The entities in the to-be-processed entity information set and the reference entity information set are weightedly summed, and the similarity of the two entity information sets is calculated. The similarity of the entity information sets is used as the data similarity between the to-be-processed industry data and the reference industry data.

[0145] In practical applications, in order to ensure the quality of training data, the text length of data from various industries is set to no less than 64 tokens and no longer than 521 tokens. This setting helps to ensure the representativeness of each text and ensures that the model can learn on samples of different lengths and complexities.

[0146] Step 204: Determine the positive sample industry data set and the negative sample industry data set corresponding to each industry data according to the data similarity between the industry data.

[0147] After determining the data similarity between the data of each industry, the positive sample industry data set and the negative sample industry data set corresponding to the data of each industry can be determined.

[0148] In practical applications, each industry data includes corresponding positive sample industry data sets and negative sample industry data sets. The positive sample industry data set can be understood as a set of industry data whose data similarity with the industry data is greater than or equal to a preset similarity threshold, and the negative sample data set can be understood as a set of industry data whose data similarity with the industry data is less than a preset similarity threshold.

[0149] For example, taking the example of 10 industry data, for industry data 1, there are 4 industry data whose similarity with industry data 1 is greater than or equal to the preset similarity threshold, then these 4 industry data constitute the positive sample industry data set corresponding to industry data 1; there are 5 industry data whose similarity with industry data 1 is less than the preset similarity threshold, then these 5 industry data constitute the negative sample industry data set corresponding to industry data 1.

[0150] In a specific implementation provided in this specification, the positive sample industry data set and the negative sample industry data set corresponding to each industry data are determined according to the data similarity between the industry data, including:

[0151] Adding reference industry data whose data similarity is less than a preset threshold to the negative sample industry data set of the industry data to be processed;

[0152] Input the reference industry data and the industry data to be processed whose data similarity is greater than or equal to a preset threshold into the industry big model, and obtain the judgment result output by the industry big model, wherein the judgment result is whether the reference industry data and the industry data to be processed are similar;

[0153] If the judgment result is similar, then adding the reference industry data to the positive sample industry data set of the industry data to be processed;

[0154] If the judgment result is dissimilar, the reference industry data is added to the negative sample industry data set of the industry data to be processed.

[0155] In practical applications, since the similarity of two industry data is calculated based on the similarity of entity information in the two industry data, it is possible that there is more similar entity information in the two industry data, while the two industry data are not similar. Therefore, it is necessary to further screen the data in the positive sample industry data set.

[0156] Specifically, if the data similarity is less than a preset threshold, the reference industry data is added to the negative sample industry data set corresponding to the industry data to be processed.

[0157] If the data similarity is greater than or equal to the preset threshold, the reference industry data and the industry data to be processed can also be input into the industry big model for analysis, and the industry big model determines whether the reference industry data and the industry data to be processed are similar in semantics and context, thereby determining the judgment result. The judgment result is used to determine whether the reference industry data and the industry data to be processed are similar.

[0158] If the judgment result is similar, it means that the reference industry data and the industry data to be processed are similar, and the reference industry data is added to the positive sample industry data set of the industry data to be processed; if the judgment result is dissimilar, it means that although the reference industry data and the industry data to be processed are similar in entity information, they are irrelevant after semantic and context analysis. Therefore, the reference industry data can be added to the negative sample industry data set of the industry data to be processed.

[0159] Step 206: Generate multiple positive and negative sample ternary training sets according to the positive sample industry data set and the negative sample industry data set corresponding to each industry data.

[0160] After determining the positive sample industry data set and the negative sample industry data set corresponding to each industry data, multiple positive and negative sample ternary training sets corresponding to each industry data can be generated.

[0161] Among them, the positive and negative sample ternary training set includes 3 data, namely sample industry data, positive sample industry data corresponding to the sample industry data, and negative sample industry data. By generating multiple positive and negative sample ternary training sets, the model is trained using the contrastive learning method.

[0162] In a specific implementation provided in this specification, multiple positive and negative sample ternary training sets are generated according to the positive sample industry data set and the negative sample industry data set corresponding to each industry data, including:

[0163] Determine the industry data to be trained and a positive sample industry data set and a negative sample industry data set corresponding to the industry data to be trained, wherein the industry data to be trained is any one of the industry data;

[0164] Selecting one positive sample industry data to be trained and one negative sample industry data to be trained from the positive sample industry data set and the negative sample industry data set respectively;

[0165] The industry data to be trained, the positive sample industry data to be trained and the negative sample industry data to be trained form a positive and negative sample ternary training set.

[0166] In this embodiment, one of the industry data is taken as an example for explanation, that is, one industry data to be trained is selected from multiple industry data, and a positive sample industry data set and a negative sample industry data set corresponding to the industry data to be trained are determined at the same time.

[0167] Randomly select one positive sample industry data to be trained and one negative sample industry data to be trained from the positive sample industry data set and the negative sample industry data set respectively. And the industry data to be trained, the positive sample industry data to be trained and the negative sample industry data to be trained form a positive and negative sample ternary training set. Repeat the above process multiple times to obtain multiple positive and negative sample ternary training sets.

[0168] Step 208: Train the industry vector matching model according to the multiple positive and negative sample triple training sets until the training stop condition of the industry vector matching model is reached.

[0169] After obtaining multiple positive and negative sample triple training sets, the contrastive learning method can be used to train the industry vector matching model until the model training stop condition is reached.

[0170] In a specific implementation provided in this specification, the FlagEmbedding training framework is used to train the constructed industry vector matching model. During the training process, the MRL Loss loss function (Minimum Reflection Loss) is preferably used, the purpose of which is to increase the ability of the model to adapt to different vector dimensions in order to meet diverse recall requirements.

[0171] Through the model training method of comparative learning, the similarity between the industry data to be trained and the industry data of the positive samples to be trained is shortened, while the similarity between the industry data to be trained and the industry data of the negative samples to be trained is shortened, so that the model can give two similar texts a greater similarity and two dissimilar texts a smaller similarity.

[0172] During the model training process, after a period of training, the industry vector matching model can be used to reconstruct the positive sample industry data set and the negative sample industry data set of each industry data, and the reconstructed positive sample industry data set and the negative sample industry data set can be used to construct the positive and negative sample ternary training set, and continue to train the industry vector matching model until the model's training stop condition is reached to improve the model's processing performance.

[0173] The model training method provided in one or more embodiments of the present specification combines the industry knowledge graph with the industry data in the industry database to mine the corresponding positive and negative samples of each industry data. Through similarity calculation and text normalization processing, it is possible to generate a high-quality positive and negative sample ternary training set for training the industry vector matching model, thereby enhancing the learning ability of the model and improving the accuracy of subsequent data recall.

[0174] Through repeated training and adjustment, the adaptability of the model is enhanced based on the positive and negative sample ternary training set and contrastive learning method, so that the model can be continuously optimized until the model performance reaches the expected goal.

[0175] The following combination Figure 3 , taking the application of the data processing method provided in this specification in the medical field as an example, the data processing method is further described. Figure 3 A processing flow chart of a data processing method applied to the medical field provided by an embodiment of the present specification is shown, which specifically includes the following steps.

[0176] Step 302: Acquire an initial problem to be processed in the medical field, and extract at least one initial entity information in the initial problem to be processed.

[0177] In this embodiment, the data processing method can be applied to a terminal, which obtains an initial problem to be processed in the medical field and extracts at least one initial entity information in the initial problem to be processed. Specifically, the corresponding medical terms are extracted from the initial problem to be processed, including medical entity words and medical qualifiers. Among them, the medical entity words include disease names and medical drug names. For example, the disease names of medical entity words include "cancer", "diabetes", "hypertension", etc.; the medical drug names of medical entity words include "fentanyl", "oseltamivir", "erlotinib", "hydrochloric acid", "methotrexate", etc.; medical qualifiers include "long-term treatment", "natural treatment", "clinical treatment", "clinical trial", etc.

[0178] Step 304: Expand the at least one initial entity information based on the medical knowledge graph corresponding to the medical field to obtain at least one entity information to be processed.

[0179] In this embodiment, after the medical terms are determined, knowledge walking can be performed in the medical knowledge graph based on the medical terms to obtain corresponding synonyms, derivatives, hyponyms, etc. For example, hyponyms of "cancer" include "lung cancer", "liver cancer", "bone cancer", "gastric cancer", etc.; derivatives of "fentanyl" include "sufentanil", "remifentanil", etc.; synonyms of "minimally invasive surgery" include "laparoscopic surgery", "endoscopic surgery", "small incision surgery", etc.

[0180] Step 306: Generate a question expansion prompt text based on the initial problem to be processed and each entity information to be processed, and input the question expansion prompt text into the medical big model corresponding to the medical field to obtain at least one expanded problem to be processed.

[0181] In this embodiment, after obtaining multiple entity information to be processed, the initial problem to be processed and the multiple entity information to be processed can be used to generate a problem expansion prompt text, and the problem expansion prompt text can be input into the medical big model for processing to obtain at least one expanded problem to be processed.

[0182] See also Figure 4 , Figure 4 FIG. 1 shows a schematic diagram of generating and expanding a problem to be processed provided by an embodiment of this specification. Figure 4 As shown in the figure, the initial problem to be processed is "Natural Therapy for Tumors", and entity information is extracted for the problem. The two entity information of "tumor" and "natural therapy" are obtained, and then knowledge walking is performed in the medical knowledge graph respectively. According to the two entity information of "tumor" and "natural therapy", it can be determined as follows Figure 4 The entity information to be processed is shown. Then the entity information to be processed and the initial problem to be processed "natural treatment of tumors" are spliced ​​and input into the medical model. Multiple expanded problems to be processed output by the medical model are obtained. Including: "natural treatment of benign tumors", "alternative treatment of malignant tumors", "holistic treatment of breast cancer", "natural medicine for lung cancer", "natural treatment of colon cancer", "complementary treatment of neuroma", "natural medicine for lymphoma".

[0183] Step 308: Based on each expanded problem to be processed, a medical database corresponding to the medical field is queried to obtain a data query result.

[0184] After obtaining a plurality of expanded questions to be processed, data query can be performed in the medical database according to the plurality of expanded questions to be processed, thereby obtaining a final data query result.

[0185] In practical applications, in order to speed up data query, each expanded problem to be processed can be converted into a corresponding problem vector, and the medical knowledge vector corresponding to each medical knowledge in the medical database can be obtained. Then, the similarity between the problem vector and the medical knowledge vector is calculated respectively through the pre-trained medical vector matching model, and the medical knowledge corresponding to the medical knowledge vector whose vector similarity is greater than the preset threshold is used as the final data query result.

[0186] The training of the medical vector matching model is obtained by training the positive and negative sample ternary training set generated by combining the medical database and the medical knowledge graph. In practical applications, the medical knowledge existing in any two texts in the medical database is extracted, including drug names, chemical drug names, disease names and treatment methods, etc., and the extracted medical knowledge is matched and weighted summed to calculate the similarity between the two texts. By setting a similarity threshold, the texts less than the similarity threshold are used as negative sample pairs. The texts greater than or equal to the similarity threshold are input into the medical big model to determine whether the two texts are similar. If the medical big model determines that the two texts are similar, they are used as positive sample pairs. If the medical big model determines that the two texts are not similar, they are used as negative sample pairs.

[0187] Through the above method, the positive sample set and the negative sample set corresponding to each medical data can be obtained, and then multiple positive and negative sample triplet training sets can be obtained based on this, and the positive and negative sample triplet training sets include sample data, positive samples and negative samples. The medical vector matching model is trained by multiple positive and negative sample triplet training sets until the model training stop condition is reached.

[0188] Through the method provided in the embodiments of this specification, it is possible to intelligently identify the initial entity information in the initial pending question input by the user, ensuring that the content related to the user's intention can be processed first. The initial entity information is expanded in combination with the medical knowledge graph to obtain multiple entity information to be processed, and the various expressions corresponding to the initial pending question can be understood, thereby improving the relevance and diversity of subsequent retrieval. The medical big model is then used to perform a fusion analysis of the entity information to be processed and the initial pending question to generate an expanded pending question that is more in line with the query intent, thereby providing more accurate search results. Finally, the expanded pending question is queried in the medical database, thereby ensuring the breadth and richness of the query results, which can meet the diverse needs of users.

[0189] In addition, the medical knowledge graph is used to normalize entities and mine relationships to provide support for subsequent calculations. The medical data vectors corresponding to the medical data are kept in the medical database. The use of pre-trained medical vector matching models can greatly improve data processing efficiency and speed up data processing. The required data query results can be quickly fed back to users.

[0190] See also Figure 5 , Figure 5 A flow chart of a data processing method applied to a cloud-side device provided in an embodiment of the present specification is shown, which specifically includes the following steps.

[0191] Step 502: receiving the initial pending issues for the target industry sent by the terminal-side device, and extracting at least one initial entity information in the initial pending issues.

[0192] Step 504: Expand the at least one initial entity information based on the industry knowledge graph corresponding to the target industry to obtain at least one entity information to be processed.

[0193] Step 506: Generate a problem expansion prompt text based on the initial problem to be processed and each entity information to be processed, and input the problem expansion prompt text into the industry big model corresponding to the target industry to obtain at least one expanded problem to be processed.

[0194] Step 508: Based on each expanded problem to be processed, query the industry database corresponding to the target industry to obtain data query results.

[0195] Step 510: Send the data query result to the terminal device.

[0196] In practical applications, the initial pending problems, industry knowledge graphs, industry large models, and industry databases need to work closely together to obtain the final data query results. The device on which this method needs to be deployed has good computing resources. Some end-side devices may not have the corresponding processing capabilities. Therefore, the method provided in the embodiments of this specification can also be implemented on cloud-side devices. After obtaining the initial pending problems sent by the end-side, the cloud-side device performs the above steps to obtain the final data query results, and sends the data query results to the end-side device.

[0197] Corresponding to the above method embodiment, this specification also provides a data processing device embodiment, Figure 6 FIG. 1 is a schematic diagram showing the structure of a data processing device provided by an embodiment of the present specification. Figure 6 As shown, the device comprises:

[0198] An acquisition module 602 is configured to acquire initial problems to be processed for a target industry and extract at least one initial entity information in the initial problems to be processed;

[0199] An expansion module 604 is configured to expand the at least one initial entity information based on the industry knowledge graph corresponding to the target industry to obtain at least one entity information to be processed;

[0200] A generating module 606 is configured to generate a problem expansion prompt text according to the initial problem to be processed and each entity information to be processed, and input the problem expansion prompt text into the industry macro model corresponding to the target industry to obtain at least one expanded problem to be processed;

[0201] The query module 608 is configured to query the industry database corresponding to the target industry based on each expanded problem to be processed, and obtain a data query result.

[0202] Optionally, the expansion module 604 is further configured to:

[0203] Determine the initial entity information to be processed, wherein the initial entity information to be processed is any one of the at least one initial entity information;

[0204] Determine a target entity node corresponding to the initial entity information to be processed in the industry knowledge graph corresponding to the target industry;

[0205] Determine at least one reference entity node corresponding to the target entity node based on the industry knowledge graph, wherein the reference entity node is a synonym and / or hyponym corresponding to the target entity node;

[0206] The entity information corresponding to each reference entity node is determined as the entity information to be processed.

[0207] Optionally, the initial entity information includes industry entity information and industry-limited information;

[0208] The expansion module 604 is further configured to:

[0209] Based on the industry knowledge graph corresponding to the target industry, the industry entity information and the industry limitation information are expanded to obtain the industry entity information to be processed corresponding to the industry entity information and the industry limitation information to be processed corresponding to the industry limitation information.

[0210] Optionally, the generating module 606 is further configured to:

[0211] Get the preset extended prompt text template;

[0212] The initial problem to be processed and each entity information to be processed are added to the preset extended prompt text template to generate a problem extended prompt text.

[0213] Optionally, the query module 608 is further configured to:

[0214] Determine a target expanded problem to be processed, wherein the target expanded problem to be processed is any one of the expanded problems to be processed;

[0215] Based on the target expanded problem to be processed, query the industry database corresponding to the target industry to obtain the problem query result corresponding to the target expanded problem to be processed;

[0216] The data query result is determined according to the question query results corresponding to each expanded question to be processed.

[0217] Optionally, the query module 608 is further configured to:

[0218] Obtain a target problem vector corresponding to the target expanded problem to be processed, and obtain an industry data vector of each industry data in an industry database;

[0219] Calculate the vector similarity between the target problem vector and the data vectors of each industry to obtain a vector similarity result;

[0220] Sorting the data vectors of each industry and the target problem vector according to the vector similarity results, and selecting a preset number of industry data vectors corresponding to the vector similarities as the target industry data vectors;

[0221] Determine the target industry data corresponding to each target industry data vector as the problem query result corresponding to the target expanded problem to be processed.

[0222] Optionally, the query module 608 is further configured to:

[0223] Normalizing the target expansion problem to be processed through the industry knowledge graph to obtain a normalized target expansion problem, and vectorizing the normalized target expansion problem to obtain a target problem vector;

[0224] The industry data in the industry database are normalized according to the industry knowledge graph to obtain normalized industry data corresponding to each industry data, and each normalized industry data is vectorized to obtain industry data vectors corresponding to each industry data.

[0225] Optionally, the query module 608 is further configured to:

[0226] Selecting a target industry data vector, wherein the target industry data vector is any one of the industry data vectors;

[0227] Inputting the target problem vector and the target industry data vector into an industry vector matching model corresponding to the target industry, and obtaining a vector similarity between the target problem vector and the target industry data vector output by the industry vector matching model;

[0228] The vector similarity result is formed according to the similarity of each vector.

[0229] Optionally, the device further comprises a training module configured to:

[0230] Calculate the data similarity between the data of each industry in the industry database according to the industry knowledge graph;

[0231] Determine the positive sample industry data set and the negative sample industry data set corresponding to each industry data according to the data similarity between the industry data;

[0232] Generate multiple positive and negative sample ternary training sets based on the positive sample industry data set and the negative sample industry data set corresponding to each industry data;

[0233] The industry vector matching model is trained according to multiple positive and negative sample triple training sets until the training stop condition of the industry vector matching model is reached.

[0234] Optionally, the training module is further configured to:

[0235] Selecting to-be-processed industry data and reference industry data from the industry database;

[0236] Extracting a set of entity information to be processed in the industry data to be processed and a set of reference entity information in the reference industry data based on the industry knowledge graph;

[0237] The data similarity between the to-be-processed entity information set and the reference entity information set is calculated.

[0238] Optionally, the training module is further configured to:

[0239] Adding reference industry data whose data similarity is less than a preset threshold to the negative sample industry data set of the industry data to be processed;

[0240] Input the reference industry data and the industry data to be processed whose data similarity is greater than or equal to a preset threshold into the industry big model to obtain a judgment result output by the industry big model, wherein the judgment result is whether the reference industry data and the industry data to be processed are similar;

[0241] If the judgment result is similar, then adding the reference industry data to the positive sample industry data set of the industry data to be processed;

[0242] If the judgment result is dissimilar, the reference industry data is added to the negative sample industry data set of the industry data to be processed.

[0243] Optionally, the training module is further configured to:

[0244] Determine the industry data to be trained and a positive sample industry data set and a negative sample industry data set corresponding to the industry data to be trained, wherein the industry data to be trained is any one of the industry data;

[0245] Selecting one positive sample industry data to be trained and one negative sample industry data to be trained from the positive sample industry data set and the negative sample industry data set respectively;

[0246] The industry data to be trained, the positive sample industry data to be trained and the negative sample industry data to be trained form a positive and negative sample ternary training set.

[0247] Optionally, the device further includes a knowledge graph creation module configured to:

[0248] Obtain at least one industry data corresponding to the target industry;

[0249] Extract industry entities in various industry data and entity relationships between industry entities;

[0250] An industry knowledge graph corresponding to the target industry is created based on the industry entities and the entity relationships between the industry entities.

[0251] The device provided by the embodiments of this specification can intelligently identify the initial entity information in the initial pending questions input by the user, ensuring that the content related to the user's intention can be processed first. The initial entity information is expanded in combination with the industry knowledge graph to obtain multiple entity information to be processed, and the various expressions corresponding to the initial pending questions can be understood, thereby improving the relevance and diversity of subsequent retrieval. The industry big model is then used to perform a fusion analysis of the entity information to be processed and the initial pending questions to generate expanded pending questions that are more in line with the query intent, thereby providing more accurate search results. Finally, the expanded pending questions are queried in the industry database, thereby ensuring the breadth and richness of the query results, which can meet the diverse needs of users.

[0252] In addition, the industry knowledge graph is used to normalize entities and mine relationships to provide support for subsequent calculations. The industry data vectors corresponding to the industry data are kept in the industry database. The use of pre-trained industry vector matching models can greatly improve data processing efficiency and speed up data processing. The required data query results can be quickly fed back to users.

[0253] The above is a schematic scheme of a data processing device of this embodiment. It should be noted that the technical scheme of the data processing device and the technical scheme of the above data processing method belong to the same concept, and the details not described in detail in the technical scheme of the data processing device can be referred to the description of the technical scheme of the above data processing method.

[0254] See also Figure 7 , Figure 7 An architecture diagram of a data processing system provided by an embodiment of the present specification is shown, and the data processing system may include a client 100 and a server 200;

[0255] The client 100 is used to send initial pending issues for the target industry to the server 200;

[0256] The server 200 is used to extract at least one initial entity information from the initial problem to be processed; expand the at least one initial entity information based on the industry knowledge graph corresponding to the target industry to obtain at least one entity information to be processed; generate a problem expansion prompt text according to the initial problem to be processed and each entity information to be processed, and input the problem expansion prompt text into the industry big model corresponding to the target industry to obtain at least one expanded problem to be processed; query the industry database corresponding to the target industry based on each expanded problem to be processed to obtain a data query result; send the data query result to the client 100;

[0257] The client 100 is also used to receive the data query result sent by the server 200.

[0258] The data processing system may include multiple clients 100 and a server 200, wherein the client 100 may be referred to as a client-side device and the server 200 may be referred to as a cloud-side device. Multiple clients 100 may establish a communication connection through the server 200. In a data query scenario, the server 200 is used to provide data query services between multiple clients 100. Multiple clients 100 may serve as a sender or a receiver, respectively, and achieve communication through the server 200.

[0259] The user can interact with the server 200 through the client 100 to receive data sent by other clients 100, or send data to other clients 100, etc. In the data query scenario, the user can publish a data stream to the server 200 through the client 100, and the server 200 generates a data query result based on the data stream and pushes the data query result to other clients that have established communication.

[0260] The client 100 and the server 200 are connected via a network. The network provides a medium for a communication link between the client 100 and the server 200. The network may include various connection types, such as wired or wireless communication links or optical fiber cables, etc. The data transmitted by the client 100 may need to be encoded, transcoded, compressed, etc. before being released to the server 200.

[0261] The client 100 can be a browser, an APP (Application), or a web application such as an H5 (HyperText Markup Language 5, Hypertext Markup Language Version 5) application, or a light application (also known as a mini-program, a lightweight application) or a cloud application, etc. The client 100 can be based on the software development kit (SDK, Software Development Kit) of the corresponding service provided by the server 200, such as based on the real-time communication (RTC, Real Time Communication) SDK development and acquisition. The client 100 can be deployed in an electronic device, and needs to rely on the device to run or some APPs in the device to run. For example, the electronic device can have a display screen and support information browsing, such as a personal mobile terminal such as a mobile phone, a tablet computer, a personal computer, etc. Various other types of applications can also be configured in the electronic device, such as human-computer dialogue applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0262] The server 200 may include servers that provide various services, such as servers that provide communication services to multiple clients, servers for background training that provide support for models used on clients, and servers that process data sent by clients. It should be noted that the server 200 can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. The server can also be a server of a distributed system, or a server combined with a blockchain. The server can also be a cloud server for basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0263] It is worth noting that the data processing method provided in the embodiments of this specification is generally executed by the server, but in other embodiments of this specification, the client may also have similar functions to the server, thereby executing the data processing method provided in the embodiments of this specification. In other embodiments, the data processing method provided in the embodiments of this specification may also be jointly executed by the client and the server.

[0264] Figure 8The block diagram of a computing device 800 according to an embodiment of the present specification is shown. The components of the computing device 800 include but are not limited to a memory 810 and a processor 820. The processor 820 is connected to the memory 810 via a bus 830, and a database 850 is used to store data.

[0265] The computing device 800 also includes an access device 840 that enables the computing device 800 to communicate via one or more networks 860. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 840 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).

[0266] In one embodiment of the present specification, the above components of the computing device 800 and Figure 8 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 8 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0267] The computing device 800 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 800 may also be a mobile or stationary server.

[0268] The processor 820 is used to execute the following computer program / instructions, which implement the steps of the above-mentioned data processing method when executed by the processor.

[0269] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the computing device embodiment, since it is basically similar to the data processing method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the data processing method embodiment.

[0270] An embodiment of the present specification further provides a computer-readable storage medium storing a computer program / instruction, which implements the steps of the above-mentioned data processing method when executed by a processor.

[0271] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the computer-readable storage medium embodiment, since it is basically similar to the data processing method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the data processing method embodiment.

[0272] An embodiment of the present specification also provides a computer program product, including a computer program / instruction, which implements the steps of the above data processing method when executed by a processor.

[0273] The above is a schematic scheme of a computer program product of this embodiment. It should be noted that the technical scheme of the computer program product and the technical scheme of the above data processing method belong to the same concept, and the details not described in detail in the technical scheme of the computer program product can be referred to the description of the technical scheme of the above data processing method.

[0274] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0275] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0276] It should be noted that the above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of the present specification.

[0277] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0278] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can understand and use this specification well. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data processing method, comprising: Acquire initial pending issues for a target industry, and extract at least one initial entity information from the initial pending issues; Expanding the at least one initial entity information based on the industry knowledge graph corresponding to the target industry to obtain at least one entity information to be processed; Generate a problem expansion prompt text according to the initial problem to be processed and each entity information to be processed, and input the problem expansion prompt text into the industry macro model corresponding to the target industry to obtain at least one expanded problem to be processed; Based on each expanded problem to be processed, the industry database corresponding to the target industry is queried to obtain data query results.

2. The method according to claim 1, expanding the at least one initial entity information based on the industry knowledge graph corresponding to the target industry to obtain at least one entity information to be processed, comprising: Determine the initial entity information to be processed, wherein the initial entity information to be processed is any one of the at least one initial entity information; Determine a target entity node corresponding to the initial entity information to be processed in the industry knowledge graph corresponding to the target industry; Determine at least one reference entity node corresponding to the target entity node based on the industry knowledge graph, wherein the reference entity node is a synonym and / or hyponym corresponding to the target entity node; The entity information corresponding to each reference entity node is determined as the entity information to be processed.

3. The method of claim 1, wherein the initial entity information includes industry entity information and industry-defined information; Expanding the at least one initial entity information based on the industry knowledge graph corresponding to the target industry to obtain at least one entity information to be processed includes: Based on the industry knowledge graph corresponding to the target industry, the industry entity information and the industry limitation information are expanded to obtain the industry entity information to be processed corresponding to the industry entity information and the industry limitation information to be processed corresponding to the industry limitation information.

4. The method according to claim 1, querying the industry database corresponding to the target industry based on each expanded problem to be processed to obtain data query results, comprises: Determine a target expanded problem to be processed, wherein the target expanded problem to be processed is any one of the expanded problems to be processed; Based on the target expanded problem to be processed, query the industry database corresponding to the target industry to obtain the problem query result corresponding to the target expanded problem to be processed; The data query result is determined according to the question query results corresponding to each expanded question to be processed.

5. The method according to claim 4, querying the industry database corresponding to the target industry based on the target expanded problem to be processed, and obtaining the problem query result corresponding to the target expanded problem to be processed, comprises: Obtain a target problem vector corresponding to the target expanded problem to be processed, and obtain an industry data vector of each industry data in an industry database; Calculate the vector similarity between the target problem vector and the data vectors of each industry to obtain a vector similarity result; Sorting the data vectors of each industry and the target problem vector according to the vector similarity results, and selecting a preset number of industry data vectors corresponding to the vector similarities as the target industry data vectors; Determine the target industry data corresponding to each target industry data vector as the problem query result corresponding to the target expanded problem to be processed.

6. The method according to claim 5, calculating the vector similarity between the target problem vector and the data vectors of each industry to obtain the vector similarity result, comprises: Selecting a target industry data vector, wherein the target industry data vector is any one of the industry data vectors; Inputting the target problem vector and the target industry data vector into an industry vector matching model corresponding to the target industry, and obtaining a vector similarity between the target problem vector and the target industry data vector output by the industry vector matching model; The vector similarity result is formed according to the similarity of each vector.

7. The method according to claim 6, wherein the industry vector matching model is obtained by training through the following steps: Calculate the data similarity between the data of each industry in the industry database according to the industry knowledge graph; Determine the positive sample industry data set and the negative sample industry data set corresponding to each industry data according to the data similarity between the industry data; Generate multiple positive and negative sample ternary training sets based on the positive sample industry data set and the negative sample industry data set corresponding to each industry data; The industry vector matching model is trained according to multiple positive and negative sample triple training sets until the training stop condition of the industry vector matching model is reached.

8. A data processing method, comprising: Acquire an initial problem to be processed in the medical field, and extract at least one initial entity information in the initial problem to be processed; Expanding the at least one initial entity information based on the medical knowledge graph corresponding to the medical field to obtain at least one entity information to be processed; Generate a problem expansion prompt text according to the initial problem to be processed and each entity information to be processed, and input the problem expansion prompt text into the medical big model corresponding to the medical field to obtain at least one expanded problem to be processed; Based on each expanded problem to be processed, a medical database corresponding to the medical field is queried to obtain a data query result.

9. A data processing method, applied to a cloud-side device, comprising: receiving an initial pending problem for a target industry sent by a terminal-side device, and extracting at least one initial entity information in the initial pending problem; Expanding the at least one initial entity information based on the industry knowledge graph corresponding to the target industry to obtain at least one entity information to be processed; Generate a problem expansion prompt text according to the initial problem to be processed and each entity information to be processed, and input the problem expansion prompt text into the industry macro model corresponding to the target industry to obtain at least one expanded problem to be processed; Based on each expanded problem to be processed, query the industry database corresponding to the target industry to obtain data query results; The data query result is sent to the terminal side device.

10. A computing device comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer program / instructions are executed by the processor, the steps of the method described in any one of claims 1 to 9 are implemented.

Images