Establishment method, evaluation method and device of clinical auxiliary diagnosis and treatment major large model

Abstract

The invention provides a clinical auxiliary diagnosis and treatment major large model establishing method and device and an evaluation method and device, and relates to the technical field of artificial intelligence of a large language model.The clinical auxiliary diagnosis and treatment major large model establishing method and device comprises the steps that professional literatures are deconstructed into knowledge points according to knowledge characteristics of clinical diagnosis and treatment, and the knowledge points are mapped to a knowledge space; constructing a question normal form, wherein the question normal form is used for constraining a coordinate mapping relationship between natural language expression of a question and the knowledge space; and according to the knowledge space and the questioning normal form, establishing a clinical auxiliary diagnosis and treatment professional large model. Therefore, common clinical diagnosis and treatment questions are mapped into the knowledge space coordinates, and the knowledge points in the professional literature are matched based on the knowledge space coordinates, so that accurate positioning retrieval of the question content is realized, and related recommendation is provided based on the spatial position relationship between the knowledge points.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology for large language models, and in particular to a method for establishing, evaluating, and installing a large model for clinical auxiliary diagnosis and treatment. Background Art

[0002] LLM (Large Language Model) is a language model based on a deep neural network architecture. It predicts the possibility of the next word or paragraph by learning a large amount of text data, so that computers can better understand and generate human language. Applying large model technology to the field of clinical auxiliary diagnosis and treatment, forming a professional large model system that provides clinical diagnosis and treatment knowledge questions and answers, can provide patients with convenient medical consultation and provide certain assistance to doctors in clinical diagnosis and treatment.

[0003] Due to the scientific nature and standardization of clinical diagnosis and treatment, when applying big model technology to the field of clinical auxiliary diagnosis and treatment to form a professional big model system that provides clinical diagnosis and treatment knowledge questions and answers, the following problems need to be solved: 1) Follow the principle of "evidence-based medicine": Measures must be taken to address the illusion problem of big models (that is, the text generated by the model does not follow the original text or does not conform to the facts). Clinical diagnosis and treatment follows the principle of "evidence-based medicine", and the implementation of all diagnostic and treatment behaviors must be based on solid evidence. These bases are authoritative medical literature, not texts predicted and generated by big model algorithms. It can be seen from this that the professional big model used for clinical auxiliary diagnosis and treatment must provide the source of the content in the medical literature in the answers it generates to meet the requirements of "evidence-based medicine". The current big language model generated based on text prediction is difficult to use in clinical practice because its automatically generated content cannot provide the source. 2) Accurately locate content and provide relevant recommendations: Whether it is a full-text search method based on keywords or an enhanced search method based on LLM vectorized similarity matching, on the one hand, it is difficult to grasp the direction of the question due to the lack of strict paradigm constraints on the questions; on the other hand, the search index established for the content is based on fine-grained keywords or synonyms, which leads to the search results being too broad and unable to accurately locate the corresponding content in professional literature. After solving the problem of accurate content location, users also want to know about similar content, and the system must be able to provide relevant recommendations accordingly.

[0004] Therefore, how to establish a large professional model for clinical auxiliary diagnosis and treatment is a technical problem that needs to be solved urgently. Summary of the invention

[0005] In view of the above problems, the embodiments of the present application provide a method for establishing, a method for evaluating and a device for a large clinical auxiliary diagnosis and treatment professional model, so as to overcome the above problems or at least partially solve the above problems.

[0006] In a first aspect of the embodiments of the present application, a method for establishing a large model of clinical auxiliary diagnosis and treatment is disclosed, the method comprising:

[0007] Deconstruct professional literature into knowledge points according to the knowledge characteristics of clinical diagnosis and treatment, and map the knowledge points into knowledge space;

[0008] Constructing a questioning paradigm, wherein the questioning paradigm is used to constrain the coordinate mapping relationship between the natural language expression of the question and the knowledge space;

[0009] Based on the knowledge space and the questioning paradigm, a large model of clinical auxiliary diagnosis and treatment is established.

[0010] In a second aspect of the embodiments of the present application, a method for evaluating a large model of clinical auxiliary diagnosis and treatment is disclosed, the method comprising:

[0011] Construct a test data set according to the priority order of the main dimension of the knowledge space, the form of the question paradigm, the value of the question paradigm variable, and the synonymous expression of the question paradigm;

[0012] Inputting the test data set into the clinical auxiliary diagnosis and treatment professional large model for processing to obtain the answer content of each test question in the test data set;

[0013] The correctness of the answers is evaluated by business experts, and the confidence of the clinical auxiliary diagnosis and treatment professional model is determined based on the error rate of the evaluation results.

[0014] In a third aspect of the embodiments of the present application, a device for establishing a large model of clinical auxiliary diagnosis and treatment is disclosed, the device comprising:

[0015] A knowledge space mapping module is used to deconstruct professional literature into knowledge points according to the knowledge characteristics of clinical diagnosis and treatment, and map the knowledge points to the knowledge space;

[0016] A questioning paradigm construction module, used to construct a questioning paradigm, wherein the questioning paradigm is used to constrain the coordinate mapping relationship between the natural language expression of the question and the knowledge space;

[0017] The large model building module is used to build a large model of clinical auxiliary diagnosis and treatment according to the knowledge space and the questioning paradigm.

[0018] In a fourth aspect of the embodiments of the present application, an evaluation device based on a large model of clinical auxiliary diagnosis and treatment is disclosed, the device comprising:

[0019] A data construction module is used to construct a test data set according to the priority order of the main dimensions of the knowledge space, the form of the question paradigm, the values ​​of the question paradigm variables, and the synonymous expressions of the question paradigm;

[0020] A question processing module, used for inputting the test data set into the clinical auxiliary diagnosis and treatment professional large model for processing, and obtaining the answer content of each test question in the test data set;

[0021] The answer evaluation module is used to evaluate the correctness of the answer content through business experts, and determine the confidence of the clinical auxiliary diagnosis and treatment professional large model based on the error rate of the evaluation results.

[0022] The fifth aspect of the embodiments of the present application discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method for establishing a large professional model of clinical assisted diagnosis and treatment described in the first aspect of the embodiments of the present application are implemented, or the steps of the method for evaluating the large professional model of clinical assisted diagnosis and treatment described in the second aspect of the embodiments of the present application are implemented.

[0023] The embodiments of the present application include the following advantages:

[0024] In the embodiment of the present application, in terms of the generation of answer content, professional literature is deconstructed into knowledge points according to the knowledge characteristics of clinical diagnosis and treatment, and the knowledge points are mapped to the knowledge space to achieve accurate positioning and retrieval of professional literature content; and based on the positional relationship of knowledge points in the knowledge space (i.e., knowledge space coordinates), the inclusion and adjacency relationship between knowledge points is also established, providing a basis for the association recommendation of knowledge points. In terms of understanding questions, through the questioning paradigm, the coordinate mapping relationship between the natural language expression of the question and the knowledge space is established, so as to establish a large model of clinical auxiliary diagnosis and treatment profession according to the knowledge space and the questioning paradigm.

[0025] In this way, based on this clinical auxiliary diagnosis and treatment professional large model, common clinical diagnosis and treatment questions can be mapped into knowledge space coordinates, and knowledge points in professional literature can be matched based on knowledge space coordinates, so as to achieve accurate positioning and retrieval of the question content, and provide relevant recommendations based on the spatial position relationship between knowledge points. Therefore, the problems of "lack of authoritative medical literature support for answers" and "difficulty in accurately locating content and providing relevant recommendations" in the application of large language models in clinical auxiliary diagnosis and treatment are solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0027] Figure 1It is a flowchart of the steps of a method for establishing a large model of clinical auxiliary diagnosis and treatment provided in an embodiment of the present application;

[0028] Figure 2 It is a schematic diagram of the dimensions of a knowledge space and the locations of knowledge points provided in an embodiment of the present application;

[0029] Figure 3 is a schematic diagram of a knowledge space construction method provided in an embodiment of the present application;

[0030] Figure 4 It is a flowchart of the steps of a method for generating answers to questions based on a large model of clinical auxiliary diagnosis and treatment professionals provided in an embodiment of the present application;

[0031] Figure 5 It is a flow chart of a method for generating answers to questions based on a large model of clinical auxiliary diagnosis and treatment professionals provided in an embodiment of the present application;

[0032] Figure 6 It is a flowchart of the steps of an evaluation method of a large model of clinical auxiliary diagnosis and treatment provided in an embodiment of the present application;

[0033] Figure 7 It is a structural schematic diagram of a device for establishing a large model of clinical auxiliary diagnosis and treatment provided in an embodiment of the present application;

[0034] Figure 8 It is a structural schematic diagram of an evaluation device for a large model of clinical auxiliary diagnosis and treatment provided in an embodiment of the present application;

[0035] Fig. 9 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0037] In order to overcome the problems of "lack of authoritative medical literature support (i.e. following the principle of "evidence-based medicine")" and "difficulty in accurately locating content and providing relevant recommendations" in the application of large language models in clinical auxiliary diagnosis and treatment, this application proposes the following technical ideas:

[0038] (1) Considering that the professional literature used to guide clinical auxiliary diagnosis and treatment mainly includes diagnosis and treatment guidelines and consensus compiled by experts in related fields, professional literature usually provides guidance measures for the diagnosis and treatment links such as admission examination, diagnosis, prognosis, and treatment of specific diseases, and these links are intertwined with elements such as medical examinations, patient indicators, drugs, and treatment plans, and there are also primary or secondary complications between diseases. Therefore, whether using keyword retrieval or vector similarity matching, it is difficult to grasp the direction of the question, and a large amount of content that the questioner does not care about will be retrieved, greatly reducing the availability of the system.

[0039] Therefore, in terms of generating answer content, the embodiment of the present application proposes a method for indexing document content based on knowledge space mapping, deconstructing professional documents into knowledge points according to the knowledge characteristics of clinical diagnosis and treatment, and mapping the knowledge points to the knowledge space, so as to achieve accurate positioning and retrieval of professional document content; and based on the positional relationship of knowledge points in the knowledge space, the inclusion and adjacency relationships between knowledge points are also established, providing a basis for the associated recommendation of knowledge points.

[0040] In terms of understanding questions, the embodiments of the present application analyze and summarize common problems in clinical diagnosis and treatment, and propose a question paradigm description method based on variable constraints, which establishes a coordinate mapping relationship between the natural language expression of the question and the knowledge space through the question paradigm.

[0041] A large clinical auxiliary diagnosis and treatment professional model is established based on the knowledge space and questioning paradigm. Based on this large clinical auxiliary diagnosis and treatment professional model, common clinical diagnosis and treatment questions can be mapped into knowledge space coordinates, and knowledge points in professional literature can be matched based on the knowledge space coordinates, so as to achieve accurate positioning and retrieval of question content, and provide relevant recommendations based on the spatial position relationship between knowledge points.

[0042] (2) Considering that different questions test different capabilities of the professional big model, before using the clinical assisted diagnosis and treatment professional big model, the user must know to what extent the professional big model can be trusted to answer, that is, the confidence of the answer is related to the type and method of the question. The embodiment of the present application tested the confidence of the clinical assisted diagnosis and treatment professional big model. To understand the confidence of the clinical assisted diagnosis and treatment professional big model in answering specific types of questions, it is necessary to analyze and classify the capabilities of the professional big model, and then design corresponding evaluation plans for evaluation. The capabilities of the professional big model are reflected in two aspects: one is the coverage of professional knowledge (depending on the input professional literature set); the other is the ability to understand questions and use professional knowledge to answer them.

[0043] The embodiment of the present application tests the coverage of professional knowledge by generating a test data set based on the variable value space in the questioning paradigm. For answering ability, multiple evaluation items are formulated, including: understanding of questions, knowledge point positioning, content comprehensiveness, and reasoning correctness. The test data set is input into the established clinical auxiliary diagnosis and treatment professional big model, and business experts evaluate the answer content generated based on the search results, and the confidence of the clinical auxiliary diagnosis and treatment professional big model is calculated through the error rate.

[0044] In conjunction with the accompanying drawings, the method for establishing and evaluating the clinical auxiliary diagnosis and treatment professional large model provided in the embodiment of the present application is described in detail in Sections 1.1, 1.2, 1.3 and 1.4 respectively.

[0045] 1.1 Method for establishing a large model of clinical auxiliary diagnosis and treatment:

[0046] Reference Figure 1 As shown, Figure 1 This is a flowchart of the steps of a method for establishing a large clinical auxiliary diagnosis and treatment professional model provided in an embodiment of the present application. Figure 1 As shown, the method for establishing the clinical auxiliary diagnosis and treatment professional large model may include steps S110 to S130:

[0047] Step S110: Deconstruct professional literature into knowledge points according to the knowledge characteristics of clinical diagnosis and treatment, and map the knowledge points to the knowledge space.

[0048] Step S120: constructing a questioning paradigm, wherein the questioning paradigm is used to constrain the coordinate mapping relationship between the natural language expression of the question and the knowledge space.

[0049] Step S130: Establish a large model of clinical auxiliary diagnosis and treatment based on the knowledge space and the questioning paradigm.

[0050] In the embodiment of the present application, the knowledge space represents the knowledge coverage of each knowledge point in the field of clinical diagnosis and treatment. Clinical diagnosis and treatment knowledge includes many aspects, mainly including: disease (disease and its stage, complications, etc.), diagnosis and treatment (diagnosis, prognosis, treatment, follow-up, etc.), medicine (drug combination used in treatment plan), examination (microbiological examination, imaging examination, physical examination, etc.), patient indicators (age, body surface area, prognosis score). Each clinical diagnosis and treatment knowledge point involves a combination of these elements, and all knowledge points constitute the knowledge coverage.

[0051] Therefore, the embodiment of the present application deconstructs professional literature into knowledge points according to the knowledge characteristics of clinical diagnosis and treatment, and maps the knowledge points to the knowledge space, so as to realize accurate positioning and retrieval of the content of professional literature; and based on the positional relationship of knowledge points in the knowledge space, the inclusion and adjacency relationship between knowledge points is also established, which provides a basis for the association recommendation of knowledge points.

[0052] In the embodiments of the present application, the questioning paradigm refers to the description method of asking questions. The questioning paradigm is constructed by analyzing and summarizing common clinical diagnosis and treatment problems to constrain the coordinate mapping relationship between the natural language expression of the question and the knowledge space. The questioning paradigm standardizes the expression of knowledge point questions that conforms to natural language.

[0053] Thus, a large model of clinical auxiliary diagnosis and treatment is established based on the knowledge space and questioning paradigm. In this way, based on the large model of clinical auxiliary diagnosis and treatment, common clinical diagnosis and treatment questions can be mapped into knowledge space coordinates, and knowledge points in professional literature can be matched based on the knowledge space coordinates, so as to achieve accurate positioning and retrieval of the question content, and provide relevant recommendations based on the spatial position relationship between knowledge points. Therefore, the problems of "lack of authoritative medical literature support for answers" and "difficulty in accurately locating content and providing relevant recommendations" in the application of large language models in clinical auxiliary diagnosis and treatment are solved.

[0054] The following sections 1.1.1 and 1.1.2 respectively explain in detail how to construct the knowledge space and the questioning paradigm.

[0055] 1.1.1 Knowledge space construction method:

[0056] Reference Figure 2 As shown, Figure 2 This is a schematic diagram of the dimensions and knowledge point locations of a knowledge space provided by an embodiment of the present application. The knowledge space includes a tree-like multi-level classification tree under multiple dimensions, and the multiple dimensions and the multi-level classification under each dimension constitute a knowledge space coordinate system. The multiple dimensions include: disease, drug, medical record acquisition, patient indicators, and diagnosis and treatment process.

[0057] Specifically, the knowledge space includes: knowledge points, knowledge point locations, knowledge point indexes, and knowledge point relationships. Among them, knowledge points refer to the subject of a specific classification under a certain dimension (for example, the disease dimension) in the professional literature of clinical diagnosis and treatment. In some embodiments, knowledge points refer to conceptual or operational statements made with specific classifications under other dimensions as prerequisites. For example, the knowledge point "drugs for treating lymphoma" here "lymphoma" is a prerequisite.

[0058] The position of a knowledge point refers to the coordinates of the knowledge point in the knowledge space (i.e., the knowledge space coordinates). The knowledge space coordinates include principal coordinates and subordinate coordinates. Among them, the dimension where the subject of the knowledge point is located is taken as the principal dimension, and the other dimensions where the prerequisites of the knowledge point are located are taken as conditional dimensions. The classification of the knowledge point on the principal dimension is defined as the principal coordinate, and the classification of the knowledge point on the conditional dimension is defined as the subordinate coordinate. The principal coordinate and the subordinate coordinate jointly determine the position of the knowledge point in the knowledge space.

[0059] The knowledge point index (i.e., knowledge point vector index) is used to retrieve knowledge points in the knowledge space. For example, the clinical auxiliary diagnosis and treatment professional large model can retrieve the knowledge points involved in the question based on the knowledge point index. The knowledge point index includes: the knowledge space coordinates and text vector index of the knowledge point.

[0060] The relationship between knowledge points includes: overlapping relationship, inclusion relationship and adjacent relationship. After the knowledge carried by professional documents is deconstructed into knowledge points, the relationship between knowledge points can be obtained through the relationship between the coordinates of knowledge points in the knowledge space; among them, overlapping relationship means that the coordinates of knowledge points are exactly the same; inclusion relationship means that the subordinate coordinates are exactly the same, and the main coordinates are classified as master-child relationship; adjacent relationship means that the subordinate coordinates are exactly the same, and the main coordinates are classified as the same-level sub-nodes under the same main node.

[0061] In a specific implementation, the above step S110 of "deconstructing professional literature into knowledge points according to the knowledge characteristics of clinical diagnosis and treatment, and mapping the knowledge points to the knowledge space" includes steps A1 to A4:

[0062] Step A1: Take each dimension as the root node of the tree-like multi-level classification tree to obtain the initialized knowledge space coordinate system.

[0063] Step A2: extract the content of the professional document and construct a document tree to obtain a document tree.

[0064] Step A3: extracting knowledge space coordinates according to the text content of each document node in the document tree to form knowledge points in the knowledge space.

[0065] Step A4: Refine the initialized knowledge space coordinate system according to the knowledge points, and establish an index for each knowledge point to obtain a knowledge space.

[0066] In the embodiment of the present application, multiple dimensions such as diseases, drugs, medical record acquisition, patient indicators, and diagnosis and treatment process are named as the root nodes of each tree-like multi-level classification tree to form an initialized knowledge space coordinate system.

[0067] Among them, professional literature refers to the diagnosis and treatment guidelines and consensus compiled by experts in related fields. Professional literature usually provides guidance measures for the diagnosis and treatment links such as hospitalization examination, diagnosis, prognosis, and treatment of specific diseases. The document tree refers to a tree composed of document nodes according to the paragraph and chapter structure of professional literature; the extracted content is obtained by extracting the content of professional literature, and the document tree is constructed based on the extracted content to obtain the document tree.

[0068] Specifically, step A2 of "extracting content from the professional document and constructing a document tree to obtain a document tree" includes: converting the professional document into a target format file; extracting text content and table content from the target format file to obtain plain text format paragraph content and table text; using natural paragraphs of the professional document as document nodes, and establishing a master-child document node relationship according to the chapter affiliation of the professional document; adding the plain text format paragraph content and the table text to corresponding document nodes to obtain a document tree.

[0069] In practical applications, professional documents are usually files in PDF format (Portable Document Format). In order to extract the content of professional documents, professional documents are converted into files in the target format (for example, HTML format files, HTML refers to HyperText Markup Language). PDF format professional documents can be converted into HTML format files using pdf2htmlex (pdf2htmlex is a tool for converting PDF files to HTML) or similar tools; and beautiful soup+cssutil (beautiful soup is a parser for parsing HTML documents, cssutil is a Python library that provides parsing and operation functions for cascading style sheets) or similar tools are used to extract the body text in the HTML file to obtain the paragraph content in plain text format; at the same time, opencv (opencv is an open source computer vision library that provides a wealth of image processing and computer vision algorithms) or similar tools are used to extract table data, that is, to identify the border lines of the table in the HTML file, and identify the column headers of the table based on the border lines, and identify the row data of the table line by line based on the border lines, so as to obtain the table text. After obtaining the paragraph content and table text in plain text format, a document tree is established based on the natural paragraphs of the professional document. The document tree establishes a master-child document node relationship based on the chapter affiliation of the professional document, thereby adding the paragraph content and table text in plain text format to the corresponding document nodes to obtain the document tree.

[0070] For example, the data structure of a document tree can be represented as:

[0071] {"id":"", / / string, the id (identity) of the document tree node. Each node has a unique id;

[0072] "parent":"", / / string, giving the parent node id of the node. If it is the root node of the document tree, the parent node id is equal to an empty string;

[0073] "children":[""], / / string array, each node has multiple child nodes, storing the id array of the child nodes; if there is no child node, the array length is 0;

[0074] "prev_node":"", / / string, the id of the previous sibling node;

[0075] "next_node":"", / / string, the id of the next sibling node;

[0076] "title":"", / / string, the theme of the current node;

[0077] "content":"", / / string, text content of the current node;

[0078] "summary":"", / / string, text summary of the current node;

[0079] "keyword":[""], / / string array, text keyword of the current node;

[0080] "kb_dimension":[""], / / string array, knowledge dimension array of the current node}.

[0081] After the document tree is obtained, step A3 is executed to extract the knowledge space coordinates according to the text content of each document node in the document tree to form knowledge points in the knowledge space. In specific implementation, the large model is allowed to extract the knowledge space coordinates according to the text content to form knowledge points in the knowledge space.

[0082] The knowledge space coordinates include principal coordinates and subordinate coordinates. Specifically, based on the text content of each document node in the document tree, the knowledge space coordinates are extracted, including: determining the dimension where the subject of the text content is located as the principal dimension, and determining the classification name of the text content on the principal dimension as the principal coordinate; determining the dimension where the premise of the text content is located as the conditional dimension, and determining the classification name of the text content on the conditional dimension as the subordinate coordinate.

[0083] In some embodiments, the step A3 of “extracting knowledge space coordinates according to the text content of each document node in the document tree” may specifically include sub-steps A31 to A33:

[0084] Step A31: Use the multiple dimensions and one alternative dimension as classification options for the text content.

[0085] Step A32: When the text content is classified as the candidate dimension, determine that the knowledge space coordinate is an empty coordinate.

[0086] Step A33: When the text content is classified into the multiple main dimensions, perform the following steps: 1) When the text content involves a new classification, add a new classification name to the knowledge space coordinate system as a knowledge space coordinate; 2) When the text content involves an existing classification with the same name, use the classification with the same name as the knowledge space coordinate; 3) When the text content involves an existing identical classification and the classification name is different, use the existing classification name as the knowledge space coordinate.

[0087] In the embodiment of the present application, the knowledge space coordinates include principal coordinates and subordinate coordinates, and both the principal coordinates and subordinate coordinates are determined in the manner of steps A31 to A33 described above. Among them, five dimensions of "disease, drug, medical record acquisition, patient indicators, and diagnosis and treatment process" are used as classification options for text content; at the same time, an alternative dimension is added as a classification option, and the alternative dimension is a classification other than the five dimensions; by setting the classification options, the clinical auxiliary diagnosis and treatment professional large model is required to select the content classification option to which the document node belongs. For text content classified as an alternative dimension, the knowledge space coordinates are determined to be empty coordinates, that is, the subject of the text content is not in the knowledge space dimension and non-clinical knowledge is not included. In the case where the text content is classified as the principal dimension (that is, the subject of the text content is in the knowledge space coordinate system), the knowledge space coordinates are determined in the three ways of step A33.

[0088] After obtaining the knowledge points, step A4 is executed to refine the initialized knowledge space coordinate system according to the knowledge points, and to establish an index for each knowledge point to obtain the knowledge space. Refining the initialized knowledge space coordinate system according to the knowledge points means: in the process of continuously generating new knowledge points, according to the sub-classification of the new knowledge points, the sub-classification of each dimension of the knowledge space coordinate system is continuously improved.

[0089] Among them, establishing the index of each knowledge point (i.e., the knowledge point vector index) means: establishing the knowledge space coordinates of the knowledge point and the text vector index of the knowledge point. Specifically, establishing the index of each knowledge point includes: generating a text content vector according to the text content of each document node of the document tree; and establishing a knowledge point vector index for each document node of the document tree in a vector database according to the text content vector.

[0090] Through the above implementation process, a knowledge space coordinate system consisting of five dimensions, namely diseases, drugs, medical record acquisition, patient indicators, and diagnosis and treatment process, as well as sub-categories under each dimension, was developed for clinical diagnosis and treatment professional knowledge. Professional literature was deconstructed into a document tree, and the knowledge space coordinates were extracted from the nodes of the document tree to form knowledge points, which were then mapped to the knowledge space.

[0091] For example, Figure 3It is a schematic diagram of a knowledge space construction method provided by an embodiment of the present application, and the knowledge space is obtained by extracting content from patent documents, constructing knowledge, maintaining knowledge space, and indexing. Specifically, content extraction refers to: converting professional documents into target format files (for example, converting professional documents in PDF format into HTML format files), thereby extracting the text content and table content of the target format files respectively, and obtaining plain text format paragraph content and table text; and taking the natural paragraphs of professional documents as document nodes, establishing a master-child document node relationship according to the chapter affiliation of professional documents; adding the plain text format paragraph content and table text to the corresponding document nodes to obtain a document tree.

[0092] Constructing knowledge means: extracting knowledge space coordinates from the text content of each document node in the document tree to form knowledge points in the knowledge space; and generating text content vectors based on the text content of each document node in the document tree. Knowledge space maintenance means: using knowledge points to refine the initialized knowledge space coordinate system to obtain the knowledge space. Indexing means: based on the text content vector, a knowledge point vector index is established in the vector database for each document node in the document tree. It should be noted that constructing knowledge, maintaining the knowledge space and establishing the index are carried out simultaneously, that is, in the process of extracting knowledge points, the knowledge space is also maintained and the knowledge point vector index is established.

[0093] In this way, accurate positioning and retrieval of professional literature content can be achieved through the knowledge space; and based on the positional relationship of knowledge points in the knowledge space, the inclusion and adjacency relationships between knowledge points are also established, providing a basis for the associated recommendation of knowledge points.

[0094] 1.1.2 Questioning paradigm construction method:

[0095] In a specific implementation, the “constructing a questioning paradigm” in the above step S120 includes steps B1 and B2:

[0096] Step B1: Determine the subject of the knowledge point involved in the question as the main dimension of the paradigm, and determine the dimension where the premise of the knowledge point involved in the question is located as the conditional dimension of the paradigm.

[0097] Step B2: constructing a questioning paradigm based on the main dimension and the conditional dimension; wherein the questioning paradigm includes: an original paradigm including variables and rule symbols, and an intermediate paradigm including variables.

[0098] In the embodiment of the present application, the questioning paradigm is used to standardize the expression of knowledge point questions that conforms to natural language, and the questioning paradigm contains one or more variables. Compared with the intermediate paradigm, the original paradigm contains rule symbols. If the original paradigm is removed from the rule symbols, the questioning paradigm is the intermediate paradigm.

[0099] For example, for the original paradigm: [1 can][1X101], what diseases can be differentially diagnosed with A001?

[0100] Variable description: X101 is a lubricating word variable, and its value range is: "required", "should", and A001 is a disease name variable, and its value range is common disease names.

[0101] Rule description: The content enclosed in "[]" is optional content; the number following "[" represents mutually exclusive groups, that is, at most one of all the contents in the same group.

[0102] Correspondingly, examples of the intermediate paradigm corresponding to the above original paradigm include:

[0103] A: What diseases can be differentially diagnosed with A001?

[0104] B: What diseases can be differentially diagnosed between X101 and A001?

[0105] C: What diseases can be differentially diagnosed with A001?

[0106] It should be noted that the question paradigms have synonymy. When the variable values of the question paradigms are the same, the two paradigms express the same meaning, and they are called "synonymous paradigms" with each other. Examples are as follows:

[0107] 1. What diseases can be differentially diagnosed between A001X101 and?

[0108] 2. [1 can][1X101], what diseases can be differentially diagnosed with A001?

[0109] In the embodiments of the present application, the question paradigms have a main dimension and / or a conditional dimension. The theme of the knowledge points involved in the question is called the main dimension of the paradigm; the prerequisite conditions of the knowledge points involved in the question are called the conditional dimension of the paradigm. Examples of the question paradigms for different main dimensions and conditional dimensions can be seen in Table 3 in Section 1.4 below. The variable names in the question paradigms are composed of one capital letter representing the dimension / category and three Arabic numerals, a total of 126. For examples of the variable names, see Tables 4 to 13 in Section 1.4 below.

[0110] It can be understood that when a question paradigm contains one variable, the question paradigm only includes the main dimension and does not include the conditional dimension. For example, for the question paradigm "What are the precursor diseases of A001?" in item 1 of Table 3, the dimension where the variable A001 is located is the main dimension, and the question paradigm does not include the conditional dimension.

[0111] When the questioning paradigm contains multiple variables, the questioning paradigm includes the main dimension and the conditional dimension. For example, for the questioning paradigm of item 2 in Table 3, "What are the complications of [1] when [using] Y001 to treat A001 [1]?", the dimension where the variable Y001 is located is the conditional dimension, and the dimension where the variable A001 is located is the main dimension.

[0112] In a specific implementation, a questioning paradigm is constructed according to the main dimension and the conditional dimension, including: when the questioning paradigm includes multiple variables, determining variable constraints according to the values ​​of the multiple variables; and constructing a questioning paradigm according to the variable constraints, the main dimension and the conditional dimension.

[0113] In the embodiment of the present application, if a questioning paradigm has more than two variables, there may be constraints between the variable values, and the variable constraints are determined according to the values ​​of multiple variables, so as to construct a questioning paradigm based on the variable constraints. For example, taking the value constraints between the variable name "A001" and other variable names as an example, the variable constraints are shown in Tables 14 to 17.

[0114] In the embodiment of the present application, the expression of knowledge point questions in accordance with natural language is standardized through the questioning paradigm. The questioning paradigm contains one or more variables. If multiple variables are included in the same paradigm, variable constraints are allowed to be added between multiple variables. Thus, a large model of clinical auxiliary diagnosis and treatment is established based on the knowledge space and the questioning paradigm. In this way, based on the large model of clinical auxiliary diagnosis and treatment, common clinical diagnosis and treatment questions can be mapped to knowledge space coordinates, and knowledge points in professional literature can be matched based on the knowledge space coordinates, so as to achieve accurate positioning and retrieval of the question content, and provide relevant recommendations based on the spatial position relationship between knowledge points.

[0115] 1.2 Question answer generation method based on the clinical auxiliary diagnosis and treatment professional model:

[0116] Reference Figure 4 As shown, Figure 4 This is a flowchart of the steps of a method for generating answers to questions based on a large clinical auxiliary diagnosis and treatment professional model provided by an embodiment of the present application. This method is applied to the large clinical auxiliary diagnosis and treatment professional model established by the above method. Figure 4 As shown, the clinical auxiliary diagnosis and treatment professional large model can generate answers to questions according to the following steps S410 to S430.

[0117] Step S410: obtaining a target question raised by a user according to a questioning paradigm, and extracting a first knowledge space coordinate from the target question, wherein the questioning paradigm is used to constrain the coordinate mapping relationship between the natural language expression of the question and the knowledge space.

[0118] Step S420: According to the first knowledge space coordinates, knowledge points are matched from the knowledge space to obtain target knowledge points that satisfy the first knowledge space coordinates.

[0119] Step S430: Generate an answer to the target question based on the target knowledge point, and feed back the answer to the user.

[0120] In the embodiment of the present application, the clinical auxiliary diagnosis and treatment professional large model extracts the first knowledge space coordinates from the target question based on the knowledge space coordinates of the current dialogue context (i.e., the historical knowledge space coordinates). And compares the first knowledge space coordinates in the knowledge space with the knowledge point mapping formed by the deconstruction of professional literature to find the target knowledge point that meets the first knowledge space coordinates; thus, the target knowledge points are summarized and sorted based on the clinical auxiliary diagnosis and treatment professional large model to form a summary text (i.e., the answer to the target question), and finally the answer to the target question is fed back to the user.

[0121] In this way, based on the natural language processing capabilities of the large model of clinical auxiliary diagnosis and treatment, combined with the question paradigm, the coordinate information of the corresponding knowledge space is extracted from the target question, and the knowledge points in the professional literature are matched in the knowledge space based on the knowledge space coordinates in the target question, so as to achieve accurate positioning and retrieval of the question content, thereby ensuring the accuracy of the final answer to the question.

[0122] In a specific implementation, the above step S420 of "matching knowledge points from the knowledge space according to the first knowledge space coordinates to obtain target knowledge points that satisfy the first knowledge space coordinates" includes steps F1 to F4:

[0123] Step F1: Perform vector search routing based on the first knowledge space coordinates and the knowledge space coordinates of the knowledge point, and narrow the search scope from all documents to documents whose knowledge space coordinates of the root node of the document node intersect with the first knowledge space coordinates.

[0124] Step F2: Process the target question into a target question vector, and match the target question vector with the knowledge point vector index within the search range to obtain a set of semantically similar knowledge points and the similarity of each knowledge point.

[0125] Step F3: performing intersection calculation on the second knowledge space coordinates corresponding to each knowledge point and the first knowledge space coordinates to obtain a set of knowledge points satisfying the first knowledge space coordinates.

[0126] Step F4: sort each knowledge point in the knowledge point set according to the similarity of each knowledge point, and take the top N knowledge points as the target knowledge points, where N is an integer greater than 0.

[0127] In the embodiment of the present application, the documents are searched and routed according to the knowledge space coordinates of the knowledge points, which narrows the search scope of the target question vector, thereby reducing the search calculation amount and shortening the search response time. The knowledge space includes a knowledge point vector index, which vectorizes the target question, and matches the semantic similarity between the target question vector and the knowledge point vector index within the search scope, and recalls a set of semantically similar knowledge points and the similarity of each knowledge point. In this way, the knowledge points are filtered, and the second knowledge space coordinates corresponding to each knowledge point are intersected with the first knowledge space coordinates. Specifically, it is determined whether the second knowledge space coordinates corresponding to each knowledge point overlap with the first knowledge space coordinates, and the overlapping knowledge points are determined as the set of knowledge points that meet the first knowledge space coordinates. After that, the knowledge points are sorted according to the similarity of each knowledge point in step F2, and the top N knowledge points with the highest similarity are obtained as the target knowledge points.

[0128] In this way, by using knowledge space coordinates to filter knowledge points with similar semantics, irrelevant information is removed and accurate positioning of content is achieved.

[0129] In a specific embodiment, the method further comprises the following steps E1 to E3:

[0130] Step E1: Obtain a reference document set corresponding to the target knowledge point.

[0131] Step E2: Generate a set of recommended questions based on the knowledge space coordinates of the current dialogue context, where the knowledge space coordinates of the current dialogue context represent the historical knowledge space coordinates.

[0132] Step E3: Feedback the answer to the question, the reference document set and the recommended question set to the user.

[0133] In the embodiment of the present application, the corresponding references and sources can be given based on the content organization of the knowledge points, which follows the principle of evidence-based medicine for clinical diagnosis and treatment. In addition, based on the knowledge space coordinates of the current dialogue context, a set of recommended questions is generated (i.e., the adjacent coordinates and parent coordinates are returned from the knowledge space coordinates of the current dialogue context as recommended questions) to recommend adjacent or related knowledge to users. By using this method in the dialogue context, the knowledge space coordinates are used instead of the dialogue history, which can greatly shorten the context length and reduce the performance requirements for the large model operating environment.

[0134] In a specific implementation, the knowledge space coordinates of the current dialogue context are maintained in the following manner: when the knowledge space coordinates of the current dialogue context overlap with the first knowledge space coordinates, the knowledge space coordinates of the current dialogue context remain unchanged; when the knowledge space coordinates of the current dialogue context do not overlap with the first knowledge space coordinates, the first knowledge space coordinates are used as the knowledge space coordinates of the current dialogue context.

[0135] In this way, in the process of generating the answer to the target question, the knowledge space coordinates of the current dialogue context are maintained, ensuring the accuracy of the knowledge space coordinates of the current dialogue context, thereby achieving precise positioning of the content.

[0136] For example, Figure 5 This is a flowchart of a method for generating answers to questions based on a large clinical auxiliary diagnosis and treatment professional model provided in an embodiment of the present application. Specifically, the following steps G1 to G9 are included:

[0137] Step G1: In the conversation context between the user and the clinical auxiliary diagnosis and treatment professional large model, obtain the target question raised by the user according to the questioning paradigm.

[0138] Step G2: Send the target question to the clinical auxiliary diagnosis and treatment professional big model, and require the clinical auxiliary diagnosis and treatment professional big model to extract the first knowledge space coordinate (including extraction based on synonyms of subcategories) Kn from the target question based on the knowledge space coordinate Kl of the current dialogue context; if the first knowledge space coordinate Kn overlaps with the knowledge space coordinate Kl of the current dialogue context, Kl will not be changed, otherwise Kn is used instead of Kl as the knowledge space coordinate of the current dialogue context.

[0139] Step G3: Perform vector retrieval routing (i.e., document retrieval routing) based on the first knowledge space coordinates and the knowledge space coordinates of the knowledge point, and narrow the retrieval scope from all documents to documents whose knowledge space coordinates of the root node of the document node intersect with the first knowledge space coordinates.

[0140] Step G4: vectorize the target question, perform semantic similarity matching on the knowledge point text vector index within the search scope, and recall a set of semantically similar knowledge points and the similarity of each knowledge point.

[0141] Step G5: Knowledge point filtering, intersecting the second knowledge space coordinates corresponding to each knowledge point with the first knowledge space coordinates to obtain a set of knowledge points that satisfy the first knowledge space coordinates (filtering rule: the first knowledge space coordinates and the second knowledge space coordinates overlap); then sort each knowledge point in the knowledge point set according to the similarity of each knowledge point, and use the top N knowledge points as the target knowledge points, thus obtaining a set T of knowledge points with the highest similarity ranking.

[0142] Step G6: The knowledge point set T is transmitted to the clinical auxiliary diagnosis and treatment professional big model, and the clinical auxiliary diagnosis and treatment professional big model is required to summarize and organize the knowledge point set T to form a summary text content S (i.e., generate the answer to the target question).

[0143] Step G7: Obtain the reference document set R corresponding to the filtered knowledge point (ie, the target knowledge point).

[0144] Step G8: Generate a recommended question set Q based on the knowledge space coordinate K1 of the current dialogue context. The recommendation rule is to return the adjacent coordinates and the parent coordinates from the knowledge space coordinate K1 of the current dialogue context as the recommended questions.

[0145] Step G9: Return the knowledge space coordinates Kl, summary text content S, reference set R, and recommended question set Q of the current dialogue context to the user.

[0146] Through the above implementation process, when generating answers to questions asked by users, knowledge space coordinates are extracted from the conversation context and questions, and documents are retrieved and routed according to the knowledge space coordinates of knowledge points, thereby narrowing the retrieval scope of the target question vector. In this way, knowledge space coordinates are used to filter knowledge points with similar semantics within the retrieval scope, and irrelevant information is removed, thereby achieving rapid and accurate positioning of content. Based on the content organization of knowledge points, corresponding references and sources can be given, which follows the evidence-based medicine principle of clinical diagnosis and treatment. In the conversation context, knowledge space coordinates are used instead of conversation history, which can greatly shorten the context length and reduce the performance requirements for large model operating environments.

[0147] 1.3 Evaluation method of the large model of clinical auxiliary diagnosis and treatment:

[0148] In the embodiment of the present application, the confidence of the clinical auxiliary diagnosis and treatment professional model established in Section 1.1 is tested from the two aspects of "coverage of professional knowledge" and "understanding questions and applying professional knowledge to answer them".

[0149] Reference Figure 6 As shown, Figure 6 This is a flowchart of the steps of an evaluation method for a large clinical auxiliary diagnosis and treatment professional model provided in an embodiment of the present application, such as Figure 6As shown, the evaluation method of the clinical auxiliary diagnosis and treatment professional large model may include steps S610 to S630:

[0150] Step S610: construct a test data set according to the priority order of the main dimension of the knowledge space, the form of the question paradigm, the value of the question paradigm variable, and the synonymous expression of the question paradigm.

[0151] Step S620: Input the test data set into the clinical auxiliary diagnosis and treatment professional large model for processing to obtain the answer content of each test question in the test data set.

[0152] Step S630: The correctness of the answer content is evaluated by business experts, and the confidence of the clinical auxiliary diagnosis and treatment professional model is determined according to the error rate of the evaluation result.

[0153] In the embodiment of the present application, the test data set is constructed according to the priority order of the main dimension of the knowledge space, the form of the question paradigm, the value of the question paradigm variable, and the synonymous expression of the question paradigm, so as to ensure the uniformity of the distribution of the test questions in the test data set. Thus, the confidence of the large model of clinical auxiliary diagnosis and treatment is measured according to the test data set; and the business experts evaluate the answer content generated based on the search results, and reach a consensus on the evaluation results, and the confidence of various aspects of the large model of clinical auxiliary diagnosis and treatment is calculated through the error rate of the evaluation results. In this way, accurate and comprehensive confidence testing is achieved.

[0154] Among them, business experts evaluate the answer content generated based on the search results and reach a consensus on the evaluation results, including the understanding of questions, knowledge point positioning, content comprehensiveness, reasoning correctness and other evaluation items. Among them, the understanding of questions refers to whether the knowledge space dimensions involved in the questions are correctly extracted from the context of the questions and the dialogue; knowledge point positioning refers to whether the corresponding knowledge point set is retrieved according to the knowledge space dimensions of the questions, and whether there are irrelevant knowledge points; content comprehensiveness refers to whether all corresponding knowledge points are retrieved; reasoning correctness refers to whether the content extracted and summarized from the retrieved knowledge points is logical. In this way, through the evaluation items of understanding questions, knowledge point positioning, content comprehensiveness, reasoning correctness and other aspects, the question-answering ability of the large model of clinical auxiliary diagnosis and treatment is comprehensively tested.

[0155] 1.3.1 How to construct the test dataset:

[0156] In some specific implementations, the above step S610 of "constructing a test data set according to the priority order of the main dimension of the knowledge space, the form of the question paradigm, the value of the question paradigm variable, and the synonymous expression of the question paradigm" includes steps C1 to C5:

[0157] Step C1: Determine the number of questions and the set of question patterns.

[0158] The question pattern set includes question patterns that need to be tested.

[0159] Step C2: Determine the target principal dimension and the first number of questions for each target principal dimension according to the number of questions and the question paradigm set.

[0160] In the embodiment of the present application, each questioning paradigm includes a main dimension, and different questioning paradigms correspond to different questioning main dimensions. In order to ensure the uniformity of the distribution of test questions, firstly, the target main dimension (i.e., the main dimension of the knowledge space to be tested) and the number of questions corresponding to each target main dimension (i.e., the first number of questions) are determined based on the number of questions and the questioning paradigm set.

[0161] Specifically, determining the target main dimension and the first number of questions for each target main dimension according to the number of questions and the question paradigm set includes:

[0162] In the case where the number of questions is less than the number of principal dimensions in the question paradigm set, a principal dimension is randomly selected from all principal dimensions as a target principal dimension, and the number of first questions of each target principal dimension is determined to be 1;

[0163] When the number of questions is not less than the number of principal dimensions in the question paradigm set, all principal dimensions are taken as target principal dimensions, and the number of questions is evenly distributed to obtain a first number of questions for each target principal dimension.

[0164] For example, for the number of questions M, if the number of questions M is less than the number of principal dimensions in the question paradigm set, it means that all principal dimensions cannot be measured, and only some principal dimensions can be measured; at this time, a principal dimension is randomly selected from all principal dimensions as the target principal dimension, and the first number of questions Dia for each target principal dimension is determined to be 1. Otherwise, all principal dimensions are taken as target principal dimensions, and the number of questions M is evenly distributed (that is, evenly distributed according to the divisible number, and then the remaining number is randomly distributed), and finally the target principal dimension Di and the first number of questions Dia for each target principal dimension are obtained.

[0165] Step C3: According to the target main dimension and the first number of questions, determine the target question paradigm under each target main dimension and the second number of questions for each target question paradigm.

[0166] In an embodiment of the present application, each target main dimension corresponds to a different target question paradigm. In order to ensure the uniformity of the distribution of test questions, the target question paradigm under each target main dimension and the first number of questions are determined, as well as the number of questions corresponding to each target question paradigm, that is, the second number of questions.

[0167] Specifically, according to the target main dimension and the first number of questions, determining a target questioning paradigm under each target main dimension and a second number of questions for each target questioning paradigm includes:

[0168] For all the question paradigms of each target main dimension, only one representative paradigm is selected from all synonymous paradigms to form a set of question paradigms to be extracted;

[0169] In the case where the first number of questions is less than the number of question paradigms in the set of question paradigms to be extracted, randomly extracting a question paradigm from the set of question paradigms to be extracted as a target question paradigm under each target main dimension, and determining the second number of questions for each target question paradigm to be 1;

[0170] When the second number of questions is not less than the number of question paradigms in the set of question paradigms to be extracted, all question paradigms in the set of question paradigms to be extracted are taken as target question paradigms, and the first number of questions is evenly distributed to obtain the target question paradigm under each of the main dimensions and the second number of questions for each target question paradigm.

[0171] For example, for all question paradigms under the target main dimension Di, only one representative paradigm is selected from all synonymous paradigms to form the question paradigm set Pi to be extracted. If the first question quantity Din is less than the number of question paradigms in the question paradigm set Pi to be extracted, it means that all question paradigms in the question paradigm set Pi to be extracted cannot be measured, and only some question paradigms can be measured. At this time, a question paradigm is randomly extracted from the question paradigm set Pi to be extracted as the target question paradigm under each target main dimension. Otherwise, the first question quantity Din is evenly distributed on each question paradigm in the question paradigm set Pi to be extracted, and the target question paradigm under the target main dimension Di and the second question quantity Qin of each target question paradigm are obtained.

[0172] Step C4: Determine a test questioning paradigm and a third number of questions for each of the test questioning paradigms according to the target questioning paradigm and the second number of questions, wherein the test questioning paradigms are all in the form of an intermediate paradigm.

[0173] In the embodiment of the present application, the questioning paradigm includes the original paradigm and the intermediate paradigm, and the question contents of the original paradigm and the intermediate paradigm are the same. Therefore, in order to ensure the uniformity of the distribution of test questions, according to the target questioning paradigm and the second question quantity, the test questioning paradigm in the form of the intermediate paradigm and the number of questions corresponding to each test questioning paradigm, i.e., the third question quantity, are determined.

[0174] Specifically, determining a test questioning paradigm and a third number of questions for each test questioning paradigm according to the target questioning paradigm and the second number of questions includes:

[0175] In the case where the test questioning paradigm is an original paradigm, extracting an intermediate paradigm from the intermediate paradigm set corresponding to the original paradigm to form a test questioning paradigm, and evenly distributing the second number of questions to obtain a third number of questions for each test questioning paradigm;

[0176] In the case where the test questioning paradigm is an intermediate paradigm, the target questioning paradigm is used as the test questioning paradigm, and the third question number of each test questioning paradigm is equal to the second question number.

[0177] For example, for each target questioning paradigm qi and the second number of questions Qin of each target questioning paradigm, if the target questioning paradigm qi contains a rule symbol, indicating that the target questioning paradigm qi is the original paradigm, then further extract the intermediate paradigm from the intermediate paradigm set corresponding to the original paradigm as the test questioning paradigm ri (the test questioning paradigm set Ri can be obtained), and evenly distribute the second number of questions Qin, thereby obtaining the test questioning paradigm and the third number of questions Rin of each test questioning paradigm. Otherwise, the target questioning paradigm qi is used as the test questioning paradigm ri, and the third number of questions Rin of each test questioning paradigm is equal to the second number of questions Qin.

[0178] Step C5: Determine variable values ​​of the questioning paradigm according to the test questioning paradigm and the third number of questions, and generate a test data set according to the variable values ​​of the questioning paradigm.

[0179] In the embodiment of the present application, the variables in the questioning paradigm include different values. In order to ensure the uniformity of the distribution of test questions, it is necessary to determine the variable values ​​of the questioning paradigm so as to generate a corresponding test data set according to the variable values ​​of the questioning paradigm.

[0180] Specifically, according to the test questioning paradigm and the third number of questions, determining the variable values ​​of the questioning paradigm, and generating a test data set according to the variable values ​​of the questioning paradigm, including:

[0181] In the case where the test questioning paradigm has variable constraints, using the variable value pairs as the variable value set to be extracted, extracting the third number of variable pairs, replacing the variable names in the test questioning paradigm, to obtain a test data set;

[0182] When there is no variable constraint in the test questioning paradigm, the variable value range is used as the variable value set to be extracted, and the variable values ​​of the third number of questions are extracted to replace the variable names in the test questioning paradigm to obtain a test data set.

[0183] For example, for each test question paradigm ri, and the third number of generated questions Rin for each test question paradigm, if there is a variable constraint, the variable value pair is used as the set of variable values ​​to be extracted, otherwise the variable value range is used as the set of variable values ​​to be extracted; thereby extracting Rin variable values / variable value pairs to replace the corresponding variable names in the test question paradigm ri, and generating a test data set.

[0184] In the embodiment of the present application, a test data set is generated through the above steps C1 to C5 in the order of priority of the main dimension of the knowledge space, the question paradigm form, the question paradigm variable value, and the synonymous expression of the question paradigm, thereby ensuring the uniformity of the distribution of test questions in the test data set.

[0185] The following takes the number of questions as 200 as an example to further illustrate the method for generating a test data set, which specifically includes steps Q1 to Q5.

[0186] Step Q1: Determine the number of questions 200 and the set of question patterns to be generated for testing.

[0187] Step Q2: The number of main dimensions of the questioning paradigm is: 5 (including 5 dimensions: disease, medicine, medical record acquisition, patient indicators, and diagnosis and treatment process); the number of questions is greater than the main dimensions of the questioning paradigm, and the number of questions is evenly distributed on each main dimension, obtaining the dimension set D ("disease", "medication", "medical record acquisition", "patient indicators", "diagnosis and treatment process") and the first number of questions Din (200 / 5=40) under each main dimension Di.

[0188] Step Q3: For all question paradigms under the main dimension Di (taking "disease" as an example here), only one representative paradigm is selected from all synonymous paradigms to form a question paradigm set Pi to be extracted. The question paradigm set Pi to be extracted has 15 question paradigms. The first question number Din (=40) is greater than the number of question paradigms in the question paradigm set Pi to be extracted (15). The first question number is evenly distributed on each question paradigm in the question paradigm set Pi to be extracted, and the target question paradigm set Qi extracted under the disease main dimension Di is obtained, as well as the second question number Qin allocated to each target question paradigm (40 / 15: 3 are allocated to 10 target question paradigms, and 2 are allocated to 5 target question paradigms).

[0189] Step Q4: Each target question paradigm qi of the target question paradigm set Qi (taking "How to diagnose A103[,][X101] and C102?" as an example), and the second number of questions to be generated Qin (the number of extractions this time is: 3), because it contains a rule symbol, further extract the paradigm in the intermediate paradigm set corresponding to the original paradigm and evenly distribute the number of questions, to obtain the test question paradigm set Ri, and the third number of questions Rin for each test question paradigm. For example, the extraction results are shown in Table 1:

[0190] Table 1. Example of extraction results of test question paradigm

[0191] Serial number Questioning Paradigm Number of questions Related variables 1 How to diagnose A103, X101 and C102? 0 A103, X101, C102 2 Diagnose A103, how to proceed to C102? 1 A103, C102 3 How to diagnose A103X101C102? 1 A103, X101, C102 4 How to diagnose A103 and C102? 1 A103, C102

[0192] Step Q5: Variable "X101" has no variable constraint, and the variable value range ("need" or "should") is used as the variable value set to be extracted. Variables "A103" and "C102" have a constraint relationship, and the variable value pair is used as the variable value set to be extracted. The results after extraction are shown in Table 2:

[0193] Table 2 Example of extraction results using variable value pairs as the set of variable values ​​to be extracted

[0194]

[0195]

[0196] Finally, for the test question pattern set Ri, the generated test questions are as follows:

[0197] A: How to perform imaging examinations to diagnose hemophilia patients with concurrent inhibitors?

[0198] B: How should pathological examination be performed to diagnose kidney damage in multiple myeloma?

[0199] C: How to perform serological tests to diagnose cytomegalovirus infection in patients with blood diseases undergoing allogeneic hematopoietic stem cell transplantation?

[0200] 1.3.2 Testing process of the large model of clinical auxiliary diagnosis and treatment:

[0201] First, the test data set is input into the clinical auxiliary diagnosis and treatment professional big model for processing to obtain the answer content of each test question in the test data set. Then, the answer content of the clinical auxiliary diagnosis and treatment professional big model is manually evaluated, including multiple evaluation items such as understanding of the question, knowledge point positioning, content comprehensiveness, and reasoning correctness.

[0202] For example, the test data set is sent to the clinical auxiliary diagnosis and treatment professional big model to obtain the answer content Ai corresponding to the i-th question Qi; multiple experts manually evaluate the answers of the clinical auxiliary diagnosis and treatment professional big model based on the evaluation item Ej and reach a consensus, and obtain the total number of "correct" evaluation results Eij (correct / wrong) for each evaluation item; thus, according to the total number of questions and the correctness of specific evaluation items, the confidence of each evaluation item j of the clinical auxiliary diagnosis and treatment professional big model under N samples is obtained. If there are 200 samples, for the evaluation item "whether the question is understood correctly", 180 of them correctly extract the knowledge space dimension information involved in the question, then it is considered that under 200 samples, the confidence in the understanding of the question is: 180 / 200 = 90%.

[0203] It should be noted that each step of the above-mentioned method for establishing a large model of clinical auxiliary diagnosis and treatment professionals, the method for generating questions and answers based on the large model of clinical auxiliary diagnosis and treatment professionals, and the method for evaluating the large model of clinical auxiliary diagnosis and treatment professionals are all implemented through electronic equipment.

[0204] 1.4 The tables mentioned above are as follows:

[0205] Table 3 Questioning paradigm for different main dimensions

[0206]

[0207]

[0208]

[0209] Table 4 Meaning of the letters and numbers of variable names

[0210]

[0211] Table 5 Disease-related variable names starting with "A", variable names and value examples

[0212]

[0213]

[0214]

[0215] Table 6 Drug-related variable names starting with "B", variable names and value examples

[0216]

[0217] Table 7 Variable names of the medical record acquisition class start with "C", variable names and value examples

[0218]

[0219]

[0220] Table 8 Variable names of patient indicator classes begin with "D", variable names and value examples

[0221]

[0223] Table 9: Variable names of prevention classes begin with "E", examples of variable names and values

[0224] Serial number Variable Name Dimensions illustrate Example 1 E001 Treatment process. Prevention "Source of infection" and "focus of infection" 2 E002 Treatment process. Prevention "Prevention", "Prevention and Treatment", "Preventive Treatment"

[0225] Table 10 Variable names of diagnosis and prognosis categories begin with "F", variable names and value examples

[0226]

[0227]

[0228] Table 11 Variable names of treatment and efficacy categories begin with "G", variable names and value examples

[0229]

[0230]

[0231]

[0232]

[0233] Table 12: Other variable names are substitutes for medical terms that start with "X". Examples of variable names and values

[0234]

[0235]

[0236]

[0237] Note: "X912" is a noun used in the "hematological field" only when it is combined with words such as "examination", "diagnosis", and "prognosis", so it is not defined in the vector space. That is, in the "full scope of hematological diagnosis and treatment" of this project, "X912" is not studied, but only the results of "X912 examination" are used.

[0239] Table 13: Variable names of combination type refer to the substitution word composed of different types of substitution words, starting with "Y", variable names and value examples

[0240]

[0241] Note: If an alternative word is composed of "other categories (alternative words starting with X)" and another alternative word, it is not counted here. For example: "B103", the essence of this alternative word is still "drug", but the options are enriched with "X631". "C101", the essence of this alternative word is still "medical record acquisition", but the options are enriched with "X912". "F001", the essence of this alternative word is still "diagnosis", but the options are enriched with "X912".

[0242] Table 14 Variable constraints between variables "A001" and "C102"

[0243]

[0244] Table 15 Variable constraints between variable "A001" and variable "D011"

[0245]

[0246]

[0247] Table 16 Variable constraints between variable "A001" and variables "F502" and "F503"

[0248]

[0249]

[0250]

[0251]

[0252] Table 17 Variable constraints between variable "A001" and variable "X902"

[0253] Serial number A001 X902 1 CMML WHO, FAB Collaboration Group 2 ALL WHO 3 MDS WHO, FAB Collaboration Group

[0254] The present application also provides a device for establishing a large clinical auxiliary diagnosis and treatment professional model, referring to Figure 7 As shown, Figure 7 : is a schematic diagram of a structure of a device for establishing a large clinical auxiliary diagnosis and treatment professional model provided in an embodiment of the present application, the device comprising:

[0255] A knowledge space mapping module 710 is used to deconstruct professional literature into knowledge points according to the knowledge characteristics of clinical diagnosis and treatment, and map the knowledge points to the knowledge space;

[0256] A questioning paradigm construction module 720 is used to construct a questioning paradigm, wherein the questioning paradigm is used to constrain the coordinate mapping relationship between the natural language expression of the question and the knowledge space;

[0257] The large model building module 730 is used to build a large model of clinical auxiliary diagnosis and treatment according to the knowledge space and the questioning paradigm.

[0258] In an optional embodiment, the knowledge space includes a tree-like multi-level classification tree under multiple dimensions, and the multiple dimensions and the multi-level classification under each dimension constitute a knowledge space coordinate system, and the multiple dimensions include: disease, drug, medical record acquisition, patient index, diagnosis and treatment process;

[0259] The knowledge space mapping module comprises:

[0260] An initialization module is used to take each dimension as the root node of a tree-like multi-level classification tree to obtain an initialized knowledge space coordinate system;

[0261] A document tree construction module is used to extract content from the professional documents and construct a document tree to obtain a document tree;

[0262] A coordinate extraction module, used to extract knowledge space coordinates according to the text content of each document node of the document tree to form knowledge points in the knowledge space;

[0263] The refined index module is used to refine the initialized knowledge space coordinate system according to the knowledge points and to establish an index for each knowledge point to obtain the knowledge space.

[0264] In an optional embodiment, the refined index module includes:

[0265] A vector generation module, used to generate a text content vector according to the text content of each document node in the document tree;

[0266] The index building module is used to build a knowledge point vector index for each document node of the document tree in a vector database according to the text content vector.

[0267] In an optional embodiment, the document tree construction module includes:

[0268] A format conversion module, used for converting the professional document into a target format file;

[0269] A content extraction module, used to extract text content and table content of the target format file to obtain plain text format paragraph content and table text;

[0270] A node construction module, used to use natural paragraphs of the professional document as document nodes and establish a master-sub-document node relationship according to the chapter affiliation of the professional document;

[0271] The content adding module is used to add the plain text format paragraph content and the table text to the corresponding document node to obtain a document tree.

[0272] In an optional embodiment, the knowledge space coordinates include a principal coordinate and a subordinate coordinate;

[0273] The coordinate extraction module comprises:

[0274] A principal coordinate extraction module, used to determine the dimension where the subject of the text content is located as the principal dimension, and to determine the category name of the text content on the principal dimension as the principal coordinate;

[0275] The secondary coordinate extraction module is used to determine the dimension where the premise condition of the text content is located as the condition dimension, and determine the category name of the text content on the condition dimension as the secondary coordinate.

[0276] In an optional embodiment, the coordinate extraction module includes:

[0277] A classification option module, used for using the multiple dimensions and an alternative dimension as classification options for the text content;

[0278] A first knowledge space coordinate module, configured to determine that the knowledge space coordinate is an empty coordinate when the text content is classified as the candidate dimension;

[0279] The second knowledge space coordinate module is used to perform the following steps when the text content is classified into the multiple dimensions: when the text content involves a new classification, adding the new classification name to the knowledge space coordinate system as the knowledge space coordinate; when the text content involves an existing classification with the same name, using the classification with the same name as the knowledge space coordinate; when the text content involves an existing identical classification and the classification name is different, using the existing classification name as the knowledge space coordinate.

[0280] In an optional embodiment, the questioning paradigm building module includes:

[0281] A first determination module is used to determine the subject of the question involving the knowledge point as the main dimension of the paradigm, and to determine the dimension where the premise of the question involving the knowledge point is located as the conditional dimension of the paradigm;

[0282] A construction submodule, used for constructing a questioning paradigm according to the main dimension and the conditional dimension;

[0283] The questioning paradigm includes: an original paradigm including variables and rule symbols, and an intermediate paradigm including variables.

[0284] In an optional embodiment, the construction submodule is further used to determine variable constraints according to the values ​​of the multiple variables when the question paradigm includes multiple variables; and construct the question paradigm according to the variable constraints, the main dimension and the conditional dimension.

[0285] In an optional embodiment, the device further includes a question answer generation module, which is used for generating question answers using a large clinical auxiliary diagnosis and treatment professional model, and the question answer generation module includes:

[0286] A question acquisition module, used to acquire a target question raised by a user according to a questioning paradigm, and extract a first knowledge space coordinate from the target question, wherein the questioning paradigm is used to constrain the mapping relationship between the natural language expression of the question and the coordinates of the knowledge space;

[0287] A knowledge matching module, configured to match knowledge points from the knowledge space according to the first knowledge space coordinates to obtain target knowledge points that satisfy the first knowledge space coordinates;

[0288] The answer generation module is used to generate the answer to the target question based on the target knowledge point and feed back the answer to the user.

[0289] In an optional embodiment, the device further includes:

[0290] A reference acquisition module, used to acquire a reference set corresponding to the target knowledge point;

[0291] A recommended question generation module, used to generate a set of recommended questions according to the knowledge space coordinates of the current dialogue context, wherein the knowledge space coordinates of the current dialogue context represent the historical knowledge space coordinates;

[0292] The feedback module is used to feed back the answer to the question, the reference set and the recommended question set to the user.

[0293] In an optional embodiment, the knowledge matching module includes:

[0294] A search routing module, used for performing vector search routing according to the first knowledge space coordinates and the knowledge space coordinates of the knowledge point, and narrowing the search range from all documents to documents whose knowledge space coordinates of the root node of the document node intersect with the first knowledge space coordinates;

[0295] A vector matching module, used to process the target question into a target question vector, and match the target question vector with a knowledge point vector index in the search range to obtain a set of semantically similar knowledge points and the similarity of each knowledge point;

[0296] An intersection calculation module, used for performing intersection calculation on the second knowledge space coordinates corresponding to each knowledge point and the first knowledge space coordinates to obtain a set of knowledge points satisfying the first knowledge space coordinates;

[0297] The sorting module is used to sort each knowledge point in the knowledge point set according to the similarity of each knowledge point, and take the top N knowledge points as the target knowledge points, where N is an integer greater than 0.

[0298] In an optional embodiment, the device further includes a maintenance module, which is used to maintain the knowledge space coordinates of the current dialogue context, and the maintenance module includes:

[0299] A coordinate overlapping module, configured to keep the knowledge space coordinates of the current dialogue context unchanged when the knowledge space coordinates of the current dialogue context overlap with the first knowledge space coordinates;

[0300] The coordinate non-overlapping module is used to use the first knowledge space coordinates as the knowledge space coordinates of the current dialogue context when the knowledge space coordinates of the current dialogue context and the first knowledge space coordinates do not overlap.

[0301] The present application also provides a clinical auxiliary diagnosis and treatment professional large model evaluation device, referring to Figure 8 As shown, Figure 8 : is a schematic diagram of a structure of an evaluation device for a large model of clinical auxiliary diagnosis and treatment provided in an embodiment of the present application, the device comprising:

[0302] A data construction module 810 is used to construct a test data set according to the priority order of the main dimension of the knowledge space, the form of the question paradigm, the value of the question paradigm variable, and the synonymous expression of the question paradigm;

[0303] The question processing module 820 is used to input the test data set into the clinical auxiliary diagnosis and treatment professional large model for processing to obtain the answer content of each test question in the test data set;

[0304] The answer evaluation module 830 is used to evaluate the correctness of the answer content through business experts, and determine the confidence of the clinical auxiliary diagnosis and treatment professional large model based on the error rate of the evaluation result.

[0305] In an optional embodiment, the data construction module includes:

[0306] The second determination module is used to determine the number of questions and the set of question paradigms;

[0307] A third determination module is used to determine the target main dimension and the first number of questions for each target main dimension according to the number of questions and the question paradigm set;

[0308] A fourth determination module is used to determine a target questioning paradigm under each target main dimension and a second questioning quantity of each target questioning paradigm according to the target main dimension and the first questioning quantity;

[0309] A fifth determination module is used to determine a test question paradigm and a third number of questions for each of the test question paradigms according to the target question paradigm and the second number of questions, wherein the test question paradigms are all in the form of an intermediate paradigm;

[0310] The sixth determination module is used to determine the variable values ​​of the questioning paradigm according to the test questioning paradigm and the third number of questions, and generate a test data set according to the variable values ​​of the questioning paradigm.

[0311] In an optional embodiment, the third determining module includes:

[0312] A first random extraction module is used to randomly extract a principal dimension from all principal dimensions as a target principal dimension when the number of questions is less than the number of principal dimensions in the question paradigm set, and to determine the first number of questions of each target principal dimension to be 1;

[0313] The first evenly distributed module is used to take all the main dimensions as target main dimensions and evenly distribute the number of questions to obtain the first number of questions for each target main dimension when the number of questions is not less than the number of main dimensions in the question paradigm set.

[0314] In an optional embodiment, the fourth determining module includes:

[0315] The first selection module is used for selecting only one representative paradigm from all synonymous paradigms of all question paradigms of each target main dimension to form a set of question paradigms to be extracted;

[0316] A second random extraction module is used for randomly extracting a question paradigm from the set of question paradigms to be extracted as a target question paradigm under each target main dimension when the first number of questions is less than the number of question paradigms in the set of question paradigms to be extracted, and determining the second number of questions for each target question paradigm to be 1;

[0317] The second evenly distributed module is used to take all question paradigms in the set of question paradigms to be extracted as target question paradigms and evenly distribute the first number of questions to obtain the target question paradigm under each of the main dimensions and the second number of questions for each target question paradigm when the second number of questions is not less than the number of question paradigms in the set of question paradigms to be extracted.

[0318] In an optional embodiment, the fifth determining module includes:

[0319] A third even distribution module is used for extracting intermediate paradigms from the intermediate paradigm set corresponding to the original paradigm to form a test question paradigm when the test question paradigm is the original paradigm, and evenly distributing the second number of questions to obtain the third number of questions for each test question paradigm;

[0320] The fourth evenly distributing module is used to use the target questioning paradigm as the test questioning paradigm when the test questioning paradigm is an intermediate paradigm, and the third question quantity of each test questioning paradigm is equal to the second question quantity.

[0321] In an optional embodiment, the sixth determining module includes:

[0322] A first replacement module is used for, when there is a variable constraint in the test questioning paradigm, taking a variable value pair as a variable value set to be extracted, extracting the third number of variable pairs, replacing the variable names in the test questioning paradigm, and obtaining a test data set;

[0323] The second replacement module is used to extract the variable values ​​of the third number of questions by taking the variable value range as the set of variable values ​​to be extracted when there is no variable constraint in the test question paradigm, so as to replace the variable name in the test question paradigm and obtain a test data set.

[0324] The present application also provides an electronic device, referring to Fig. 9 , Fig. 9 Schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig. 9 As shown, the electronic device 900 includes: a memory 910 and a processor 920. The memory 910 and the processor 920 are connected via a bus communication. A computer program is stored in the memory 910. The computer program can be executed on the processor 920 to implement the steps of the method for establishing a large professional model of clinical auxiliary diagnosis and treatment described in the embodiment of the present application, or the steps of the method for evaluating the large professional model of clinical auxiliary diagnosis and treatment described in the embodiment of the present application.

[0325] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0326] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods and devices according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0327] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0328] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0329] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0330] The above is a detailed introduction to the method for establishing, evaluating and controlling a large clinical auxiliary diagnosis and treatment professional model provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for establishing a large model of clinical auxiliary diagnosis and treatment, characterized in that: The method comprises: Deconstruct professional literature into knowledge points according to the knowledge characteristics of clinical diagnosis and treatment, and map the knowledge points into knowledge space; Constructing a questioning paradigm, wherein the questioning paradigm is used to constrain the coordinate mapping relationship between the natural language expression of the question and the knowledge space; Based on the knowledge space and the questioning paradigm, a large model of clinical auxiliary diagnosis and treatment is established.

2. The method for establishing a large clinical auxiliary diagnosis and treatment professional model according to claim 1 is characterized in that: The knowledge space includes a tree-like multi-level classification tree under multiple dimensions, and the multiple dimensions and the multi-level classification under each dimension constitute a knowledge space coordinate system, and the multiple dimensions include: disease, medicine, medical record acquisition, patient indicators, and diagnosis and treatment process; According to the knowledge characteristics of clinical diagnosis and treatment, professional literature is deconstructed into knowledge points, and the knowledge points are mapped to the knowledge space, including: Each dimension is used as the root node of the tree-like multi-level classification tree to obtain the initial knowledge space coordinate system; Extracting content and constructing a document tree for the professional documents to obtain a document tree; Extracting knowledge space coordinates according to the text content of each document node of the document tree to form knowledge points in the knowledge space; The initialized knowledge space coordinate system is refined according to the knowledge points, and an index of each knowledge point is established to obtain a knowledge space.

3. The method for establishing a large clinical auxiliary diagnosis and treatment professional model according to claim 2 is characterized in that: Create an index for each knowledge point, including: Generate a text content vector according to the text content of each document node in the document tree; According to the text content vector, a knowledge point vector index is established for each document node of the document tree in a vector database.

4. The method for establishing a large clinical auxiliary diagnosis and treatment professional model according to claim 2 is characterized in that: The professional documents are subjected to content extraction and document tree construction to obtain a document tree, including: Convert the professional documents into target format files; Extracting text content and table content from the target format file to obtain plain text format paragraph content and table text; Taking the natural paragraphs of the professional document as document nodes, establishing a master-sub-document node relationship according to the chapter affiliation of the professional document; The plain text format paragraph content and the table text are added to corresponding document nodes to obtain a document tree.

5. The method for establishing a large clinical auxiliary diagnosis and treatment professional model according to claim 2, characterized in that: The knowledge space coordinates include principal coordinates and subordinate coordinates; Extracting knowledge space coordinates according to the text content of each document node of the document tree includes: The dimension where the subject of the text content is located is determined as the main dimension, and the category name of the text content on the main dimension is determined as the main coordinate; The dimension where the prerequisite of the text content is located is determined as the conditional dimension, and the category name of the text content on the conditional dimension is determined as the secondary coordinate.

6. The method for establishing a large clinical auxiliary diagnosis and treatment professional model according to claim 2, characterized in that: Extracting knowledge space coordinates according to the text content of each document node of the document tree includes: Using the multiple dimensions and an alternative dimension as classification options for the text content; In the case where the text content is classified as the candidate dimension, determining that the knowledge space coordinate is an empty coordinate; When the text content is classified into the multiple dimensions, the following steps are performed: In the case where the text content involves a new category, the new category name is added to the knowledge space coordinate system as a knowledge space coordinate; In the case where the text content involves an existing category with the same name, the category with the same name is used as the knowledge space coordinate; When the text content involves an existing identical classification and the classification name is different, the existing classification name is used as the knowledge space coordinate.

7. The method for establishing a large clinical auxiliary diagnosis and treatment professional model according to claim 1, characterized in that: Construct a questioning paradigm, including: The subject of the question involving the knowledge point is determined as the main dimension of the paradigm, and the dimension of the premise of the question involving the knowledge point is determined as the conditional dimension of the paradigm; According to the main dimension and the conditional dimension, a questioning paradigm is constructed; The questioning paradigm includes: an original paradigm including variables and rule symbols, and an intermediate paradigm including variables.

8. The method for establishing a large clinical auxiliary diagnosis and treatment professional model according to claim 7, characterized in that: According to the main dimension and the conditional dimension, a questioning paradigm is constructed, including: In the case where the questioning paradigm includes multiple variables, determining variable constraints according to the values ​​of the multiple variables; A questioning paradigm is constructed according to the variable constraints, the main dimensions and the conditional dimensions.

9. A method for establishing a large clinical auxiliary diagnosis and treatment professional model according to any one of claims 1 to 8, characterized in that: The clinical auxiliary diagnosis and treatment professional large model generates answers to questions according to the following steps: Acquire a target question raised by a user according to a questioning paradigm, and extract a first knowledge space coordinate from the target question, wherein the questioning paradigm is used to constrain a mapping relationship between a natural language expression of the question and the coordinates of the knowledge space; According to the first knowledge space coordinates, matching knowledge points from the knowledge space to obtain target knowledge points that meet the first knowledge space coordinates; An answer to the target question is generated based on the target knowledge point, and the answer is fed back to the user.

10. The method for establishing a large clinical auxiliary diagnosis and treatment professional model according to claim 9, characterized in that: The method further comprises: Obtain a reference set corresponding to the target knowledge point; Generate a set of recommended questions according to the knowledge space coordinates of the current conversation context, wherein the knowledge space coordinates of the current conversation context represent the historical knowledge space coordinates; Feedback the answer to the question to the user, including: The answer to the question, the reference set and the recommended question set are fed back to the user.

11. The method for establishing a large clinical auxiliary diagnosis and treatment professional model according to claim 9, characterized in that: According to the first knowledge space coordinates, knowledge points are matched from the knowledge space to obtain target knowledge points that meet the first knowledge space coordinates, including: Performing vector search routing according to the first knowledge space coordinates and the knowledge space coordinates of the knowledge point, narrowing the search range from all documents to documents whose knowledge space coordinates of the root node of the document node intersect with the first knowledge space coordinates; Processing the target question into a target question vector, and matching the target question vector with a knowledge point vector index within the search range to obtain a set of semantically similar knowledge points and a similarity of each knowledge point; Intersecting the second knowledge space coordinates corresponding to each knowledge point with the first knowledge space coordinates to obtain a set of knowledge points that satisfy the first knowledge space coordinates; According to the similarity of each knowledge point, each knowledge point in the knowledge point set is sorted, and the top N knowledge points are used as the target knowledge points, where N is an integer greater than 0.

12. The method for establishing a large clinical auxiliary diagnosis and treatment professional model according to claim 10, characterized in that: The knowledge space coordinates of the current conversation context are maintained as follows: In the case where the knowledge space coordinates of the current dialogue context overlap with the first knowledge space coordinates, the knowledge space coordinates of the current dialogue context remain unchanged; In a case where the knowledge space coordinates of the current dialogue context and the first knowledge space coordinates do not overlap, the first knowledge space coordinates are used as the knowledge space coordinates of the current dialogue context.

13. A method for evaluating a large model of clinical auxiliary diagnosis and treatment, characterized in that: The method comprises: Construct a test data set according to the priority order of the main dimension of the knowledge space, the form of the question paradigm, the value of the question paradigm variable, and the synonymous expression of the question paradigm; Inputting the test data set into the clinical auxiliary diagnosis and treatment professional large model for processing to obtain the answer content of each test question in the test data set; The correctness of the answers is evaluated by business experts, and the confidence of the clinical auxiliary diagnosis and treatment professional model is determined based on the error rate of the evaluation results.

14. The evaluation method of a large clinical auxiliary diagnosis and treatment professional model according to claim 13 is characterized in that: According to the priority order of the main dimension of the knowledge space, the form of the question paradigm, the value of the question paradigm variable, and the synonymous expression of the question paradigm, a test data set is constructed, including: Determine the number of questions and the set of questioning paradigms; Determine a target main dimension and a first number of questions for each target main dimension according to the number of questions and the set of question paradigms; Determining a target questioning paradigm under each target main dimension and a second questioning quantity for each target questioning paradigm according to the target main dimension and the first number of questions; Determining a test questioning paradigm and a third number of questions for each of the test questioning paradigms according to the target questioning paradigm and the second number of questions, wherein the test questioning paradigms are all in the form of an intermediate paradigm; According to the test questioning paradigm and the third number of questions, variable values ​​of the questioning paradigm are determined, and a test data set is generated according to the variable values ​​of the questioning paradigm.

15. The evaluation method of a large clinical auxiliary diagnosis and treatment professional model according to claim 14 is characterized in that: Determining a target main dimension and a first number of questions for each target main dimension according to the number of questions and the set of question paradigms includes: In the case where the number of questions is less than the number of principal dimensions in the question paradigm set, a principal dimension is randomly selected from all principal dimensions as a target principal dimension, and the number of first questions of each target principal dimension is determined to be 1; When the number of questions is not less than the number of principal dimensions in the question paradigm set, all principal dimensions are taken as target principal dimensions, and the number of questions is evenly distributed to obtain a first number of questions for each target principal dimension.

16. The evaluation method of a large clinical auxiliary diagnosis and treatment professional model according to claim 14, characterized in that: Determining a target questioning paradigm under each target main dimension and a second questioning quantity for each target questioning paradigm according to the target main dimension and the first questioning quantity includes: For all the question paradigms of each target main dimension, only one representative paradigm is selected from all synonymous paradigms to form a set of question paradigms to be extracted; In the case where the first number of questions is less than the number of question paradigms in the set of question paradigms to be extracted, randomly extracting a question paradigm from the set of question paradigms to be extracted as a target question paradigm under each target main dimension, and determining the second number of questions for each target question paradigm to be 1; When the second number of questions is not less than the number of question paradigms in the set of question paradigms to be extracted, all question paradigms in the set of question paradigms to be extracted are taken as target question paradigms, and the first number of questions is evenly distributed to obtain the target question paradigm under each of the main dimensions and the second number of questions for each target question paradigm.

17. The evaluation method of a large clinical auxiliary diagnosis and treatment professional model according to claim 14 is characterized in that: Determining a test questioning paradigm and a third number of questions for each of the test questioning paradigms according to the target questioning paradigm and the second number of questions includes: In the case where the test questioning paradigm is an original paradigm, extracting an intermediate paradigm from the intermediate paradigm set corresponding to the original paradigm to form a test questioning paradigm, and evenly distributing the second number of questions to obtain a third number of questions for each test questioning paradigm; In the case where the test questioning paradigm is an intermediate paradigm, the target questioning paradigm is used as the test questioning paradigm, and the third question number of each test questioning paradigm is equal to the second question number.

18. The evaluation method of a large clinical auxiliary diagnosis and treatment professional model according to claim 14, characterized in that: Determining variable values ​​of the questioning paradigm according to the test questioning paradigm and the third number of questions, and generating a test data set according to the variable values ​​of the questioning paradigm, including: In the case where the test questioning paradigm has variable constraints, using the variable value pairs as the variable value set to be extracted, extracting the third number of variable pairs, replacing the variable names in the test questioning paradigm, to obtain a test data set; When there is no variable constraint in the test questioning paradigm, the variable value range is used as the variable value set to be extracted, and the variable values ​​of the third number of questions are extracted to replace the variable names in the test questioning paradigm to obtain a test data set.

19. A device for establishing a large model of clinical auxiliary diagnosis and treatment, characterized in that: The device comprises: A knowledge space mapping module is used to deconstruct professional literature into knowledge points according to the knowledge characteristics of clinical diagnosis and treatment, and map the knowledge points to the knowledge space; A questioning paradigm construction module, used to construct a questioning paradigm, wherein the questioning paradigm is used to constrain the coordinate mapping relationship between the natural language expression of the question and the knowledge space; The large model building module is used to build a large model of clinical auxiliary diagnosis and treatment according to the knowledge space and the questioning paradigm.

20. A clinical auxiliary diagnosis and treatment professional large model evaluation device, characterized in that: The device comprises: A data construction module is used to construct a test data set according to the priority order of the main dimensions of the knowledge space, the form of the question paradigm, the values ​​of the question paradigm variables, and the synonymous expressions of the question paradigm; A question processing module, used for inputting the test data set into the clinical auxiliary diagnosis and treatment professional large model for processing, and obtaining the answer content of each test question in the test data set; The answer evaluation module is used to evaluate the correctness of the answer content through business experts, and determine the confidence of the clinical auxiliary diagnosis and treatment professional large model based on the error rate of the evaluation results.

21. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the steps of the method for establishing the clinical assisted diagnosis and treatment professional large model as described in any one of claims 1-12, or implements the steps of the method for evaluating the clinical assisted diagnosis and treatment professional large model as described in any one of claims 13-18.

Images