A medical question and answer method and device, a storage medium, an electronic device and product

Abstract

The present disclosure relates to the field of machine learning, and provides a medical question and answer method, device, storage medium, electronic equipment and product. The method comprises: obtaining a medical description question, identifying a question type corresponding to the medical description question; in the case of a non-concept type, determining historical case information matched with the medical description question based on a pre-constructed case library; and determining knowledge information matched with the medical description question based on a pre-constructed knowledge graph and knowledge base; and determining reply information corresponding to the medical description question based on the historical case information and the knowledge information through a question and answer model. By matching the historical case information in the case library, the matched historical case information and the recalled knowledge information are input into the question and answer model at the same time, the historical case information is used as prior information, the hit rate of the knowledge information is improved, and the accuracy of the reply information is improved.

Description

Technical Field

[0001] This disclosure relates to the field of machine learning technology, and in particular to a medical question-answering method, apparatus, storage medium, electronic device, and product. Background Technology

[0002] With the continuous development of medical equipment, medical devices such as surgical robots have been widely used in clinical diagnosis and treatment.

[0003] Taking surgical robots as an example, there are several challenges in their use. First, users of surgical robots need to spend a lot of time learning how to operate the instruments and the details of the procedures. Second, the preoperative planning process for surgical robots involves multi-dimensional decision-making, which requires integrating heterogeneous information from multiple sources, such as the individual characteristics of the surgical patient, the surgical goals, and clinical guidelines. Currently, it mainly relies on the experience of the surgical robot operators for manual decision-making and planning, which results in low decision-making efficiency, insufficient standardization, and strong dependence on the experience of the operators. Summary of the Invention

[0004] This disclosure provides a medical question-and-answer method, device, storage medium, electronic device, and product to provide accurate answers to questions during the application of medical devices and to provide auxiliary information for the use of medical devices through a question-and-answer format.

[0005] According to one aspect of this disclosure, a medical question-and-answer method is provided, comprising: Obtain medical description questions and identify the question types corresponding to the medical description questions; In the case where the question type is non-conceptual, historical case information matching the medical description question is determined based on a pre-built case library; and knowledge information matching the medical description question is determined based on a pre-built knowledge graph and knowledge base. The question-and-answer model determines the response information corresponding to the medical description question based on the historical case information and the knowledge information.

[0006] According to another aspect of this disclosure, a medical question-and-answer device is provided, comprising: The problem type determination module is used to acquire medical description problems and identify the problem type corresponding to the medical description problems; The information retrieval module is used to determine historical case information matching the medical description question based on a pre-built case library when the question type is non-conceptual; and to determine knowledge information matching the medical description question based on a pre-built knowledge graph and knowledge base. The response information generation module is used to determine the response information corresponding to the medical description question based on the historical case information and the knowledge information through a question-and-answer model.

[0007] According to another aspect of this disclosure, an electronic device is provided, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the medical question-and-answer method according to any embodiment of this disclosure.

[0008] According to another aspect of this disclosure, a computer-readable storage medium is provided that stores computer instructions for causing a processor to execute and implement the medical question-and-answer method according to any embodiment of this disclosure.

[0009] According to another aspect of this disclosure, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the medical question-and-answer method described in any embodiment of this disclosure.

[0010] The technical solution of this disclosure uses historical case information matched from a case library and knowledge information matched from a knowledge base as input information for a question-answering model. This allows the model to analyze the historical case information and knowledge information to generate a response. It is understood that while knowledge retrieval through knowledge graphs and knowledge bases can provide comprehensive recall information through the relationships between entities in the knowledge graph, it also introduces interfering information, affecting the hit rate of the retrieved knowledge. By matching historical case information from the case library—case information that best matches the medical description question—and simultaneously inputting the matched historical case information and the retrieved knowledge information into the question-answering model, the historical case information is used as prior information, improving the hit rate of the knowledge information and thus enhancing the accuracy of the response.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of a medical question-and-answer method provided in an embodiment of this disclosure; Figure 2 This is a flowchart illustrating the construction process of a knowledge base according to an embodiment of this disclosure; Figure 3 This is a schematic diagram of the structure of a medical question-and-answer device provided in an embodiment of this disclosure; Figure 4 This is a schematic diagram of the structure of an electronic device provided in a disclosed embodiment. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present disclosure, the technical solutions of the present disclosure will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present disclosure.

[0015] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0016] Figure 1This is a flowchart of a medical question-and-answer method provided in an embodiment of this disclosure. This embodiment is applicable to situations where accurate response information is obtained through a question-and-answer mode during the use of medical devices such as surgical robots, providing auxiliary information for the use of the medical devices. This method can be executed by a medical question-and-answer device, which can be implemented in hardware and / or software. The medical question-and-answer device can be configured in electronic devices such as medical devices, mobile terminals, servers, and computers. The medical devices include, but are not limited to, mobile robots; the mobile terminals include, but are not limited to, mobile phones and tablets. Figure 1 As shown, the method includes: S110. Obtain the medical description question and identify the question type corresponding to the medical description question.

[0017] S120. If the problem type is non-conceptual, determine historical case information matching the medical description problem based on a pre-built case library; and determine knowledge information matching the medical description problem based on a pre-built knowledge graph and knowledge base.

[0018] S130. Using a question-and-answer model, determine the response information corresponding to the medical description question based on the historical case information and the knowledge information.

[0019] Medical description questions can be understood as descriptive information about problems in a medical setting. Optionally, medical description questions can be descriptive information about problems arising during the use of medical equipment. These medical description questions can be collected through the interactive pages of electronic devices or through voice acquisition devices such as microphones; there are no restrictions on the method of collection.

[0020] Optionally, taking surgical robots as an example of medical devices, medical description questions include, but are not limited to, preoperative planning questions, troubleshooting questions, operation instruction questions, and medical knowledge answering questions. It can be understood that medical devices may also include, but are not limited to, imaging scanning equipment and rehabilitation therapy equipment; different types of medical devices may correspond to different medical description questions. The specific content of the medical description questions is not limited here.

[0021] Medical description questions fall into two categories: non-conceptual and conceptual. Conceptual questions can be understood as explanations of abstract concepts such as medical terminology, disease mechanisms, physiological processes, and treatment principles. Examples of conceptual questions include "What is coronary heart disease? What are its main risk factors?" and "What is the principle behind immunotherapy for cancer?" Non-conceptual questions can be understood as practical questions, such as seeking personalized advice for specific medical scenarios. Examples of non-conceptual questions include "The perforation parameters for surgical subject A in a laparoscopic surgery scenario."

[0022] In this embodiment, different processing methods are employed for different types of medical description questions to obtain highly accurate response information. Specifically, the question type is identified by categorizing the medical description questions. Optionally, a keyword set corresponding to each question type can be pre-set. By matching the keywords in the medical description questions against the keyword sets corresponding to each question type, the question type corresponding to the medical description question is determined based on the keyword matching results. For example, the keyword set corresponding to a concept may include, but is not limited to, keywords such as "what," "definition," "explanation," "mechanism," "concept," "meaning," and "raw material."

[0023] Optionally, identifying the question type corresponding to the medical description question includes: performing semantic analysis on the medical description question using a question-answering model to obtain question semantic information, and determining the question type based on the question semantic information. The question-answering model can be a Large Language Model (LLM), capable of performing tasks such as semantic analysis and text generation. In this embodiment, semantic analysis of the medical description question is performed using a question-answering model to obtain question semantic information, and the question type is identified based on this question semantic information to determine the question type of the medical description question.

[0024] In some embodiments of this disclosure, when the question type is the concept class, knowledge information matching the medical description question is determined based on a pre-built knowledge graph and knowledge base; and response information corresponding to the medical description question is determined based on the knowledge information using a question-answering model.

[0025] The knowledge graph can be constructed from texts in the medical field, including but not limited to medical literature, reviews, clinical diagnosis and treatment texts, operating guidelines for medical equipment, and policy and regulatory texts. This knowledge graph can be constructed by identifying entities and their relationships within these medical texts. Optionally, different knowledge graphs can be set up for different medical scenarios. Accordingly, the text corresponding to each medical scenario is obtained, and a knowledge graph for that scenario is constructed. For example, different medical devices can correspond to different medical scenarios; for instance, surgical robots, imaging scanning devices, and rehabilitation therapy devices can each correspond to different medical scenarios, meaning separate knowledge graphs can be constructed for surgical robots, imaging scanning devices, and rehabilitation therapy devices. For example, taking a surgical robot as an example, the different types of surgeries it can perform can correspond to different medical scenarios, and knowledge graphs can be constructed for different surgical types. For example, medical scenarios include, but are not limited to, laparoscopic surgery. Accordingly, for each medical scenario, the text corresponding to that scenario is obtained to construct a knowledge graph for that scenario.

[0026] A knowledge base can be understood as a database storing knowledge information, which can be in the form of feature vectors. Optionally, the knowledge base may include knowledge information corresponding to multimodal data in the medical field; optionally, the knowledge base is matched with a knowledge graph, and the knowledge base and the knowledge graph may correspond to the same medical scenario.

[0027] Optionally, the knowledge graph includes relationships between entities; the knowledge information in the knowledge base satisfies feature alignment and corresponds to the entities in the knowledge graph. Specifically, the knowledge information in the knowledge base can be feature-aligned, with knowledge information having the same semantics being feature-aligned, and knowledge information with the same semantics corresponding to the same entity, where the entity can be a semantic tag of the knowledge information.

[0028] Determining knowledge information that matches the medical description problem based on a pre-built knowledge graph and knowledge base includes: identifying associated entities of the medical description problem in the knowledge graph; and identifying knowledge information that has a corresponding relationship with the associated entities from the knowledge base as the knowledge information that matches the medical description problem.

[0029] Here, an associated entity can be understood as an entity that is related to the medical description question. In some embodiments, determining the associated entity of the medical description question in the knowledge graph includes: obtaining search keywords in the medical description question; matching the search keywords in the knowledge graph to determine a first entity that meets the matching conditions; determining a second entity in the knowledge graph that is related to the first entity; and determining the first entity and the second entity as the associated entity.

[0030] Specifically, this can be achieved by segmenting the medical description question into multiple words, preprocessing these words to remove invalid words such as stop words, modal particles, and meaningless words, and obtaining search keywords for the medical description question. These search keywords are then matched against the knowledge graph. Entities in the knowledge graph that match the search keywords are identified as the first entities satisfying the matching criteria. Alternatively, the search keywords can be matched against entities in the knowledge graph to obtain matching values, and entities with matching values ​​greater than a matching threshold are identified as the first entities satisfying the matching criteria.

[0031] The knowledge graph includes the relationships between entities. Starting with the matched first entity, second entities that are related to the first entity are identified. These second entities can have a direct relationship with the first entity, or they can have an indirect relationship through other entities. For example, the first entity includes entity a, which has direct relationships with entities b and c, entity b is related to entity d, and entity d is related to entity e. Accordingly, entities b, c, d, and e are all considered as second entities. The related entities for the medical description problem include: entities a, b, c, d, and e.

[0032] There is a correspondence between the knowledge information in the knowledge base and the entities in the knowledge graph. Based on the related entities of the medical description problem, the knowledge information in the knowledge base that corresponds to the related entities is determined as the knowledge information that matches the medical description problem.

[0033] The knowledge information matching the medical description question is used as the input information of the question-answering model. The question-answering model analyzes the knowledge information matching the medical description question and generates response information as the response information to the medical description question. The response information to the medical description question is output, and the output method may include displaying the response information on the interactive page and / or outputting the corresponding voice of the response information to the medical description question through a speaker.

[0034] In some embodiments of this disclosure, when the question type is non-conceptual, to improve the accuracy of the response information for non-conceptual questions and avoid interference from knowledge information obtained from the knowledge base, a collaborative retrieval of the case base and the knowledge base is performed. Specifically, the medical description question is matched against the case base to determine historical case information matching the medical description question. Then, the medical description question is matched against the constructed knowledge graph and the knowledge base to obtain knowledge information matching the medical description question. The retrieval process for knowledge information matching the medical description question is the same as described above and will not be repeated here.

[0035] The pre-built case library can include historical case information in the medical field or a defined medical scenario, and the medical field or medical scenario corresponding to the case library, knowledge base, and knowledge graph is consistent.

[0036] Optionally, based on a pre-built case library, historical case information matching the medical description question is determined, including: performing similarity matching between the semantic information of the medical description question and the historical case information in the case library to obtain similarity data; and using the historical case information corresponding to the highest similarity data as the matching result. Specifically, the semantic information of the question is converted into a vector representation, and the historical case data in the case library is similarly vectorized; finally, matching is achieved by calculating the vector similarity data between the semantic information of the question and the historical cases.

[0037] Optionally, historical case information is stored in a case library in a structured manner. Accordingly, determining historical case information that matches the medical description question based on the pre-built case library includes: converting the medical description question into query conditions; matching the query conditions in the pre-built case library; and determining historical case information that matches the medical description question based on the matching data of each historical case in the case library with the query conditions.

[0038] The query conditions can be in the form of an SQL query statement, which includes the query conditions. A pre-defined conversion method for the SQL query statement corresponding to the text or vector is used; for example, this conversion method could be a pre-defined conversion code. This is achieved by converting the text-based medical description question into an SQL query statement, or by converting the semantic information of the question in vector form into an SQL query statement. The SQL query statement carrying the query conditions is then processed in a case database to obtain historical case information that matches the medical description question, based on the similarity criteria that satisfy the query conditions. This similarity criterion can be the maximum similarity data.

[0039] In some embodiments of this disclosure, multiple case libraries corresponding to different medical scenarios are pre-set, and each case library includes historical case information for that specific medical scenario. Accordingly, the medical scenario corresponding to the medical description question is identified; matching is performed in the case library corresponding to the medical scenario based on the query conditions. Specifically, the medical scenario corresponding to the medical description question can be identified through the question semantic information, which can be obtained by classifying the question semantic information into scenarios.

[0040] Based on the medical scenario corresponding to the medical description question, the system matches the case library corresponding to that medical scenario based on the query conditions, thereby improving the matching specificity and avoiding the waste of computing resources caused by invalid matching.

[0041] The technical solution of this embodiment uses historical case information matched from the case library and knowledge information matched from the knowledge base as input information for the question-answering model. This allows the question-answering model to analyze the historical case information and knowledge information to generate response information. It is understood that during the retrieval of knowledge information through knowledge graphs and knowledge bases, the relationships between entities in the knowledge graph can provide comprehensiveness of the recalled knowledge information, but there is also interfering information, affecting the hit rate of the recalled knowledge information. By matching historical case information from the case library—which represents the case information in the case library with the highest degree of matching to the medical description question—and simultaneously inputting the matched historical case information and the recalled knowledge information into the question-answering model, the historical case information is used as prior information to improve the hit rate of the knowledge information, thereby improving the accuracy of the response information.

[0042] In some embodiments of this disclosure, a process for constructing a knowledge base is provided; for example, see [link to relevant documentation]. Figure 2 , Figure 2 This is a flowchart illustrating the construction process of a knowledge base according to an embodiment of this disclosure. The construction process of the knowledge base specifically includes: S210. Acquire multimodal data; at least a portion of the multimodal data carries entity labels.

[0043] S220. Map the multimodal data to the semantic space to obtain the semantic features corresponding to the multimodal data.

[0044] S230. Based on the entity labels corresponding to the multimodal data, perform feature alignment on the semantic features corresponding to the multimodal data to obtain the correspondence between the semantic features corresponding to the multimodal data and the entity labels.

[0045] S240. The semantic features corresponding to the multimodal data are used as knowledge information to form the knowledge base.

[0046] The multimodal data includes at least one of the following: image data, text data, and time-series data. Image data may include historical image data from the medical field or a specific medical scenario. This image data can be images and videos, such as CT images, MRI images, and other medical scan images. Image data may also include process videos recorded by a surgical robot performing a surgery. Text data may include medical guidelines and historical case texts from the medical field or a specific medical scenario. Time-series data may include sensor data that can be collected in real time by medical devices through API interfaces, such as robotic arm pose and current data. Optionally, the multimodal data may be preprocessed, including but not limited to at least one of noise reduction, deduplication, and outlier processing.

[0047] At least a portion of the multimodal data carries entity labels, which can be set by experts to provide a basis for feature alignment during the construction of a knowledge base. These entity labels can be understood as labels used to represent at least one of the events, concepts, etc., corresponding to the data. Specifically, different parts of the same text can correspond to different entity labels, different video segments within the same video can correspond to different entity labels, and different time-series data segments from the same device can correspond to different entity labels. For example, entity labels for different video segments in a surgical procedure video recorded by a surgical robot could include, but are not limited to, "punching" and "suturing."

[0048] Optionally, multimodal data can be mapped to a semantic space using a neural network model to obtain the semantic features corresponding to the multimodal data. For example, the neural network model may include a cross-modal attention mechanism model. Alternatively, different neural network models can be set for different modalities of data, each used to perform semantic mapping on the data of different modalities. For example, the neural network model for text data may be a text embedding model, used to convert text into high-dimensional vectors; the neural network model for image data may be a U-net model, etc. The structure and type of the neural network model are not limited here, as long as it can achieve feature extraction and mapping of the data.

[0049] The semantic features obtained by mapping multimodal data are used as knowledge information to construct a knowledge base. To avoid the problem of long retrieval time due to the large amount and complexity of knowledge information in the knowledge base, feature alignment is performed on the semantic features corresponding to the multimodal data, and the knowledge base is constructed based on the feature-aligned semantic features.

[0050] Optionally, based on the entity labels corresponding to the multimodal data, feature alignment is performed on the semantic features corresponding to the multimodal data to obtain the correspondence between the semantic features corresponding to the multimodal data and the entity labels, including: for first data carrying the entity labels, establishing a correspondence between the semantic features corresponding to the first data and the entity labels; for second data not carrying the entity labels, the matching degree data between the semantic features of the second data and the semantic features of the first data is used, and the semantic features of the second data that satisfy the matching condition are established with the entity labels corresponding to the first data.

[0051] The multimodal data may include first data with the entity label and second data without the entity label. For the first data, the semantic features corresponding to the first data are mapped to the entity label. For the second data, the entity label corresponding to the second data is determined by the matching degree data between the semantic features of the second data and the semantic features of the first data. For example, in the matching degree data between the semantic features of the first data and multiple second semantic data, the semantic features of the second data that exceed the matching degree threshold are determined as the semantic features of the second data used as matching conditions. The second data with the matching conditions corresponds to the same entity label as the first data, and a mapping relationship is established between the semantic features of the second data that meet the matching conditions and the entity label corresponding to the first data.

[0052] By establishing a correspondence between the semantic features of at least one data point and entity labels, data alignment is achieved, facilitating the retrieval of multiple semantic features using entity labels. A knowledge base is then constructed by combining the semantic features and their correspondence with entity labels.

[0053] Based on the above embodiments, in order to improve the accuracy and timeliness of knowledge information in the knowledge base, the knowledge base is evolved and updated; wherein, the update method can be real-time update or periodic update.

[0054] In some embodiments of this disclosure, a PPO decision network is used to analyze the accuracy, novelty, and domain relevance of newly added knowledge to obtain decision information. For example, when the decision information is 1, it indicates that the newly added knowledge can be added to the update data package; when the decision information is 0, it indicates that the newly added knowledge will not be added to the update data package; and when the decision information is 2, it indicates that the newly added knowledge is in a pending state and will be submitted to the review end for review and further decision-making. The knowledge base is updated periodically based on the update data package.

[0055] In some embodiments of this disclosure, in order to ensure the accuracy of the knowledge information added to the knowledge base, the newly added knowledge is verified to obtain a validity value of the newly added knowledge, so as to filter out the valid knowledge that can be added to the knowledge base.

[0056] The evolutionary update method includes: acquiring new knowledge, adding the new knowledge to a candidate knowledge base, and updating the validity value of each new knowledge within a preset time period; determining valid knowledge based on the validity value of each new knowledge in the candidate knowledge base, and performing evolutionary updates on the knowledge base based on the valid knowledge, wherein the validity value of the new knowledge represents the availability of the new knowledge.

[0057] In this embodiment, a candidate knowledge base is pre-set to store newly added knowledge. This newly added knowledge may include, but is not limited to, at least one of the following: user feedback data after the response information is displayed to the user; operation logs continuously collected during the operation of the medical device; effect evaluation data after the medical device performs its functions; and expert correction instructions.

[0058] Newly added knowledge will be stored in the candidate knowledge base for at least a preset duration. During this storage process, a validity value will be determined for the new knowledge; a higher validity value indicates stronger usability. Valid knowledge for updating the knowledge base can be selected based on the validity value of the new knowledge. The initial validity value of new knowledge can be set to 0, and then updated accordingly. The validity value of new knowledge can be determined by applying multiple validation methods to obtain validation metrics, and rewards or penalties for the new knowledge will be determined based on these metrics to update its validity value.

[0059] Optionally, the method for updating the validity value of any newly added knowledge in the candidate knowledge base includes at least one of the following: obtaining the degree of deviation between the newly added knowledge and the knowledge already stored in the knowledge base, and updating the validity value of the newly added knowledge based on the degree of deviation; updating the validity value of the newly added knowledge based on the frequency of occurrence of the newly added knowledge within the preset time period; determining the change in the clustering coefficient of the entity tag corresponding to the newly added knowledge to the knowledge graph when the newly added knowledge is simulated and added to the knowledge base, and updating the validity value of the newly added knowledge based on the change in the clustering coefficient of the knowledge graph; obtaining the cross-modal combination corresponding to the newly added knowledge, and updating the validity value of the newly added knowledge based on the frequency of occurrence of the cross-modal combination when the newly added knowledge is adopted.

[0060] The degree of deviation between newly added knowledge and the knowledge already stored in the knowledge base can be determined using statistical test indicators. These indicators include, but are not limited to, the chi-square test, the KS test, and the KL divergence index. For example, the chi-square test can be used to determine the degree of deviation between newly added knowledge and the knowledge already stored in the knowledge base; a higher chi-square test value indicates a greater degree of deviation.

[0061] The updated value of the validity value of newly added knowledge is related to the magnitude of the deviation. Optionally, if the newly added knowledge violates the constraints of the existing knowledge base, the updated value of the validity value of the newly added knowledge is negative, representing a penalty for the validity value of the newly added knowledge. If the newly added knowledge does not violate the constraints of the existing knowledge base, the updated value of the validity value of the newly added knowledge is positive, representing a reward for the validity value of the newly added knowledge. The absolute values ​​of the reward and penalty are respectively related to the magnitude of the deviation. The greater the deviation, the lower the confidence level of the newly added value, and the smaller the absolute value of the reward and penalty.

[0062] Based on the known validity value of the newly added knowledge (e.g., the initial validity value or the validity value after the last refinement), add a reward value or subtract a penalty value to obtain the updated validity value.

[0063] For new knowledge that does not violate the constraints of the existing knowledge base, the higher the frequency of its occurrence, the higher its validity value. In other words, for each instance of new knowledge appearing in the candidate knowledge base, a reward value is set for that new knowledge, and its validity value is updated based on the reward data. The reward value for new knowledge decreases exponentially with its frequency of occurrence.

[0064] Adding new knowledge to the knowledge base can be understood as a hypothetical operation of adding new knowledge to the knowledge base. When new knowledge is added to the knowledge base, if the new knowledge corresponds to a new entity, the knowledge graph can be updated based on the new entity. The clustering coefficients before and after the knowledge graph update are then determined, yielding the rate of change of the clustering coefficients. The clustering coefficients of the knowledge graph can be understood as an important indicator measuring the degree of clustering between entities and their neighboring entities, used to characterize the local connectivity and density of the knowledge graph. The clustering coefficients of the knowledge graph can be determined by identifying the local clustering coefficients corresponding to each entity and then determining the global clustering coefficients of the knowledge graph based on these local clustering coefficients. The local clustering coefficients corresponding to each entity can be determined based on the ratio of the actual number of edges to the theoretical maximum number of edges for each entity. The global clustering coefficients of the knowledge graph can be the mean of the local clustering coefficients of all entities in the knowledge graph.

[0065] The reward value is determined based on the change in the clustering coefficient of the knowledge graph, and the validity value of the newly added knowledge is updated based on the reward value. The reward value may be positively correlated with the change in the clustering coefficient of the knowledge graph.

[0066] The cross-modal combination corresponding to the newly added knowledge can be understood as the combination of various data modalities in the newly added knowledge, such as the combination of impact data and pathological data. The frequency of occurrence of the cross-modal combination corresponding to the newly added knowledge in the knowledge base is determined. A reward value is set based on this frequency, and the validity value of the newly added knowledge is updated based on the reward value. This reward value is negatively correlated with the frequency of occurrence; that is, the lower the frequency of occurrence, the larger the reward value.

[0067] Each newly added knowledge in the candidate knowledge base is assigned a timestamp, which represents the time when the new knowledge entered the candidate knowledge base. If the storage time of the new knowledge in the candidate knowledge base meets the verification duration, it is determined whether the validity value of the new knowledge exceeds a set threshold. If yes, the knowledge base is updated based on the new knowledge; otherwise, the new knowledge is deleted.

[0068] The technical solution in this embodiment constructs a knowledge base based on semantic features obtained from multimodal data mapping, after feature alignment, to provide multidimensional data support for medical question-and-answer scenarios. Continuous updates to the knowledge base improve its accuracy and timeliness.

[0069] In some embodiments of this disclosure, the knowledge information in the knowledge base includes semantic features corresponding to multimodal data. In some embodiments of this disclosure, the knowledge information in the knowledge base includes multimodal data and semantic features corresponding to the content summaries of the multimodal data.

[0070] In some embodiments of this disclosure, the knowledge information in the knowledge base includes semantic features corresponding to multimodal data and semantic features corresponding to the content summary of multimodal data. The knowledge base construction process may include: obtaining the content summary of the multimodal data; mapping the content summary of the multimodal data to a semantic space to obtain the semantic features corresponding to the content summary of the multimodal data; and using at least one of the semantic features corresponding to the content summary of the multimodal data and the semantic features corresponding to the multimodal data as knowledge information to form the knowledge base.

[0071] The multimodal data can include, but is not limited to, video data, audio data, and text data. Each of the multimodal data types has a content summary, especially the video data and audio data. By setting content summaries, the content of the video data and audio data can be described in a multi-dimensional summary, which facilitates the retrieval of the video data and audio data.

[0072] The content summary of multimodal data includes labels for multiple semantic dimensions and the data range corresponding to each label. These semantic dimensions can include at least one of the following: surgical action dimension, medical scenario dimension, key events during the surgical procedure, medical entity dimension, and operational intent dimension. For video data, labels can be set from at least one of these dimensions to form a content summary; for audio data, labels can be set from at least one of the medical entity dimension and operational intent dimension to form a content summary.

[0073] The labeling of surgical actions can be understood as identifying the type of surgical action by detecting at least one of the surgical instrument movement trajectory and the surgeon's hand movements. For example, manual action types can include cutting and suturing, and different surgical action types correspond to different labels, which can be text labels. The same video data can include multiple types of surgical actions during the surgical process. Correspondingly, multiple labels can be set for the video data in the surgical action dimension. The multiple labels in the surgical action dimension can each correspond to different local video segments, i.e., data ranges. The data range corresponding to each label can be represented by start and end timestamps.

[0074] The labels for the medical scenario dimension can be understood as labels that define the medical scenario setting of a surgical procedure by identifying at least one of the anatomical structures of the surgical area and the spatial relationships of instrument interactions. For example, the anatomical structures of the surgical area could be the descending part of the duodenum operating area or the intercondylar fossa of the knee joint. Instrument interaction spatial relationships can be understood as the spatial positional relationships and movement coordination relationships between different instruments and between instruments and the surgical area, used to describe the placement and coordination of instruments in space. For example, the spatial relationship between instruments could be a needle holder and a suture needle in a clamping and fitting state, or the spatial relationship between instruments and the surgical area could be a laparoscope lens facing the gallbladder triangle operating area. The labels for the medical scenario dimension are determined based on the mapping relationship between the anatomical structures of the surgical area and / or the spatial relationships of instrument interactions, respectively, and the labels for the medical scenario dimension. For example, the labels for the medical scenario dimension could include gastrointestinal surgery, orthopedic joint surgery, etc.

[0075] The labels for key events during the surgical phase can be understood as labels assigned to key events during the surgical process. For example, key events may include the occurrence of bleeding points. Different types of events may have different key events, and different types of key events may correspond to different labels. Each key event can correspond to a local video segment, that is, the data range of that key event, which can be represented by the start and end timestamps of the local video segment.

[0076] The labels for medical entities can be understood as labels set for at least one of the entities related to lesions or medical devices. For video data, by identifying lesions, labels can be set for local video frames showing lesions; similarly, by identifying medical device types, labels can be set for local video frames showing the same device type. For audio data, by converting the audio to text, labels can be set for the lesion name (or abbreviation, code, etc.) and the medical device name (or abbreviation, code, etc.) appearing in the converted text. Similarly, the data range corresponding to each label can be represented by start and end timestamps.

[0077] The labels for the operational intent dimension can be understood as labels set for the intention of switching between surgical stages during the operation, or labels set according to the emergency status and surgical operation actions corresponding to the speech recognition in the audio and video data.

[0078] In some embodiments, obtaining a content summary of the multimodal data includes: performing semantic analysis on the multimodal data to identify tags for multiple semantic dimensions; and generating a content summary of the multimodal data based on the tags for multiple semantic dimensions. Specifically, semantic analysis can be performed on the multimodal data using a semantic analysis model, and tags for multiple semantic dimensions can be set for the multimodal data. The semantic analysis model here can be, for example, a neural network model or a large language model; the specific structure of the semantic analysis model is not limited.

[0079] A content summary is formed based on tags with multiple semantic dimensions and the data range corresponding to each tag. The content summary is then associated with multimodal data, and the content summary is converted into semantic features. At least one of the semantic features corresponding to the content summary of the multimodal data and the semantic features corresponding to the multimodal data is used as knowledge information to form the knowledge base.

[0080] It is understandable that during the construction of the knowledge base, before / after determining the content summary of the multimodal data, feature alignment of the semantic features corresponding to the multimodal data is also performed based on the entity tags corresponding to the multimodal data.

[0081] In some embodiments, determining the knowledge information matching the medical description question based on a pre-built knowledge graph and knowledge base includes: identifying the associated entity of the medical description question in the knowledge graph; determining the knowledge information corresponding to the associated entity from the knowledge base; and, if the knowledge information is a semantic feature corresponding to the content summary of multimodal data, using the multimodal data and its content summary as the knowledge information matching the medical description question. For example, if the semantic feature corresponding to the content summary of any type of original data (video data or audio data) in the multimodal data successfully matches the associated entity of the medical description question, using the original data (video data or audio data) and its content summary as the knowledge information improves the comprehensiveness of the knowledge information and provides more comprehensive dependency data for generating response information.

[0082] Here, when the original data in the multimodal data has a content summary, the matching between the semantic features corresponding to the content summary and the associated entities of the medical description problem can replace the matching between the original data and the associated entities of the medical description problem in the multimodal data. This simplifies the knowledge retrieval process, reduces the computational load of the knowledge retrieval process, and improves the efficiency of knowledge retrieval.

[0083] In some embodiments of this disclosure, upon receiving a response, the response, the historical case information upon which the response depends, the multimodal data corresponding to the knowledge information, and a content summary of the multimodal data are displayed. By displaying the multimodal data corresponding to the knowledge information and the content summary, it facilitates quick browsing of the summary's overview of the multimodal data's content. Furthermore, it allows for rapid and accurate location of the desired local data based on the data range corresponding to the tags in the content summary, avoiding interference from other data within the multimodal data.

[0084] In some embodiments of this disclosure, local multimodal data corresponding to the response information, the historical case information on which the response information depends, and the knowledge information are displayed. This local multimodal data can be local data corresponding to the data range of tags that successfully match the associated entities of the medical description question. It can be understood that the content summary includes tags with multiple semantic dimensions of the multimodal data, each tag corresponding to a data range. The associated entities of the medical description question can be those that successfully match one or more tags in the content summary of the multimodal data. Local data of the multimodal data is extracted based on the data range corresponding to the successfully matched tags and displayed to the user, facilitating targeted viewing and reducing data loading pressure.

[0085] In some embodiments, during the presentation of the response information, the response information, the historical case information on which the response information depends, and the multimodal data corresponding to the knowledge information are displayed. The loading priority of the local multimodal data corresponding to the knowledge information and other local multimodal data within the multimodal data differs. The local multimodal data corresponding to the knowledge information is loaded first to facilitate priority viewing. Then, other local multimodal data besides the local multimodal data corresponding to the knowledge information are loaded to facilitate overall viewing of the multimodal data.

[0086] The technical solution provided in this embodiment sets a content summary for multimodal data, provides a general description of the multimodal data, realizes a deep understanding of the multimodal data and a description of the multi-semantic dimensions, facilitates the retrieval of multimodal data, and provides guidance for viewing multimodal data.

[0087] Based on the above embodiments, a medical question-answering method for surgical robot application scenarios includes: obtaining a preoperative planning question (medical description question) about the surgical robot, which may include surgical-related information and problem description information of the surgical object, such as the type of surgery and basic information of the surgical object. The preoperative planning question is analyzed from multiple dimensions using a question-answering model, and the question is classified to obtain at least one sub-question and a question type for each sub-question. The question type may include conceptual and non-conceptual types. For conceptual sub-questions, knowledge information matching the sub-question can be determined based on a pre-built knowledge graph and knowledge base; the answer information corresponding to the sub-question is determined based on the matching knowledge information of the sub-question using the question-answering model. For non-conceptual sub-questions, historical case information matching the medical description question can be determined based on a pre-built case library; and the answer information corresponding to the sub-question is determined based on the historical case information and the knowledge information, using the question-answering model.

[0088] Optionally, the responses to preoperative planning questions may include at least one of the following: similar cases, instrument motion planning, force feedback thresholds, and safety monitoring strategies. These responses are visualized on the surgical robot's control panel, and multi-level confirmation mechanisms are implemented at key operational nodes to ensure that the recommended plan is always executed under human supervision. This information provides a reference for the surgical robot's operator, enabling them to quickly determine the robot's workflow and complete preoperative planning.

[0089] Optionally, the response information may also include the source of the response information, and the original data can be retrieved by referring to the source.

[0090] Figure 3 This is a schematic diagram of the structure of a medical question-and-answer device provided in an embodiment of this disclosure. Figure 3 As shown, the device includes: Problem type determination module 310 is used to acquire medical description problems and identify the problem type corresponding to the medical description problems; The information retrieval module 320 is used to determine historical case information matching the medical description question based on a pre-built case library when the question type is non-conceptual; and to determine knowledge information matching the medical description question based on a pre-built knowledge graph and knowledge base. The response information generation module 330 is used to determine the response information corresponding to the medical description question based on the historical case information and the knowledge information through a question-and-answer model.

[0091] The technical solution of this embodiment uses historical case information matched from the case library and knowledge information matched from the knowledge base as input information for the question-answering model. This allows the question-answering model to analyze the historical case information and knowledge information to generate response information. It is understood that during the retrieval of knowledge information through knowledge graphs and knowledge bases, the relationships between entities in the knowledge graph can provide comprehensiveness of the recalled knowledge information, but there is also interfering information, affecting the hit rate of the recalled knowledge information. By matching historical case information from the case library—which represents the case information in the case library with the highest degree of matching to the medical description question—and simultaneously inputting the matched historical case information and the recalled knowledge information into the question-answering model, the historical case information is used as prior information to improve the hit rate of the knowledge information, thereby improving the accuracy of the response information.

[0092] Based on the above embodiments, optionally, the question type determination module 310 is used to: perform semantic analysis on the medical description question through a question-answering model to obtain question semantic information, and determine the question type based on the question semantic information; the question type includes non-conceptual type and conceptual type.

[0093] Optionally, the information retrieval module 320 is further configured to convert the medical description question into query conditions, match the query conditions in the pre-built case library, and determine historical case information that matches the medical description question based on the matching data of each historical case in the case library with the query conditions.

[0094] Optionally, the information retrieval module 320 is also used to identify the medical scenario corresponding to the medical description question; and to match the case library corresponding to the medical scenario based on the query conditions.

[0095] Based on the above embodiments, optionally, the knowledge graph includes relationships between entities; the knowledge information in the knowledge base satisfies feature alignment and has a corresponding relationship with the entities in the knowledge graph; The information retrieval module 320 is further configured to: determine the associated entities of the medical description problem in the knowledge graph; and determine the knowledge information that corresponds to the associated entities from the knowledge base as the knowledge information that matches the medical description problem.

[0096] Optionally, the information retrieval module 320 is further configured to obtain search keywords in the medical description question, match the search keywords in the knowledge graph, determine a first entity that meets the matching conditions, determine a second entity in the knowledge graph that is associated with the first entity, and determine the first entity and the second entity as the associated entities.

[0097] Based on the above embodiments, optionally, the device further includes a knowledge base construction module for acquiring multimodal data; at least a portion of the multimodal data carries entity labels; mapping the multimodal data to a semantic space to obtain semantic features corresponding to the multimodal data; performing feature alignment on the semantic features corresponding to the multimodal data based on the entity labels corresponding to the multimodal data to obtain the correspondence between the semantic features corresponding to the multimodal data and the entity labels; and using the semantic features corresponding to the multimodal data as knowledge information to form the knowledge base.

[0098] Optionally, the knowledge base construction module is also used to establish a correspondence between the semantic features corresponding to the first data and the entity tags for the first data carrying the entity tags; For the second data that does not carry the entity label, the matching degree data between the semantic features of the second data and the semantic features of the first data is used to establish a correspondence between the semantic features of the second data that meet the matching conditions and the entity label corresponding to the first data.

[0099] Optionally, based on the above embodiments, the device may further include a knowledge base update module for evolving and updating the knowledge base; The evolutionary update method includes: acquiring new knowledge, adding the new knowledge to a candidate knowledge base, and updating the validity value of each new knowledge within a preset time period; determining valid knowledge based on the validity value of each new knowledge in the candidate knowledge base, and performing evolutionary updates on the knowledge base based on the valid knowledge, wherein the validity value of the new knowledge represents the availability of the new knowledge.

[0100] Optionally, the knowledge base update module performs at least one of the following: obtaining the degree of deviation between the newly added knowledge and the knowledge already stored in the knowledge base, and updating the validity value of the newly added knowledge based on the degree of deviation; The validity value of the new knowledge is updated based on the frequency of its occurrence within the preset time period. When the newly added knowledge is simulated and added to the knowledge base, the change in the clustering coefficient of the knowledge graph corresponding to the entity tag of the newly added knowledge is determined, and the validity value of the newly added knowledge is updated based on the change in the clustering coefficient of the knowledge graph. Obtain the cross-modal combination corresponding to the newly added knowledge. When the newly added knowledge is adopted, update the validity value of the newly added knowledge based on the frequency of occurrence of the cross-modal combination.

[0101] Based on the above embodiments, optionally, the information retrieval module 320 is further configured to: determine knowledge information matching the medical description question based on a pre-built knowledge graph and knowledge base when the question type is the concept class; The response information generation module 330 is also used to determine the response information corresponding to the medical description question based on the knowledge information through a question-and-answer model.

[0102] The medical question-and-answer device provided in this disclosure can execute the medical question-and-answer method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of executing the method.

[0103] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0104] like Figure 4As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0105] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0106] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as medical question-answering methods.

[0107] In some embodiments, the medical question-and-answer method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via read-only memory (ROM) 12 and / or communication unit 19. When the computer program is loaded into random access memory (RAM) 13 and executed by processor 11, one or more steps of the medical question-and-answer method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the medical question-and-answer method by any other suitable means (e.g., by means of firmware).

[0108] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0109] Computer programs used to implement the medical question-and-answer method of this disclosure can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0110] This disclosure also provides a computer-readable storage medium storing computer instructions for causing a processor to execute a medical question-answering method, the method comprising: The system acquires medical description questions and identifies the question type corresponding to the medical description questions. If the question type is non-conceptual, it determines historical case information that matches the medical description questions based on a pre-built case library. It also determines knowledge information that matches the medical description questions based on a pre-built knowledge graph and knowledge library. Finally, it uses a question-answering model to determine the response information corresponding to the medical description questions based on the historical case information and the knowledge information.

[0111] In the context of this disclosure, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0112] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0113] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0114] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0115] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the medical question-and-answer method according to any embodiment of this disclosure.

[0116] In implementing a computer program product, computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0117] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this disclosure can be achieved, and this is not limited herein.

[0118] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A medical question-and-answer method, characterized in that, include: Obtain medical description questions and identify the question types corresponding to the medical description questions; In the case where the question type is non-conceptual, historical case information matching the medical description question is determined based on a pre-built case library; and knowledge information matching the medical description question is determined based on a pre-built knowledge graph and knowledge base. The question-and-answer model determines the response information corresponding to the medical description question based on the historical case information and the knowledge information.

2. The medical question-and-answer method according to claim 1, characterized in that, The identification of the question type corresponding to the medical description question includes: The medical description question is semantically analyzed using a question-answering model to obtain question semantic information, and the question type is determined based on the question semantic information. The problem types include non-conceptual and conceptual.

3. The medical question-and-answer method according to claim 1, characterized in that, Based on a pre-built case library, historical case information matching the medical description question is determined, including: The medical description question is converted into query conditions, and matching is performed on the pre-built case library based on the query conditions; Based on the matching data of each historical case in the case library with the query conditions, historical case information that matches the medical description question is determined.

4. The medical question-and-answer method according to claim 3, characterized in that, The method further includes: Identify the medical scenario corresponding to the medical description problem; The query criteria are used to match cases in the medical scenario's corresponding case library.

5. The medical question-and-answer method according to claim 1, characterized in that, The knowledge graph includes relationships between entities; the knowledge information in the knowledge base satisfies feature alignment and corresponds to the entities in the knowledge graph. The process of determining knowledge information that matches the medical description problem based on a pre-built knowledge graph and knowledge base includes: Identify the relevant entities for the medical description problem in the knowledge graph; Knowledge information that corresponds to the associated entity is determined from the knowledge base and used as knowledge information to match the medical description question.

6. The medical question-and-answer method according to claim 5, characterized in that, Identifying the associated entities for the medical description problem in the knowledge graph includes: Obtain the search keywords from the medical description question, match the search keywords in the knowledge graph, and determine the first entity that meets the matching conditions; Identify a second entity in the knowledge graph that is associated with the first entity; The first entity and the second entity are identified as the associated entities.

7. The medical question-and-answer method according to claim 1 or 5, characterized in that, The methods for constructing the knowledge base include: Acquire multimodal data; at least a portion of the multimodal data carries entity labels; The multimodal data is mapped to a semantic space to obtain the semantic features corresponding to the multimodal data; Based on the entity labels corresponding to the multimodal data, feature alignment is performed on the semantic features corresponding to the multimodal data to obtain the correspondence between the semantic features corresponding to the multimodal data and the entity labels; The semantic features corresponding to the multimodal data are used as knowledge information to form the knowledge base.

8. The medical question-and-answer method according to claim 7, characterized in that, The step of aligning the semantic features of the multimodal data with the entity labels corresponding to the multimodal data to obtain the correspondence between the semantic features of the multimodal data and the entity labels includes: For the first data carrying the entity label, establish the correspondence between the semantic features corresponding to the first data and the entity label; For the second data that does not carry the entity label, the matching degree data between the semantic features of the second data and the semantic features of the first data is used to establish a correspondence between the semantic features of the second data that meet the matching conditions and the entity label corresponding to the first data.

9. The method according to claim 7, characterized in that, The method further includes: Obtain a content summary of the multimodal data; The content summary of the multimodal data is mapped to the semantic space to obtain the semantic features corresponding to the content summary of the multimodal data; The knowledge base is formed by using at least one of the semantic features corresponding to the content summary of the multimodal data and the semantic features corresponding to the multimodal data as knowledge information.

10. The method according to claim 9, characterized in that, Obtain a content summary of the multimodal data, including: Semantic analysis is performed on the multimodal data to identify labels for multiple semantic dimensions in the multimodal data; Based on the multimodal data, a content summary of the multimodal data is generated for tags of multiple semantic dimensions. The content summary includes tags of each semantic dimension and the data range corresponding to each tag. The multiple semantic dimensions include at least one of surgical action dimension, medical scenario dimension, key event dimension in surgical stage dimension, medical entity dimension, and operation intention dimension.

11. The method according to claim 9, characterized in that, The method further includes: The system displays the response information, the historical case information on which the response information depends, and the multimodal data corresponding to the knowledge information, as well as a content summary of the multimodal data.

12. The medical question-and-answer method according to claim 1, characterized in that, The method further includes: The knowledge base is evolved and updated; The evolution update method includes: acquiring new knowledge, adding the new knowledge to the candidate knowledge base, and updating the validity value of each piece of new knowledge within a preset time period; Valid knowledge is determined based on the validity value of each newly added knowledge in the candidate knowledge base, and the knowledge base is updated and evolved based on the valid knowledge, wherein the validity value of the newly added knowledge represents the availability of the newly added knowledge.

13. The medical question-and-answer method according to claim 12, characterized in that, The validity value update method for any newly added knowledge in the candidate knowledge base includes at least one of the following: Obtain the degree of deviation between the newly added knowledge and the knowledge already stored in the knowledge base, and update the validity value of the newly added knowledge based on the degree of deviation; The validity value of the new knowledge is updated based on the frequency of its occurrence within the preset time period. When the newly added knowledge is simulated and added to the knowledge base, the change in the clustering coefficient of the knowledge graph corresponding to the entity tag of the newly added knowledge is determined, and the validity value of the newly added knowledge is updated based on the change in the clustering coefficient of the knowledge graph. Obtain the cross-modal combination corresponding to the newly added knowledge. When the newly added knowledge is adopted, update the validity value of the newly added knowledge based on the frequency of occurrence of the cross-modal combination.

14. The medical question-and-answer method according to claim 2, characterized in that, The method further includes: When the problem type is the concept class, knowledge information matching the medical description problem is determined based on a pre-built knowledge graph and knowledge base. The question-and-answer model is used to determine the response information corresponding to the medical description question based on the knowledge information.

15. A medical question-and-answer device, characterized in that, include: The problem type determination module is used to acquire medical description problems and identify the problem type corresponding to the medical description problems; The information retrieval module is used to determine historical case information matching the medical description question based on a pre-built case library when the question type is non-conceptual; and to determine knowledge information matching the medical description question based on a pre-built knowledge graph and knowledge base. The response information generation module is used to determine the response information corresponding to the medical description question based on the historical case information and the knowledge information through a question-and-answer model.

16. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the medical question-and-answer method according to any one of claims 1-14.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the medical question-and-answer method according to any one of claims 1-14.

18. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the medical question-and-answer method according to any one of claims 1-14.

Images