Knowledge enhancement-based apnea knowledge question and answer model construction method and device

Abstract

The invention discloses an apnea knowledge question and answer model construction method and device based on knowledge enhancement, and the method comprises the steps: constructing a database containing apnea knowledge and a test set, and carrying out the apnea knowledge question and answer model on the basis of the database through employing a predetermined external knowledge decoding strategy, a deep and shallow knowledge decoding strategy and preset probability distribution, and constructing an apnea knowledge question-answer model, and testing the apnea knowledge question-answer model by using the test set. Therefore, according to the apnea knowledge question and answer model construction method and device, on the basis of the question and the constructed apnea knowledge database, the model decoding emphasizes external knowledge, emphasizes high-level knowledge and reduces an algorithm of shallow-layer knowledge, so that the accuracy of answering the apnea question by a large model is improved; the problem that in a specific scene of apnea knowledge questions and answers, a natural language processing method is poor in answer accuracy due to insufficient professionality is solved.

Description

Technical Field

[0001] This invention relates to the field of medical language processing methods, specifically to a method and apparatus for constructing a knowledge-based question-and-answer model for sleep apnea. Background Technology

[0002] Sleep apnea is a common sleep disorder that significantly impacts patients' health and quality of life. To raise public awareness of this condition and provide timely and accurate medical consultation services, this paper proposes a knowledge-enhanced sleep apnea knowledge-based question-answering model. Recently, large-scale models have demonstrated capabilities exceeding expectations on certain tasks. However, due to the specialized medical knowledge involved in the medical field, large-scale models may produce incorrect or unreasonable answers to sleep apnea-related questions, and these incorrect or unreasonable answers could potentially lead to serious medical accidents. Summary of the Invention

[0003] To address this issue, this invention provides a method and apparatus for constructing a knowledge-enhanced sleep apnea knowledge question-answering model, in order to solve the problem of poor accuracy in answering questions by natural language processing methods in the specific scenario of sleep apnea knowledge question-answering due to insufficient professionalism.

[0004] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions: This invention provides a method for constructing a knowledge-enhanced sleep apnea question-answering model, the method comprising: Build a database and test suite containing knowledge about sleep apnea; Based on the database, a sleep apnea knowledge question-answering model is constructed using a predetermined external knowledge decoding strategy, a deep and shallow knowledge decoding strategy, and a preset probability distribution. The sleep apnea knowledge question-answering model was tested using the test set.

[0005] In some embodiments, a database containing knowledge about sleep apnea is constructed, specifically including: Extract text data related to sleep apnea knowledge from medical literature, professional databases, and online health forums; In response to the question prompt, the corresponding answer phrase is extracted from the text data using a large language model; The question prompts and corresponding answer phrases are compiled and the database is constructed.

[0006] In some embodiments, the external knowledge decoding strategy is used to improve the model's attention to contextual knowledge by comparing the differences in probability distributions between the model's attention to tacit knowledge and external knowledge during the text generation process.

[0007] In some embodiments, the expression for the external knowledge decoding strategy is:

[0008] in, express The model represented by each transformer module This represents the retrieved external knowledge. This indicates the input information. Represents the input sequence , This represents the output information of the model. This indicates that it is controlled by retrieved external knowledge and input information. Each transformer module represents the logical value of the model. Indicates being controlled by input information Each transformer module represents the logical value of the model. This represents a hyperparameter.

[0009] In some embodiments, the deep and shallow knowledge decoding strategy specifically includes: By comparing the differences between high-level probability distributions and shallow probability distributions, we can strengthen high-level knowledge and downplay shallow knowledge. In some embodiments, the expression for the deep and shallow knowledge decoding strategy is:

[0010]

[0011] in, express The model represented by each transformer module express The model represented by each transformer module, and , This represents the retrieved external knowledge. This indicates the input information. Represents the input sequence , This represents the output information of the model. This indicates that it is controlled by retrieved external knowledge and input information. Each transformer module represents the logical value of the model. This indicates that it is controlled by retrieved external knowledge and input information. The logical values ​​of the model represented by each transformer module. This represents a hyperparameter.

[0012] In some embodiments, the probability distribution is expressed as:

[0013] in, This indicates the external knowledge decoding strategy. This represents the strategy for decoding deep and shallow knowledge.

[0014] The present invention also provides a knowledge-enhanced sleep apnea knowledge question-answering model construction device, the device comprising: Database building unit, used to build a database and test set containing knowledge about sleep apnea; The question-answering model building unit is used to build a sleep apnea knowledge question-answering model based on the database, using a predetermined external knowledge decoding strategy, a deep and shallow knowledge decoding strategy, and a preset probability distribution. The question-and-answer model testing unit is used to test the sleep apnea knowledge question-and-answer model using the test set.

[0015] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method described above.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0017] The present invention provides a knowledge-enhanced method and apparatus for constructing a sleep apnea knowledge question-answering model. This involves building a database and a test set containing sleep apnea knowledge, and then constructing a sleep apnea knowledge question-answering model based on the database using a pre-determined external knowledge decoding strategy, a deep and shallow knowledge decoding strategy, and a preset probability distribution. The test set is then used to test the sleep apnea knowledge question-answering model. Thus, this method and apparatus, based on the question and the constructed sleep apnea knowledge database, and employing an algorithm that emphasizes external knowledge, high-level knowledge, and de-emphasizes shallow knowledge, improves the accuracy of large-scale models in answering sleep apnea questions. This solves the problem of poor accuracy in specific scenarios like sleep apnea knowledge question-answering caused by insufficient specialization in natural language processing methods. Attached Figure Description

[0018] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0019] The structures, proportions, sizes, etc. illustrated in this specification are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed herein, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0020] Figure 1 A flowchart illustrating the method for constructing a knowledge-enhanced sleep apnea knowledge question-answering model provided by this invention; Figure 2 The structural block diagram of the knowledge-enhanced sleep apnea knowledge question-answering model construction device provided by the present invention; Figure 3 This is a structural block diagram of a computer device provided by the present invention. Detailed Implementation

[0021] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] To address the aforementioned technical problems, this invention provides a knowledge-enhanced sleep apnea knowledge question-answering model. On one hand, inspired by the difference in attention between implicit and contextual knowledge sources during text generation, this method aims to improve the model's attention to contextual knowledge by comparing the differences in their probability distributions. On the other hand, by emphasizing high-level knowledge and downplaying low-level knowledge, it aims to improve the model's attention to factual knowledge. Combining these two approaches, a knowledge-enhanced sleep apnea knowledge question-answering model is created, improving the model's fidelity to external authoritative sleep apnea knowledge bases, thereby providing the public with timely and accurate medical consultation services.

[0023] In one specific implementation, please refer to Figure 1The method for constructing a knowledge-enhanced sleep apnea knowledge question-answering model provided by this invention includes the following steps: S110: Build a database and test set containing knowledge about sleep apnea; S120: Based on the database, a sleep apnea knowledge question-answering model is constructed using a predetermined external knowledge decoding strategy, a deep and shallow knowledge decoding strategy, and a preset probability distribution; S130: Test the sleep apnea knowledge question-answering model using the test set.

[0024] Step S110 involves creating a question-and-answer test set for sleep apnea knowledge. Specifically, this involves obtaining question-and-answer knowledge pairs related to sleep apnea from sleep apnea experts and saving them as a JSON file.

[0025] Construct a database containing knowledge about sleep apnea, specifically including: Extract text data related to sleep apnea knowledge from medical literature, professional databases, and online health forums; In response to the question prompt, the corresponding answer phrase is extracted from the text data using a large language model; The question prompts and corresponding answer phrases are compiled and the database is constructed.

[0026] Specifically, the establishment of the sleep apnea knowledge database in step S120 is implemented according to the following steps: Extract textual data related to sleep apnea knowledge from medical literature, professional databases, and online health forums; Leveraging the powerful language understanding and text generation capabilities of the ChatGLM4 model, and based on text data related to the apnea paragraph, the system prompts "Please summarize the paragraph content based on the relevant text knowledge": Record the responses from the ChatGLM4 model and save them as a sleep apnea knowledge database.

[0027] In S120, the external knowledge decoding strategy is used to improve the model's attention to contextual knowledge by comparing the differences in probability distribution between the model's attention to implicit knowledge and external knowledge during the text generation process.

[0028] The expression for the external knowledge decoding strategy is as follows:

[0029] in, express The model represented by each transformer module This represents the retrieved external knowledge. This indicates the input information. Represents the input sequence , This represents the output information of the model. This indicates that it is controlled by retrieved external knowledge and input information. Each transformer module represents the logical value of the model. Indicates being controlled by input information Each transformer module represents the logical value of the model. This represents a hyperparameter.

[0030] In some embodiments, the deep and shallow knowledge decoding strategy specifically includes: By comparing the differences between high-level and shallow probability distributions, and strengthening high-level knowledge while downplaying shallow knowledge, the expression for the deep-shallow knowledge decoding strategy is as follows:

[0031]

[0032] in, express The model represented by each transformer module express The model represented by each transformer module, and , This represents the retrieved external knowledge. This indicates the input information. Represents the input sequence , This represents the output information of the model. This indicates that it is controlled by retrieved external knowledge and input information. Each transformer module represents the logical value of the model. This indicates that it is controlled by retrieved external knowledge and input information. The logical values ​​of the model represented by each transformer module. This represents a hyperparameter.

[0033] In step S120, the expression for the probability distribution is:

[0034] in, This indicates the external knowledge decoding strategy. This represents the strategy for decoding deep and shallow knowledge.

[0035] Specifically, the model building in step S120 is implemented according to the following steps: This approach emphasizes decoding strategies for external knowledge. Inspired by the differences in attention models pay to tacit and external knowledge during text generation, it aims to improve the model's focus on contextual knowledge by comparing the differences in their probability distributions. It is expressed as follows: (1) In the above formula, express The model is represented by several transformer modules, and the model has a total of A transformer module This represents the retrieved external knowledge. Indicates query, Represents a token sequence , Indicates the next token, This represents the logistic value of the autoregressive model of the query, which is controlled by the retrieved external knowledge. This represents the logistic value of an autoregressive model controlled by the query. To represent a hyperparameter, By comparing the differences in the model's attention to implicit and external knowledge during the text generation process, the aim is to improve the model's attention to external knowledge. The decoding strategy that emphasizes high-level knowledge and downplays shallow knowledge aims to compare the differences between high-level and shallow probability distributions. The dynamic selection of shallow knowledge can be represented as follows: (2) (3) In the above formula, This indicates Jensen-Shannon divergence. Before the table The model represented by each transformer module, and , Represents a set , express and The model is controlled by the difference between external knowledge and the probability distribution of the post-query autoregressive model. The index representing the largest difference in probability distribution; Decoding strategy that emphasizes high-level knowledge and downplays shallow knowledge It is expressed as follows: (4) (5) In the above formula, Let be a hyperparameter designed to select the set of token categories with high confidence, denoted as . , The aim is to compare the differences between high-level probability distributions and shallow probability distributions; Furthermore, the probability distribution of the knowledge-enhanced sleep apnea knowledge question-answering model. It is expressed as follows: (6) In the above specific embodiments, the knowledge-enhanced sleep apnea knowledge question-answering model construction method provided by the present invention constructs a database and a test set containing sleep apnea knowledge. Based on the database, a sleep apnea knowledge question-answering model is constructed using a pre-determined external knowledge decoding strategy, a deep and shallow knowledge decoding strategy, and a preset probability distribution. The test set is then used to test the sleep apnea knowledge question-answering model. Thus, this sleep apnea knowledge question-answering model construction method, based on the question and the constructed sleep apnea knowledge database, and with an algorithm that emphasizes external knowledge, high-level knowledge, and de-emphasizes shallow knowledge in model decoding, improves the accuracy of large models in answering sleep apnea questions. This solves the problem of poor answer accuracy in the specific scenario of sleep apnea knowledge question-answering caused by insufficient specialization in natural language processing methods.

[0036] In addition to the methods described above, this invention also provides a knowledge-enhanced sleep apnea knowledge question-answering model construction device, such as... Figure 2 As shown, the device includes: Database building unit 210 is used to build a database and test set containing knowledge of sleep apnea; The question-answering model construction unit 220 is used to construct a sleep apnea knowledge question-answering model based on the database, using a predetermined external knowledge decoding strategy, a deep and shallow knowledge decoding strategy, and a preset probability distribution. The question-and-answer model testing unit 230 is used to test the sleep apnea knowledge question-and-answer model using the test set.

[0037] In some embodiments, a database containing knowledge about sleep apnea is constructed, specifically including: Extract text data related to sleep apnea knowledge from medical literature, professional databases, and online health forums; In response to the question prompt, the corresponding answer phrase is extracted from the text data using a large language model; The question prompts and corresponding answer phrases are compiled and the database is constructed.

[0038] In some embodiments, the external knowledge decoding strategy is used to improve the model's attention to contextual knowledge by comparing the differences in probability distributions between the model's attention to tacit knowledge and external knowledge during the text generation process.

[0039] In some embodiments, the expression for the external knowledge decoding strategy is:

[0040] in, express The model represented by each transformer module This represents the retrieved external knowledge. This indicates the input information. Represents the input sequence , This represents the output information of the model. This indicates that it is controlled by retrieved external knowledge and input information. Each transformer module represents the logical value of the model. Indicates being controlled by input information Each transformer module represents the logical value of the model. This represents a hyperparameter.

[0041] In some embodiments, the deep and shallow knowledge decoding strategy specifically includes: By comparing the differences between high-level probability distributions and shallow probability distributions, we can strengthen high-level knowledge and downplay shallow knowledge. In some embodiments, the expression for the deep and shallow knowledge decoding strategy is:

[0042]

[0043] in, express The model represented by each transformer module express The model represented by each transformer module, and , This represents the retrieved external knowledge. This indicates the input information. Represents the input sequence , This represents the output information of the model. This indicates that it is controlled by retrieved external knowledge and input information. Each transformer module represents the logical value of the model. This indicates that it is controlled by retrieved external knowledge and input information. The logical values ​​of the model represented by each transformer module. This represents a hyperparameter.

[0044] In some embodiments, the probability distribution is expressed as:

[0045] in, This indicates the external knowledge decoding strategy. This represents the strategy for decoding deep and shallow knowledge.

[0046] In the above specific embodiments, the knowledge-enhanced sleep apnea knowledge question-answering model construction device provided by the present invention constructs a database and a test set containing sleep apnea knowledge. Based on the database, it constructs a sleep apnea knowledge question-answering model using a pre-determined external knowledge decoding strategy, a deep and shallow knowledge decoding strategy, and a preset probability distribution. The test set is then used to test the sleep apnea knowledge question-answering model. Thus, this sleep apnea knowledge question-answering model construction device, based on the question and the constructed sleep apnea knowledge database, and with an algorithm that emphasizes external knowledge, high-level knowledge, and de-emphasizes shallow knowledge in model decoding, improves the accuracy of large models in answering sleep apnea questions. This solves the problem of poor answer accuracy in the specific scenario of sleep apnea knowledge question-answering caused by insufficient specialization in natural language processing methods.

[0047] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and model predictions. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The model predictions of the computer device store static and dynamic information data. The network interface of the computer device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the above method embodiments.

[0048] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0049] Corresponding to the above embodiments, this invention also provides a computer storage medium containing one or more program instructions. These one or more program instructions are used to execute the method described above.

[0050] The present invention also provides a computer program product, the computer program product including a computer program, the computer program being stored on a non-transitory computer-readable storage medium, and the computer being able to perform the above-described method when the computer program is executed by a processor.

[0051] In this embodiment of the invention, the processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0052] The various methods, steps, and logic diagrams disclosed in the embodiments of this invention can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The processor reads information from the storage medium and, in conjunction with its hardware, completes the steps of the above methods.

[0053] The storage medium can be memory, such as volatile memory or non-volatile memory, or may include both volatile and non-volatile memory.

[0054] Among them, non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory.

[0055] Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (Synchlink DRAM, SLDRAM), and direct memory bus RAM (DRRAM).

[0056] The storage media described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memory.

[0057] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this invention can be implemented using a combination of hardware and software. When applied as software, the corresponding functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transmission of computer programs from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0058] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for constructing a knowledge-enhanced sleep apnea question-answering model, characterized in that, The method includes: Build a database and test suite containing knowledge about sleep apnea; Based on the database, a sleep apnea knowledge question-answering model is constructed using a predetermined external knowledge decoding strategy, a deep and shallow knowledge decoding strategy, and a preset probability distribution. The sleep apnea knowledge question-answering model was tested using the test set.

2. The method for constructing a knowledge-enhanced sleep apnea question-answering model according to claim 1, characterized in that, Construct a database containing knowledge about sleep apnea, specifically including: Extract text data related to sleep apnea knowledge from medical literature, professional databases, and online health forums; In response to the question prompt, the corresponding answer phrase is extracted from the text data using a large language model; The question prompts and corresponding answer phrases are compiled and the database is constructed.

3. The method for constructing a knowledge-enhanced sleep apnea question-answering model according to claim 1, characterized in that, The external knowledge decoding strategy is used to improve the model's attention to contextual knowledge by comparing the differences in probability distribution between the model's attention to implicit knowledge and external knowledge during the text generation process.

4. The method for constructing a knowledge-enhanced sleep apnea question-answering model according to claim 3, characterized in that, The expression for the external knowledge decoding strategy is: in, express The model represented by each transformer module This represents the retrieved external knowledge. This indicates the input information. Represents the input sequence , This represents the output information of the model. This indicates that it is controlled by retrieved external knowledge and input information. Each transformer module represents the logical value of the model. Indicates control over input information Each transformer module represents the logical value of the model. This represents a hyperparameter.

5. The method for constructing a knowledge-enhanced sleep apnea question-answering model according to claim 1, characterized in that, The deep and shallow knowledge decoding strategy specifically includes: By comparing the differences between high-level probability distributions and shallow probability distributions, we can strengthen high-level knowledge and downplay shallow knowledge.

6. The method for constructing a knowledge-enhanced sleep apnea question-answering model according to claim 5, characterized in that, The expression for the deep and shallow knowledge decoding strategy is: in, express The model represented by each transformer module express The model represented by each transformer module, and , This represents the retrieved external knowledge. This indicates the input information. Represents the input sequence , This represents the output information of the model. This indicates that it is controlled by retrieved external knowledge and input information. Each transformer module represents the logical value of the model. This indicates that it is controlled by retrieved external knowledge and input information. The logical values ​​of the model represented by each transformer module. This represents a hyperparameter.

7. The method for constructing a knowledge-enhanced sleep apnea question-answering model according to claim 1, characterized in that, The expression for the probability distribution is: in, This indicates the external knowledge decoding strategy. This represents the strategy for decoding deep and shallow knowledge.

8. A device for constructing a knowledge-enhanced sleep apnea question-and-answer model, characterized in that, The device includes: Database building unit, used to build a database and test set containing knowledge about sleep apnea; The question-answering model building unit is used to build a sleep apnea knowledge question-answering model based on the database, using a predetermined external knowledge decoding strategy, a deep and shallow knowledge decoding strategy, and a preset probability distribution. The question-and-answer model testing unit is used to test the sleep apnea knowledge question-and-answer model using the test set.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in any one of claims 1-7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-7.

Images