Rehabilitation prescription generation system and method based on multi-modal information and large language model

Abstract

The invention discloses a rehabilitation prescription generation system and method based on multi-modal information and a large language model. The system comprises a data acquisition module used for acquiring multi-modal data of a target individual, wherein the multi-modal data comprises subjective information and objective information; the feature extraction module is used for carrying out structured processing on the multi-modal data and converting the multi-modal data into a multi-modal feature vector to serve as a digital physiological portrait of a target individual; the retrieval module is used for comparing the multi-modal feature vector with a set rehabilitation rule knowledge base to obtain a corresponding rehabilitation rule, and then combining the multi-modal feature vector to organize a comprehensive cue word; and the big language model module is used for generating a personalized prescription recommendation scheme by utilizing a big language model based on the comprehensive cue word. According to the method, more personalized prescription recommendation can be dynamically generated, and the method is particularly suitable for people with urination and defecation dysfunction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, and more particularly to a rehabilitation prescription generation system and method based on multi-modal information and a large language model. BACKGROUND

[0002] Research on rehabilitation training prescription recommendation for two functional disorders is of great significance to improve the treatment effect of patients and enhance social cognition. Rehabilitation training for bedridden old people with two functional disorders is a top priority. A good targeted prescription can efficiently prompt the weak muscles of patients and accelerate the rehabilitation process of the elderly. Existing rehabilitation training prescription recommendations for two functional disorders are scarce and traditional, and lack of targeting, making it difficult to give prescriptions according to the actual situation of individual patients. In view of the above problems, the personalized prescription recommendation system for two functional disorders can effectively save the number of clinical nursing personnel, solve the most difficult "toilet nursing problem" in nursing work, assist the application and promotion of smart elderly care robots, and has a broad market prospect.

[0003] In the prior art, patent application CN2022115036547 proposes a pelvic floor function overall evaluation method based on multi-modal data fusion, which obtains pelvic floor function overall evaluation information through multi-module signal fusion of an optical imaging module, an ultrasonic module, a muscle strength and electromyography detection module. This method mainly evaluates the pelvic floor function through objective data and is only suitable for part of the two functional disorder evaluation.

[0004] Patent application CN2020109141956 proposes a urinary incontinence sling evaluation system and image evaluation method, which implants the invented sling into the human body and performs a magnetic resonance examination to check whether the structure of the sling and the positional relationship between the relaxed pelvic floor and the urethra are correct, thereby evaluating whether the structural properties such as extensibility of the sling are normal and maintaining the efficacy of the urinary incontinence sling to the greatest extent. This method has a certain invasiveness and is suitable for part of the post-treatment urinary incontinence disorder evaluation, but is not suitable for other two functional disorder evaluations.

[0005] Patent application CN2019101010588 provides a method and device for evaluating constipation risk and constipation degree based on the content of multiple intestinal bacteria, which utilizes the correlation between the imbalance degree of constipation-related bacteria and the severity of constipation, so that the content of constipation-related bacteria in the intestine can be used as a marker for constipation evaluation. By detecting the content of constipation-related bacteria in feces, the imbalance degree of constipation-related intestinal bacteria is evaluated, and then the degree of constipation is evaluated. This method is suitable for constipation disorder evaluation, but not for other two functional disorder evaluations.

[0006] Through analysis, the existing schemes mainly have the following defects: 1) Recommending prescriptions through the system's built-in case library or prescription template library: The system matches and recommends a fixed rehabilitation plan from the preset plan library based on the patient's disease type and severity level. This method cannot provide truly differentiated rehabilitation training prescriptions.

[0007] 2) Prescription recommendations are mainly based on the patient's subjective assessment results. This method ignores the physiological data that can objectively reflect the function of the pelvic floor muscles, which may lead to a mismatch between the prescription and the patient's actual muscle function status.

[0008] 3) Use a rule engine to generate personalized prescriptions. However, this method requires writing a massive number of rules, and the application of these rules is rather rigid and inflexible.

[0009] In summary, most bowel and bladder dysfunction currently requires prescriptions from clinicians. Given the large number of patients and the shortage of specialized physicians, it is difficult for doctors to tailor treatment plans to each patient's specific characteristics. Often, they offer more general rehabilitation programs. While existing treatment plans address bowel and bladder dysfunction, they lack personalization. Furthermore, some prescription recommendation methods only target the characteristics of a single symptom. Therefore, research on a method that integrates multimodal data and provides personalized prescription recommendations applicable to four types of bowel and bladder dysfunction is of great significance. Summary of the Invention

[0010] The purpose of this invention is to overcome the shortcomings of the prior art and provide a rehabilitation prescription generation system and method based on multimodal information and a large language model.

[0011] According to a first aspect of the present invention, a rehabilitation prescription generation system based on multimodal information and a large language model is provided. The system includes: Data acquisition module: used to acquire multimodal data of the target individual, the multimodal data including subjective information and objective information; Feature extraction module: performs structured processing on the multimodal data and converts it into multimodal feature vectors as a digital physiological profile of the target individual; Retrieval module: By comparing the multimodal feature vector with the set rehabilitation rule knowledge base, the corresponding rehabilitation rule is obtained, and then combined with the multimodal feature vector to organize it into comprehensive prompt words, wherein the rehabilitation rule knowledge base reflects the correspondence between the sample multimodal feature vector and the rehabilitation rules of historical case patterns; Large Language Model Module: Based on the comprehensive prompt words, a personalized prescription recommendation scheme is generated using a large language model.

[0012] According to a second aspect of the present application, a rehabilitation prescription generation method based on multi-modal information and a large language model is provided. The method comprises the following steps: Obtaining multi-modal data of a target individual, the multi-modal data comprising subjective information and objective information; Structurally processing the multi-modal data and converting it into a multi-modal feature vector as a digital physiological portrait of the target individual; By comparing the multi-modal feature vector with a set of rehabilitation rule knowledge base, the corresponding rehabilitation rule is obtained, and then combined with the multi-modal feature vector, it is organized into a comprehensive prompt word, wherein the rehabilitation rule knowledge base reflects the corresponding relationship between the sample multi-modal feature vector and the rehabilitation rule of the historical case mode; Based on the comprehensive prompt word, a large language model is used to generate a personalized prescription recommendation scheme.

[0013] Compared with the prior art, the present application has the advantages that, in view of the problems of rigidity, one-sidedness and lack of individualization in existing prescription recommendation technical schemes, the present application proposes a two-way rehabilitation prescription generation scheme based on multi-modal information and a large language model, which deeply integrates the subjective knowledge judgment of doctors and the objective physiological data collected by patients themselves, and combines the semantic understanding ability of an advanced large language model to search the rule base and generate a personalized prescription recommendation scheme for patients. The present application uses a general large language model and searches rules in combination with a vector knowledge base, which can deeply understand the rehabilitation significance of each physiological indicator and dynamically generate a complete rehabilitation prescription containing precise training parameters, thereby improving rehabilitation efficiency and effect and effectively solving the problem of shortage of professional nursing manpower.

[0014] Other features and advantages of the present application will become apparent from the following detailed description of exemplary embodiments thereof, taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0015] The accompanying drawings incorporated in and forming a part of the specification illustrate embodiments of the present application and, together with the description, serve to explain the principles of the application.

[0016] Figure 1 is a schematic diagram of subjective and objective information content according to an embodiment of the present application; Figure 2 is a general framework diagram of a two-way dysfunction personalized prescription recommendation system according to an embodiment of the present application. DETAILED DESCRIPTION

[0017] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. Note that the relative arrangement, numerical expressions, and numerical values of components and steps set forth in these embodiments are not limiting to the scope of the present application unless otherwise specifically stated.

[0018] The following description of at least one exemplary embodiment is merely exemplary in nature and is in no way intended to limit the scope of the application, its application, or uses.

[0019] Techniques, methods, and apparatus known to those of ordinary skill in the relevant art can not be discussed in detail herein, but should be considered as part of the specification.

[0020] In all of the examples shown and discussed herein, any specific values should be interpreted as merely exemplary, and not as a limitation. Thus, other examples of exemplary embodiments can have different values.

[0021] It should be noted that like reference numerals and letters in the various figures indicate similar items, and thus, once any component is defined in one figure, it should not require further discussion in subsequent figures.

[0022] The present application aims to provide highly personalized rehabilitation guidance for patients with defecation dysfunction, including fecal incontinence, urinary incontinence, constipation, and urinary retention. In general, the rehabilitation prescription generation system based on multi-modal information and large language model provided includes: a data acquisition module for acquiring multi-modal data of a target individual, the multi-modal data including subjective information and objective information; a feature extraction module for structuring the multi-modal data and converting it into a multi-modal feature vector as a digital physiological portrait of the target individual; a retrieval module for obtaining corresponding rehabilitation rules by comparing the multi-modal feature vector with a set of rehabilitation rule knowledge base, and then organizing comprehensive prompt words in combination with the multi-modal feature vector, wherein the rehabilitation rule knowledge base reflects the corresponding relationship between sample multi-modal feature vectors and historical case patterns of rehabilitation rules; a large language model module for generating personalized prescription recommendation schemes based on the comprehensive prompt words using a large language model. The present application integrates multi-modal features such as subjective information and objective physiological information of patients, and retrieves corresponding rules based on similarity to construct a comprehensive prompt word, and finally inputs it to a large language model to give a detailed personalized prescription containing specific rehabilitation actions and parameters.

[0023] Figure 1is a schematic diagram of collecting the subjective and objective information of the patient comprehensively. The subjective information includes the basic data of the patient's physical examination input through the system interface and the standardized assessment questionnaire filled in by the patient online. The objective information is mainly the surface electromyography signal of the patient's pelvic floor muscles in different states such as rest, contraction and relaxation, which is collected in real time through the electromyography sensor pasted on the corresponding part of the patient's body.

[0024] Figure 2 is the overall framework of the two-bathroom rehabilitation prescription generation system. In order to enable the computer to understand and process the collected raw information, the system will structure the multi-modal data. For example, for the subjective information, based on the questionnaire, the quantitative scoring is carried out, and the keyword extraction and labeling are carried out for the text information such as medical history and chief complaint. For the objective electromyography signal, first, the noise reduction processing is carried out through the Butterworth filter, and then a series of key physiological characteristics are extracted, including: the average electromyography value reflecting the muscle contraction force, the median frequency value reflecting the muscle fatigue resistance, the coefficient of variation reflecting the muscle contraction stability, and the rise and fall time reflecting the muscle excitability. Finally, the system integrates and converts all the processed subjective and objective characteristics into a high-dimensional multi-modal feature vector, which constitutes the unique digital physiological portrait of the patient. The subsequent is the rule matching based on similarity retrieval. The system internally pre-constructs a rehabilitation rule knowledge base, which stores a large number of rehabilitation rule fragments. When the multi-modal feature vector of the new patient is generated, the system uses the cosine similarity algorithm to search in the knowledge base, finds out the most similar multi-modal feature vector of the current patient, and extracts the corresponding rehabilitation rule. Then, the construction and input of the comprehensive prompt word are carried out. The system organizes the above extracted patient structured features and retrieved rehabilitation rules into a comprehensive prompt word with rich content and clear logic. The prompt word not only contains the specific situation description of the patient, but more importantly, it contains clear guiding principles. Finally, the personalized prescription is generated.

[0025] For example, in actual application, the system inputs the constructed comprehensive prompt word into the large language model. Under the guidance of the specific patient data and clear rehabilitation rule received by the large language model, the large language model can deeply understand the individualized needs of the patient and generate a complete personalized rehabilitation prescription. The content of the prescription is detailed, including the rehabilitation action name, specific execution group number, and accurate parameters such as the number of times for each group, which are customized for the patient, and finally presented to the user in a clear format.

[0026] Correspondingly, the application also provides a rehabilitation prescription generation method based on multi-modal information and a large language model. The method comprises: acquiring multi-modal data of a target individual, wherein the multi-modal data comprises subjective information and objective information; performing structured processing on the multi-modal data and converting the multi-modal data into a multi-modal feature vector as a digital physiological portrait of the target individual; comparing the multi-modal feature vector with a set rehabilitation rule knowledge base to obtain corresponding rehabilitation rules, and then combining the multi-modal feature vector to organize comprehensive prompt words, wherein the rehabilitation rule knowledge base reflects the corresponding relationship between sample multi-modal feature vectors and rehabilitation rules of historical case patterns; and generating an individualized prescription recommendation scheme by using a large language model based on the comprehensive prompt words.

[0027] It should be noted that the above-mentioned embodiments can be appropriately changed or modified by those skilled in the art without departing from the spirit and scope of the application. For example, the definition of subjective and objective information can be changed, the number of disorders of two functional disorders can be changed, the score of the four types of disorder questionnaire score table can be changed, or the pelvic floor muscle electromyographic signal can be collected by an invasive electrode, etc.

[0028] In summary, the application designs a set of individualized prescription generation system for bedridden elderly two functional disorders, which fuses multi-modal information such as subjective and objective information of patients and generates an individualized rehabilitation training prescription for patients with four two functional disorders of fecal incontinence, urinary incontinence, constipation and urinary retention by combining a large language model. The application can construct a two functional disorder intelligent evaluation model based on the chief complaint, physical examination basic data, evaluation scale and electromyographic data information of the two functional disorder patients, fuse the subjective and objective information of the new patients, modify the prescription of case reasoning by combining the rule base through the large language model, and then generate a more personalized prescription. The application fills the gap of personalized prescription recommendation based on multi-modal fusion, and has been verified to be able to dynamically generate a more personalized prescription recommendation, especially for two functional disorder patients.

[0029] The application can be a system, a method and / or a computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions loaded thereon, which are used to enable a processor to implement various aspects of the application.

[0030] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0031] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0032] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, Python, etc., and conventional procedural programming languages ​​such as "C" or similar languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0033] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0034] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0035] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0036] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.

[0037] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims

1. A rehabilitation prescription generation system based on multimodal information and a large language model, comprising: Data acquisition module: used to acquire multimodal data of the target individual, the multimodal data including subjective information and objective information; Feature extraction module: performs structured processing on the multimodal data and converts it into multimodal feature vectors as a digital physiological profile of the target individual; Retrieval module: By comparing the multimodal feature vector with the set rehabilitation rule knowledge base, the corresponding rehabilitation rule is obtained, and then combined with the multimodal feature vector to organize it into comprehensive prompt words, wherein the rehabilitation rule knowledge base reflects the correspondence between the sample multimodal feature vector and the rehabilitation rules of historical case patterns; Large Language Model Module: Based on the comprehensive prompt words, a personalized prescription recommendation scheme is generated using a large language model.

2. The system according to claim 1, characterized in that, The subjective information refers to the basic data of the target individual entered by the user and the standardized assessment questionnaire filled out online.

3. The system according to claim 2, characterized in that, The objective information refers to the electromyographic signals on the surface of the pelvic floor muscles of the target individual in different states of rest, contraction, and relaxation.

4. The system according to claim 3, characterized in that, The electromyographic signals on the surface of the pelvic floor muscles are collected using electromyographic sensors or electrodes.

5. The system according to claim 3, characterized in that, The structuring process for the multimodal data includes: For the subjective information, the standardized assessment questionnaire is quantified and scored, and keywords are extracted and labeled from the medical history and chief complaint text information to obtain subjective characteristics; The electromyographic signals on the surface of the pelvic floor muscles are denoised using a Butterworth filter, and physiological features are extracted as objective features. The subjective features and objective features are then integrated and transformed into a high-dimensional multimodal feature vector.

6. The system according to claim 5, characterized in that, The physiological characteristics include the mean electromyographic value reflecting muscle contractile force, the median frequency value reflecting muscle fatigue resistance, the coefficient of variation reflecting muscle contractile stability, and the rise and fall times reflecting muscle excitability.

7. The system according to claim 1, characterized in that, The retrieval module performs retrieval based on cosine similarity.

8. The system according to claim 1, characterized in that, The personalized prescription recommendation plan includes the name of the rehabilitation exercise, the number of sets to be performed, and the number of repetitions per set.

9. A method for generating rehabilitation prescriptions based on multimodal information and a large language model, comprising the following steps: Acquire multimodal data of the target individual, wherein the multimodal data includes subjective information and objective information; The multimodal data is structured and transformed into multimodal feature vectors, which serve as a digital physiological profile of the target individual. By comparing the multimodal feature vectors with a set rehabilitation rule knowledge base, corresponding rehabilitation rules are obtained, and then combined with the multimodal feature vectors to organize them into comprehensive prompt words. The rehabilitation rule knowledge base reflects the correspondence between the sample multimodal feature vectors and the rehabilitation rules of historical case patterns. Based on the comprehensive prompts, a personalized prescription recommendation scheme is generated using a large language model.

10. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by the processor, it implements the steps of the method according to claim 9.

Images