Medical and nursing combined nurse teaching system and method based on virtual reality

Abstract

The invention discloses a medical and nursing combined nurse teaching method based on virtual reality. The method comprises the following steps: constructing a medical scene and an analysis template; associating the scene with the analysis module; acquiring a scene selected by a student, starting virtual reality simulation, generating a corresponding virtual scene, and acquiring voice information and operation information sent by a user; the voice information is converted into character information, at least one score group is formed by using the character information, the operation information and characters of conversation of patients in the medical scene, and each score group further comprises score group generation time; and combining each score group with the analysis template to obtain the score of each score group, and integrating the scores of all the score groups to obtain the score of the student test. According to the invention, professional operation training can be carried out on a plurality of nurse students at the same time, instant feedback can be provided during training to help the students to rapidly improve, the training cost is reduced, and the repeatability and expandability of training are enhanced.

Description

Technical Field

[0001] This invention relates to the field of medical education technology, and in particular to a virtual reality-based integrated medical and nursing nursing teaching system and method. Background Technology

[0002] Currently, most existing nurse mannequin training models are primarily offline, which has certain limitations. In this traditional offline training method, trainees are often only able to learn mannequin operation in a specific classroom environment, strictly following the on-site guidance of an instructor. Class time is limited, and instructors need to attend to many trainees during the training process, making it impossible to fully answer every trainee's questions. After the training course, many trainees hope to study independently or form groups with other students to further consolidate and improve their skills. However, once they are separated from the real-time guidance of an instructor, they find it difficult to overcome difficulties on their own or with their classmates when they encounter operational errors or do not understand theoretical knowledge. Because operating a nurse mannequin involves professional medical knowledge and delicate operational skills, without professional guidance from an instructor, trainees may repeatedly try in the wrong direction, wasting a lot of time and energy, ultimately leading to slow learning progress and seriously affecting teaching efficiency. Summary of the Invention

[0003] To address the aforementioned shortcomings, the present invention aims to propose a virtual reality-based integrated medical and elderly care nursing teaching system and method, which solves the problems of low efficiency in existing nursing teaching and the inability to train multiple nursing students simultaneously.

[0004] To achieve this objective, the present invention adopts the following technical solution: a virtual reality-based nursing teaching method for integrated medical and elderly care, comprising the following steps: Step S1: Construct the medical scenario and analysis template; associate the scenario and analysis module; Step S2: Obtain the scene selected by the student, start the virtual reality simulation, generate the corresponding virtual scene, and obtain the voice information and operation information issued by the user; Step S3: Convert the voice information into text information, and use the text information, operation information and the text of the patient's dialogue in the medical scenario to form at least one scoring group, wherein each scoring group also includes the time when the scoring group was generated. Step S4: Combine each scoring group with the analysis template to obtain the score for each scoring group. Combine the scores of all scoring groups to obtain the score of the student in this test.

[0005] Preferably, the medical scenario includes patient pathology, patient dialogue, and patient actions, wherein the patient pathology has a unique identifier, and the patient dialogue and patient actions are bound to the patient pathology according to the unique identifier. Patient dialogues and actions are assigned priority levels, with lower-priority patient dialogues or actions only appearing after higher-priority patient dialogues or actions.

[0006] Preferably, in step S4, each scoring group is combined with the analysis template to obtain the score for each scoring group, including dialogue score, action score, and time ratio. Based on the patient's pathology, patient dialogue, and patient actions, the corresponding analysis template is selected; The steps for determining the motion frequency score are as follows: Obtain the operation code clicked by the student, and determine whether the operation code is the same as the code in the analysis template. If they are the same, obtain the action score; if they are different, do not obtain the action score. The steps for determining the dialogue score are as follows: The text information in the scoring group is segmented to obtain multiple sentence segments; The individual sentence segments and the standard sentences in the analysis template are input into the trained semantic model to determine whether the semantics of the sentence segments and the standard sentences are the same or similar. If they are the same or similar, a dialogue score is obtained. If they are different, the next sentence segment is replaced and the semantic judgment is re-executed until all sentence segments have been input into the semantic model. If so, the dialogue is judged to receive no score. The rules for using time proportions are as follows: Obtain the percentage of the time generated by the scoring group compared to the standard time, the generation time ratio, and adjust the sum of the dialogue score and the action score based on the time ratio.

[0007] Preferably, the following steps need to be performed before step S3: Step A: Construct a Standard Mandarin text database and at least one dialect text database; establish the mapping relationship between the Standard Mandarin text database and each dialect text database; Step B: Construct a standard speech database and at least one dialect speech database, establish a text mapping relationship between the standard speech database and the standard Mandarin text database, establish a mapping relationship between the dialect text database and the dialect speech database, and set the recognition priority of the dialect speech database.

[0008] Preferably, the step of converting speech information into text information in step S3 is as follows: Step S31: Obtain the first voice information input by the student as the first information, and convert the first information into Chinese Pinyin through the Mandarin voice database; Step S32: Based on the removal of tone information from the Chinese Pinyin syllables, input them into the Pinyin to Chinese character engine to generate one or more possible candidate Chinese character sequences for the input Chinese Pinyin syllables; Step S33: Input the candidate Chinese character sequence into the Chinese character to Pinyin library to obtain its standardized Pinyin representation with numerical tones, as a candidate Chinese Pinyin sequence. Select the Chinese character ranked first in the Chinese Pinyin sequence as the translation based on the removed tone information, and use the translation as the preset text. Step S34: Calculate the preset dialogue score using the preset text. If the preset dialogue score is greater than the score threshold, call the Mandarin speech library and Mandarin text library as the language conversion tools used in this test to convert the subsequent speech information into text information. If the preset dialogue score is less than the score threshold, then proceed to step S35; Step S35: Select the dialect speech library according to priority order, and convert the first information into dialect pinyin through the dialect speech library; Based on the above, after removing the tone information from the dialect pinyin syllables, the input is fed into the pinyin to Chinese character engine to generate one or more possible candidate dialect character sequences for the input dialect pinyin syllables; Step S36: Input the candidate dialect text sequence into the corresponding dialect-to-pinyin library to obtain its standardized pinyin representation with numerical tones, as the candidate dialect pinyin sequence. Select the dialect with the highest ranking in the dialect pinyin sequence as the translation based on the removed tone information. Use the translation as the dialect text. Based on the established mapping relationship between the dialect text library and the dialect speech library, generate the dialect text as the converted text. Use the converted text to calculate the converted dialogue score. If the converted dialogue score is greater than the score threshold, call the corresponding dialect speech library and dialect text library as the language conversion tool used in this test, and execute step S37. If the converted dialogue score is less than the score threshold, then repeat step S35. Step S37: Convert the subsequent speech information into dialect text. Through the mapping relationship between the Mandarin text library and various dialect text libraries, convert the dialect text into Mandarin text as text information.

[0009] Preferably, if a student's test score is lower than the score threshold, the voice message is sent to the corresponding communication address for manual verification.

[0010] A virtual reality-based nursing education system for integrated medical and elderly care uses a virtual reality-based teaching method for integrated medical and elderly care, including a construction module, a testing module, a transformation module, and a scoring module. The building module is used to construct medical scenarios and analysis templates; and to link the scenarios and analysis modules. The testing module is used to obtain the scene selected by the trainee, start the virtual reality simulation, generate the corresponding virtual scene, and obtain the voice information and operation information issued by the user. The conversion module is used to convert voice information into text information. The text information, operation information and patient dialogue in medical scenarios are used to form at least one scoring group, and each scoring group also includes the time when the scoring group was generated. The scoring module combines each scoring group with the analysis template to obtain the score for each scoring group. By combining the scores of all scoring groups, the total score for the student's test is obtained.

[0011] Preferably, the conversion module includes a first submodule, a second submodule, a third submodule, a fourth submodule, a fifth submodule, a sixth submodule, and a seventh submodule; The first submodule is used to obtain the first voice information input by the student, and convert the first information into Chinese Pinyin through the ordinary voice library; The second submodule is used to generate one or more possible candidate Chinese character sequences for the input Chinese Pinyin syllables after removing the tone information from the Pinyin syllables. The third submodule is used to input the candidate Chinese character sequence into the Chinese character to Pinyin library to obtain its standardized Pinyin representation with numerical tones, which is used as the candidate Chinese Pinyin sequence. Based on the removed tone information, the Chinese character ranked first in the Chinese Pinyin sequence is selected as the translation, and the translation is used as the preset text. The fourth submodule is used to calculate the preset dialogue score using preset text. If the preset dialogue score is greater than the score threshold, the ordinary speech library and the Mandarin text library are called as the language conversion tools used in this test to convert the subsequent speech information into text information. If the preset score is less than the score threshold, then the fifth submodule is invoked; The fifth submodule is used to select dialect speech libraries according to priority order, and the first information is converted into dialect pinyin through the dialect speech library; Based on the above, after removing the tone information from the dialect pinyin syllables, the input is fed into the pinyin to Chinese character engine to generate one or more possible candidate dialect character sequences for the input dialect pinyin syllables; The sixth submodule is used to input the candidate dialect text sequence into the corresponding dialect-to-pinyin library to obtain its standardized pinyin representation with numerical tones as the candidate dialect pinyin sequence. Based on the removed tone information, the dialect with the highest ranking in the dialect pinyin sequence is selected as the translation. The translation is used as the dialect text. Based on the established mapping relationship between the dialect text library and the dialect speech library, the dialect text is generated as the converted text. The converted text is used to calculate the converted dialogue score. If the converted dialogue score is greater than the score threshold, the corresponding dialect speech library and dialect text library are called as the language conversion tools used in this test, and the seventh submodule is called. If the conversion score is less than the score threshold, the fifth submodule will be called again. The seventh submodule is used to convert subsequent speech information into dialect text. Through the mapping relationship between the Mandarin text library and various dialect text libraries, the dialect text is converted into Mandarin text as text information.

[0012] Preferably, the scoring module includes an action scoring submodule, a dialogue scoring submodule, and a time adjustment submodule; The action scoring submodule is used to obtain the operation code clicked by the trainee, and determine whether the operation code is the same as the code in the analysis template. If they are the same, the action score is obtained; if they are different, the action score is not obtained. The dialogue scoring submodule is used to segment the text information in the scoring group and obtain multiple sentence segments; The individual sentence segments and the standard sentences in the analysis template are input into the trained semantic model to determine whether the semantics of the sentence segments and the standard sentences are the same or similar. If they are the same or similar, a dialogue score is obtained. If they are different, the next sentence segment is replaced and the semantic judgment is re-executed until all sentence segments have been input into the semantic model. If so, the dialogue is judged to receive no score. The time adjustment submodule is used to obtain the percentage of the time generated by the scoring group compared to the standard time, generate a time ratio, and adjust the sum of the dialogue score and the action score based on the time ratio.

[0013] Preferably, it also includes a manual processing module, which is used to send voice information to the corresponding communication address for manual verification when the student's test score is lower than the score threshold.

[0014] One of the above technical solutions has the following advantages or beneficial effects: it can provide professional operational training to multiple nurse trainees at the same time, provide immediate feedback during training to help trainees improve quickly, reduce training costs, and enhance the repeatability and scalability of training. Attached Figure Description

[0015] Figure 1 This is a flowchart of one embodiment of the method of the present invention.

[0016] Figure 2 This is a schematic diagram of the structure of one embodiment of the system of the present invention. Detailed Implementation

[0017] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0018] In the description of embodiments of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0019] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0020] like Figures 1-2 As shown, a virtual reality-based nursing education method for integrated medical and elderly care includes the following steps: Step S1: Construct the medical scenario and analysis template; associate the scenario and analysis module; Step S2: Obtain the scene selected by the student, start the virtual reality simulation, generate the corresponding virtual scene, and obtain the voice information and operation information issued by the user; Step S3: Convert the voice information into text information, and use the text information, operation information and the text of the patient's dialogue in the medical scenario to form at least one scoring group, wherein each scoring group also includes the time when the scoring group was generated. Step S4: Combine each scoring group with the analysis template to obtain the score for each scoring group. Combine the scores of all scoring groups to obtain the score of the student in this test.

[0021] The method of this invention can be used in conjunction with a game interface. When testing nurses' teaching and training, a scene can be displayed on the computer interface. After the trainee clicks on the corresponding scene, a virtual reality simulation will be performed. Using existing rendering technology, the corresponding patient, patient pathology, and patient actions will be rendered. A microphone is connected to the computer to collect the trainee's voice information. Corresponding action options will also appear on the computer, and the trainee can select the action option based on the patient's status displayed on the computer. For example, in a scenario where the patient needs to give an injection, the patient will engage in some dialogue or a corresponding operation option will pop up, such as applying iodine, applying alcohol, or giving an injection. At this time, the background will collect the options clicked by the trainee as operation information, while the voice information will be converted into text information. Then, the text information, operation information, and the text of the patient's dialogue in the medical scenario will be used to form at least one scoring group. Finally, each scoring group is compared with an analysis template, and the score is calculated by an algorithm (such as natural language processing or rule matching). The scores of all groups are combined to obtain the final test score. The entire process is completed immediately after the simulation ends, realizing the testing of nurse teaching and training.

[0022] The method proposed in this invention can provide professional operational training to multiple nursing trainees simultaneously, offering immediate feedback during training to help trainees improve quickly, while reducing training costs and enhancing the repeatability and scalability of the training.

[0023] Preferably, the medical scenario includes patient pathology, patient dialogue, and patient actions, wherein the patient pathology has a unique identifier, and the patient dialogue and patient actions are bound to the patient pathology according to the unique identifier. Patient dialogues and actions are assigned priority levels, with lower-priority patient dialogues or actions only appearing after higher-priority patient dialogues or actions.

[0024] In this invention, a unique identifier is created for each patient's pathology. For example, the unique identifier for "acute myocardial infarction" is 001 or the unique identifier for "tension pneumothorax" is 002. All patient dialogues (such as the patient's complaint "I have chest pain") and patient actions (such as experiencing difficulty breathing or signs of shock) are bound to the specific pathology through this unique identifier, ensuring a high degree of consistency between symptoms, signs, and causes, and avoiding confusion in the scene logic.

[0025] Furthermore, priority is added to patient actions and dialogue. For example, "suddenly collapsing and losing consciousness" has a low priority, while "patting one's chest" has a high priority. Only after "patting one's chest" occurs is "suddenly collapsing and losing consciousness" likely to occur. The scenario of "patting one's chest" occurring after "suddenly collapsing and losing consciousness" will not happen. This invention strictly follows the priority rules for triggering, ensuring that the randomly generated virtual reality simulation conforms to the dynamic performance of medical common sense.

[0026] Preferably, in step S4, each scoring group is combined with the analysis template to obtain the score for each scoring group, including dialogue score, action score, and time ratio. Based on the patient's pathology, patient dialogue, and patient actions, the corresponding analysis template is selected; The steps for determining the motion frequency score are as follows: Obtain the operation code clicked by the student, and determine whether the operation code is the same as the code in the analysis template. If they are the same, obtain the action score; if they are different, do not obtain the action score. The steps for determining the dialogue score are as follows: The text information in the scoring group is segmented to obtain multiple sentence segments; The individual sentence segments and the standard sentences in the analysis template are input into the trained semantic model to determine whether the semantics of the sentence segments and the standard sentences are the same or similar. If they are the same or similar, a dialogue score is obtained. If they are different, the next sentence segment is replaced and the semantic judgment is re-executed until all sentence segments have been input into the semantic model. If so, the dialogue is judged to receive no score. The rules for using time proportions are as follows: Obtain the percentage of the time generated by the scoring group compared to the standard time, the generation time ratio, and adjust the sum of the dialogue score and the action score based on the time ratio.

[0027] The following example illustrates this: First, in the action scoring judgment, the system obtains the operation code clicked by the trainee and compares it with the code in the analysis template. For example, for a patient with "anaphylactic shock," the first step should be to check the airway. If the trainee clicks the option "inject adrenaline" on the interface, the operation code for injecting adrenaline is different from the code on the analysis template. Therefore, the action score for this scoring group is 0 points. If the trainee clicks the option "check the airway" on the interface, the action score for this scoring group is 10 points.

[0028] According to the pathology described above, when anaphylactic shock occurs, the nurse should first check the breathing. If, during this process, the trainee says into the microphone, "Check his breathing, he looks like he's having difficulty breathing," the statement will be segmented, obtaining statement segment A ("Check his breathing") and statement segment B ("He looks like he's having difficulty breathing"). Sentence segment A is then input into a trained semantic model (such as BERT or a similar model) and compared with the standard statement in the analysis template. Sentence segment A is semantically similar to the standard statement "Check his breathing," and a dialogue score of 10 is obtained. At this point, it is not necessary to check statement segment B.

[0029] Finally, the percentage of the scoring group's generation time of 10 seconds to the standard time of 8 seconds is obtained, and the time ratio is calculated as 8 / 10=0.8. Then, the sum of the dialogue score and the action score (10+10=20 points) is adjusted to obtain the final score of 20×0.8=16 points.

[0030] The advantage of this scoring rule design lies in its comprehensive improvement of the accuracy, fairness, and effectiveness of medical training assessment through a multi-dimensional, adaptive, and objective evaluation mechanism. Firstly, the scoring rules combine dialogue scoring, action scoring, and time proportioning to ensure that trainees' clinical competence is assessed from multiple key perspectives: Action scoring verifies whether trainees performed the correct basic operations by directly comparing operational codes with analysis templates, ensuring the standardization and reliability of skill operations; Dialogue scoring utilizes a trained semantic model to segment trainees' statements and compare their semantic similarity with standard statements, rather than relying on strict keyword matching, thus more flexibly accommodating the diversity of language expression, reducing misjudgments caused by wording differences, and more realistically reflecting trainees' communication and clinical decision-making abilities; Time proportioning introduces an efficiency dimension, adjusting scores by calculating the percentage of time generated by the scoring group compared to the standard time, emphasizing the importance of time management in medical emergency scenarios, and encouraging trainees not only to do things correctly but also to do them promptly.

[0031] Preferably, the following steps need to be performed before step S3: Step A: Construct a Standard Mandarin text database and at least one dialect text database; establish the mapping relationship between the Standard Mandarin text database and each dialect text database; Step B: Construct a standard speech database and at least one dialect speech database, establish a text mapping relationship between the standard speech database and the standard Mandarin text database, establish a mapping relationship between the dialect text database and the dialect speech database, and set the recognition priority of the dialect speech database.

[0032] Since the usage scenario of the present invention is in Guangdong, a large number of nurses use Cantonese to communicate with patients. To ensure that the voice information can be correctly converted into the corresponding text information in step S3, in the present invention, a Mandarin text library and at least one dialect text library (Cantonese dialect text library) will be constructed first. There is a corresponding mapping relationship between the two text databases. For example, "睇" in the Cantonese dialect text library corresponds to "看" in the Mandarin text library, and "係咪" in the Cantonese dialect text library corresponds to "是不是" in the Mandarin text library. This is convenient for subsequently converting the dialect text into Mandarin text, and finally using the Mandarin text as the text information.

[0033] At the same time, on this basis, a common voice library and at least one dialect voice library will still be built, and the voice library and the text library will be associated and mapped to facilitate the subsequent conversion of voice to text.

[0034] Preferably, the steps of converting voice information into text information in step S3 are as follows: Step S31: Obtain the first voice information input by the trainee as the first information, and convert the first information into Chinese pinyin through the Mandarin voice library; Step S32: After removing the tone information from the Chinese pinyin syllables, input them into the pinyin-to-Chinese character engine to generate one or more possible candidate Chinese character sequences for the input Chinese pinyin syllables; Step S33: Input the candidate Chinese character sequence into the Chinese character-to-pinyin library to obtain its standard pinyin representation with digital tones as the candidate Chinese pinyin sequence. Select the Chinese character ranked first in the Chinese pinyin sequence according to the removed tone information as the translation, and use the translation as the preset text; Step S34: Calculate the preset dialogue score using the preset text. If the preset dialogue score is greater than the score threshold, call the Mandarin voice library and the Mandarin text library as the language conversion tools used in this test to convert the subsequent voice information into text information; If the preset dialogue score is less than the score threshold, then execute step S35; Step S35: Select the dialect voice library in the order of priority, and convert the first information into dialect pinyin through the dialect voice library; After removing the tone information from the dialect pinyin syllables, input them into the pinyin-to-Chinese character engine to generate one or more possible candidate dialect Chinese character sequences for the input dialect pinyin syllables; Step S36: Input the candidate dialect text sequence into the corresponding dialect-to-pinyin library to obtain its standardized pinyin representation with numerical tones, as the candidate dialect pinyin sequence. Select the dialect with the highest ranking in the dialect pinyin sequence as the translation based on the removed tone information. Use the translation as the dialect text. Based on the established mapping relationship between the dialect text library and the dialect speech library, generate the dialect text as the converted text. Use the converted text to calculate the converted dialogue score. If the converted dialogue score is greater than the score threshold, call the corresponding dialect speech library and dialect text library as the language conversion tool used in this test, and execute step S37. If the converted dialogue score is less than the score threshold, then repeat step S35. Step S37: Convert the subsequent speech information into dialect text. Through the mapping relationship between the Mandarin text library and various dialect text libraries, convert the dialect text into Mandarin text as text information.

[0035] In order to correctly select the text and voice libraries used in this test, the first voice message entered by the trainee will be selected as the first message, because in the practice of nurses, the first voice message is basically asking questions of the patient, and most trainees can get a dialogue score.

[0036] As Mandarin is the most widely used language, the first piece of information is input into the Mandarin speech database, which then converts it into Pinyin. Since the amount of information in the first piece of information is relatively small, it is difficult to accurately translate the text, so the speech is first converted into Pinyin. Then, after removing the tone information from the Chinese Pinyin syllables, the Pinyin to Chinese character engine is input to generate multiple candidate Chinese character sequences. This helps reduce recognition errors caused by tone variations or non-standard pronunciation, because although tone is important in Chinese, it is easily affected by individual accents. After removing the tone, the engine can focus more on syllable matching and expand the candidate range. The Chinese character to Pinyin library (such as pypinyin) re-acquires candidate Pinyin sequences with tone and verifies them in combination with the previously removed tone information, thereby selecting the top-ranked Chinese character as the preset text. This process realizes secondary tone verification, effectively corrects possible homophone errors, and improves the accuracy of text conversion. After selection, the preset text is obtained, and the dialogue score is calculated using the preset text. If the preset dialogue score is greater than the score threshold (i.e., the dialogue score is obtained), it means that the current student is using Mandarin for the dialogue. Therefore, in the subsequent test, the Mandarin speech library and Mandarin text library are used as the language conversion tools to convert the subsequent speech information into text information. If there is no preset dialogue score less than the score threshold, it means that the trainee is not speaking Mandarin. At this time, the dialect speech library is selected according to the priority for conversion. Similarly, the language is first converted into pinyin, and then the corresponding dialect text is selected through the pinyin. Since Mandarin is used for dialogue scoring during dialogue evaluation, the dialect text needs to be converted into the "converted text" in Mandarin, and then the dialogue score is calculated through the converted text. If the converted dialogue score is greater than the score threshold (i.e., the dialogue score is obtained), it means that the current trainee is using the dialect for dialogue, and the corresponding dialect speech library and dialect text library are called as the language conversion tools used in this test. Subsequently, the voice information is converted into dialect characters, and through the mapping relationship between the Mandarin text library and each dialect text library, the dialect characters are converted into Mandarin text as the text information. If the converted dialogue score is less than the score threshold, the next dialect speech library is replaced for matching calculation.

[0037] It is worth mentioning that if there is a situation where the converted dialogue scores of all dialect speech libraries are less than the score threshold after matching, the ordinary speech library and Mandarin text library are used as the language conversion tools for this test.

[0038] The following is an example for explanation: For example, when the trainee asks in Cantonese "係咪唔舒服", it will be converted into the pinyin "ni hai mei hao" through the Mandarin speech library, and will finally be converted into "你还没好" through the candidate Chinese character sequence. This obviously does not conform to the dialogue between a nurse and a patient during their first meeting. Then the Cantonese speech library is replaced for pinyin conversion, and at this time, the pinyin "hai mai m syu1 fuk" will be obtained. Finally, the text obtained through conversion of the Chinese character sequence will be "係咪唔舒服", and then through the mapping relationship between the Mandarin text library and the Cantonese text library, the converted text "是不是不舒服" will be output. If a converted dialogue score is calculated for this converted text, finally the Cantonese speech library and Cantonese text library will be used as the language conversion tools for this test.

[0039] Preferably, if the test score of the trainee is lower than the score threshold, the voice information is sent to the corresponding communication address for manual verification.

[0040] Since the dialogue evaluation has a relatively large impact on the score, and errors may occur when selecting the dialect speech library and dialect text library, resulting in a relatively low overall score for the trainee. At this time, the voice information during the test process needs to be sent to the corresponding communication address for manual verification, and it is judged manually whether it is necessary to re-score the dialogue evaluation.

[0041] A virtual reality-based nursing education system for integrated medical and elderly care uses a virtual reality-based teaching method for integrated medical and elderly care, including a construction module, a testing module, a transformation module, and a scoring module. The building module is used to construct medical scenarios and analysis templates; and to link the scenarios and analysis modules. The testing module is used to obtain the scene selected by the trainee, start the virtual reality simulation, generate the corresponding virtual scene, and obtain the voice information and operation information issued by the user. The conversion module is used to convert voice information into text information. The text information, operation information and patient dialogue in medical scenarios are used to form at least one scoring group, and each scoring group also includes the time when the scoring group was generated. The scoring module combines each scoring group with the analysis template to obtain the score for each scoring group. By combining the scores of all scoring groups, the total score for the student's test is obtained.

[0042] Preferably, the conversion module includes a first submodule, a second submodule, a third submodule, a fourth submodule, a fifth submodule, a sixth submodule, and a seventh submodule; The first submodule is used to obtain the first voice information input by the student, and convert the first information into Chinese Pinyin through the ordinary voice library; The second submodule is used to generate one or more possible candidate Chinese character sequences for the input Chinese Pinyin syllables after removing the tone information from the Pinyin syllables. The third submodule is used to input the candidate Chinese character sequence into the Chinese character to Pinyin library (such as pypinyin) to obtain its standardized Pinyin representation with numerical tones, which is used as the candidate Chinese Pinyin sequence. Based on the removed tone information, the Chinese character ranked first in the Chinese Pinyin sequence is selected as the translation, and the translation is used as the preset text. The fourth submodule is used to calculate the preset dialogue score using preset text. If the preset dialogue score is greater than the score threshold, the ordinary speech library and the Mandarin text library are called as the language conversion tools used in this test to convert the subsequent speech information into text information. If the preset score is less than the score threshold, then the fifth submodule is invoked; The fifth submodule is used to select dialect speech libraries according to priority order, and the first information is converted into dialect pinyin through the dialect speech library; Based on the above, after removing the tone information from the dialect pinyin syllables, the input is fed into the pinyin to Chinese character engine to generate one or more possible candidate dialect character sequences for the input dialect pinyin syllables; The sixth submodule is used to input the candidate dialect text sequence into the corresponding dialect-to-pinyin library to obtain its standardized pinyin representation with numerical tones as the candidate dialect pinyin sequence. Based on the removed tone information, the dialect with the highest ranking in the dialect pinyin sequence is selected as the translation. The translation is used as the dialect text. Based on the established mapping relationship between the dialect text library and the dialect speech library, the dialect text is generated as the converted text. The converted text is used to calculate the converted dialogue score. If the converted dialogue score is greater than the score threshold, the corresponding dialect speech library and dialect text library are called as the language conversion tools used in this test, and the seventh submodule is called. If the conversion score is less than the score threshold, the fifth submodule will be called again. The seventh submodule is used to convert subsequent speech information into dialect text. Through the mapping relationship between the Mandarin text library and various dialect text libraries, the dialect text is converted into Mandarin text as text information.

[0043] Preferably, the scoring module includes an action scoring submodule, a dialogue scoring submodule, and a time adjustment submodule; The action scoring submodule is used to obtain the operation code clicked by the trainee, and determine whether the operation code is the same as the code in the analysis template. If they are the same, the action score is obtained; if they are different, the action score is not obtained. The dialogue scoring submodule is used to segment the text information in the scoring group and obtain multiple sentence segments; The individual sentence segments and the standard sentences in the analysis template are input into the trained semantic model to determine whether the semantics of the sentence segments and the standard sentences are the same or similar. If they are the same or similar, a dialogue score is obtained. If they are different, the next sentence segment is replaced and the semantic judgment is re-executed until all sentence segments have been input into the semantic model. If so, the dialogue is judged to receive no score. The time adjustment submodule is used to obtain the percentage of the time generated by the scoring group compared to the standard time, generate a time ratio, and adjust the sum of the dialogue score and the action score based on the time ratio.

[0044] Preferably, it also includes a manual processing module, which is used to send voice information to the corresponding communication address for manual verification when the student's test score is lower than the score threshold.

[0045] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0046] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A teaching method for nurses integrating medical and elderly care based on virtual reality, characterized in that, Includes the following steps: Step S1: Construct the medical scenario and analysis template; associate the scenario and analysis module; Step S2: Obtain the scene selected by the student, start the virtual reality simulation, generate the corresponding virtual scene, and obtain the voice information and operation information issued by the user; Step S3: Convert the voice information into text information, and use the text information, operation information and the text of the patient's dialogue in the medical scenario to form at least one scoring group, wherein each scoring group also includes the time when the scoring group was generated. Step S4: Combine each scoring group with the analysis template to obtain the score for each scoring group. Combine the scores of all scoring groups to obtain the score of the student in this test.

2. The method for teaching nurses integrating medical and elderly care based on virtual reality according to claim 1, characterized in that, The medical scenario includes patient pathology, patient dialogue, and patient actions, wherein the patient pathology has a unique identifier, and the patient dialogue and patient actions are bound to the patient pathology according to the unique identifier. Patient dialogues and actions are assigned priority levels, with lower-priority patient dialogues or actions only appearing after higher-priority patient dialogues or actions.

3. The method for teaching nurses integrating medical and elderly care based on virtual reality according to claim 1, characterized in that, In step S4, each scoring group is combined with the analysis template to obtain the score for each scoring group, including dialogue score, action score, and time ratio. Based on the patient's pathology, patient dialogue, and patient actions, the corresponding analysis template is selected; The steps for determining the motion frequency score are as follows: Obtain the operation code clicked by the student, and determine whether the operation code is the same as the code in the analysis template. If they are the same, obtain the action score; if they are different, do not obtain the action score. The steps for determining the dialogue score are as follows: The text information in the scoring group is segmented to obtain multiple sentence segments; The individual sentence segments and the standard sentences in the analysis template are input into the trained semantic model to determine whether the semantics of the sentence segments and the standard sentences are the same or similar. If they are the same or similar, a dialogue score is obtained. If they are different, the next sentence segment is replaced and the semantic judgment is re-executed until all sentence segments have been input into the semantic model. If so, the dialogue is judged to receive no score. The rules for using time proportions are as follows: Obtain the percentage of the time generated by the scoring group compared to the standard time, the generation time ratio, and adjust the sum of the dialogue score and the action score based on the time ratio.

4. The method for teaching nurses integrating medical and elderly care based on virtual reality according to claim 3, characterized in that, The following steps need to be performed before step S3: Step A: Construct a Standard Mandarin text database and at least one dialect text database; establish the mapping relationship between the Standard Mandarin text database and each dialect text database; Step B: Construct a standard speech database and at least one dialect speech database, establish a text mapping relationship between the standard speech database and the standard Mandarin text database, establish a mapping relationship between the dialect text database and the dialect speech database, and set the recognition priority of the dialect speech database.

5. The teaching method for nurses integrating medical and elderly care based on virtual reality according to claim 4, characterized in that, The step of converting speech information into text information in step S3 is as follows: Step S31: Obtain the first voice information input by the student as the first information, and convert the first information into Chinese Pinyin through the Mandarin voice database; Step S32: Based on the removal of tone information from the Chinese Pinyin syllables, input them into the Pinyin to Chinese character engine to generate one or more possible candidate Chinese character sequences for the input Chinese Pinyin syllables; Step S33: Input the candidate Chinese character sequence into the Chinese character to Pinyin library to obtain its standardized Pinyin representation with numerical tones, as a candidate Chinese Pinyin sequence. Select the Chinese character ranked first in the Chinese Pinyin sequence as the translation based on the removed tone information, and use the translation as the preset text. Step S34: Calculate the preset dialogue score using the preset text. If the preset dialogue score is greater than the score threshold, call the Mandarin speech library and Mandarin text library as the language conversion tools used in this test to convert the subsequent speech information into text information. If the preset dialogue score is less than the score threshold, then proceed to step S35; Step S35: Select the dialect speech library according to priority order, and convert the first information into dialect pinyin through the dialect speech library; Based on the above, after removing the tone information from the dialect pinyin syllables, the input is fed into the pinyin to Chinese character engine to generate one or more possible candidate dialect character sequences for the input dialect pinyin syllables; Step S36: Input the candidate dialect text sequence into the corresponding dialect-to-pinyin library to obtain its standardized pinyin representation with numerical tones, as the candidate dialect pinyin sequence. Select the dialect with the highest ranking in the dialect pinyin sequence as the translation based on the removed tone information. Use the translation as the dialect text. Based on the established mapping relationship between the dialect text library and the dialect speech library, generate the dialect text as the converted text. Use the converted text to calculate the converted dialogue score. If the converted dialogue score is greater than the score threshold, call the corresponding dialect speech library and dialect text library as the language conversion tool used in this test, and execute step S37. If the converted dialogue score is less than the score threshold, then repeat step S35. Step S37: Convert the subsequent speech information into dialect text. Through the mapping relationship between the Mandarin text library and various dialect text libraries, convert the dialect text into Mandarin text as text information.

6. The method for teaching nurses integrating medical and elderly care based on virtual reality according to claim 1, characterized in that, If a student's test score is below the score threshold, a voice message will be sent to the corresponding communication address for manual verification.

7. A virtual reality-based nursing education system for integrated medical and elderly care, using the virtual reality-based nursing education method for integrated medical and elderly care as described in any one of claims 1 to 6, characterized in that, It includes a build module, a test module, a conversion module, and a scoring module; The building module is used to construct medical scenarios and analysis templates; and to link the scenarios and analysis modules. The testing module is used to obtain the scene selected by the trainee, start the virtual reality simulation, generate the corresponding virtual scene, and obtain the voice information and operation information issued by the user. The conversion module is used to convert voice information into text information. The text information, operation information and patient dialogue in medical scenarios are used to form at least one scoring group, and each scoring group also includes the time when the scoring group was generated. The scoring module combines each scoring group with the analysis template to obtain the score for each scoring group. By combining the scores of all scoring groups, the total score for the student's test is obtained.

8. A virtual reality-based nursing education system for integrated medical and elderly care, as described in claim 7, is characterized in that... The conversion module includes a first submodule, a second submodule, a third submodule, a fourth submodule, a fifth submodule, a sixth submodule, and a seventh submodule; The first submodule is used to obtain the first voice information input by the student, and convert the first information into Chinese Pinyin through the ordinary voice library; The second submodule is used to generate one or more possible candidate Chinese character sequences for the input Chinese Pinyin syllables after removing the tone information from the Pinyin syllables. The third submodule is used to input the candidate Chinese character sequence into the Chinese character to Pinyin library to obtain its standardized Pinyin representation with numerical tones, which is used as the candidate Chinese Pinyin sequence. Based on the removed tone information, the Chinese character ranked first in the Chinese Pinyin sequence is selected as the translation, and the translation is used as the preset text. The fourth submodule is used to calculate the preset dialogue score using preset text. If the preset dialogue score is greater than the score threshold, the ordinary speech library and the Mandarin text library are called as the language conversion tools used in this test to convert the subsequent speech information into text information. If the preset score is less than the score threshold, then the fifth submodule is invoked; The fifth submodule is used to select dialect speech libraries according to priority order, and the first information is converted into dialect pinyin through the dialect speech library; Based on the above, after removing the tone information from the dialect pinyin syllables, the input is fed into the pinyin to Chinese character engine to generate one or more possible candidate dialect character sequences for the input dialect pinyin syllables; The sixth submodule is used to input the candidate dialect text sequence into the corresponding dialect-to-pinyin library to obtain its standardized pinyin representation with numerical tones as the candidate dialect pinyin sequence. Based on the removed tone information, the dialect with the highest ranking in the dialect pinyin sequence is selected as the translation. The translation is used as the dialect text. Based on the established mapping relationship between the dialect text library and the dialect speech library, the dialect text is generated as the converted text. The converted text is used to calculate the converted dialogue score. If the converted dialogue score is greater than the score threshold, the corresponding dialect speech library and dialect text library are called as the language conversion tools used in this test, and the seventh submodule is called. If the conversion score is less than the score threshold, the fifth submodule will be called again; The seventh submodule is used to convert subsequent speech information into dialect text. Through the mapping relationship between the Mandarin text library and various dialect text libraries, the dialect text is converted into Mandarin text as text information.

9. A virtual reality-based nursing education system for integrated medical and elderly care, as described in claim 7, is characterized in that... The scoring module includes an action scoring submodule, a dialogue scoring submodule, and a time adjustment submodule; The action scoring submodule is used to obtain the operation code clicked by the trainee, and determine whether the operation code is the same as the code in the analysis template. If they are the same, the action score is obtained; if they are different, the action score is not obtained. The dialogue scoring submodule is used to segment the text information in the scoring group and obtain multiple sentence segments; The individual sentence segments and the standard sentences in the analysis template are input into the trained semantic model to determine whether the semantics of the sentence segments and the standard sentences are the same or similar. If they are the same or similar, a dialogue score is obtained. If they are different, the next sentence segment is replaced and the semantic judgment is re-executed until all sentence segments have been input into the semantic model. If so, the dialogue is judged to receive no score. The time adjustment submodule is used to obtain the percentage of the time generated by the scoring group compared to the standard time, generate a time ratio, and adjust the sum of the dialogue score and the action score based on the time ratio.

10. A virtual reality-based nursing education system for integrated medical and elderly care, as described in claim 7, is characterized in that... It also includes a manual processing module, which is used to send voice information to the corresponding communication address for manual verification when a trainee's test score is lower than the score threshold.

Images