Interaction equipment, medical scanning system and control method of interaction equipment

Abstract

The invention relates to interaction equipment, a medical scanning system and a control method of the interaction equipment, and the interaction equipment comprises an interaction terminal device and a processing device. The interactive terminal device is used for collecting inquiry information of an object to be inquired; the processing device is connected with the interaction terminal device and is used for inputting the inquiry information into a completely trained target language model for recognition processing and outputting a first language recognition result; the target language model is obtained by training based on a preset medical scanning knowledge database; the processing device is further used for performing first retrieval processing from the medical scanning knowledge database based on the first language recognition result to obtain a first retrieval result, and generating a target reply result for the to-be-queried object according to the first retrieval result. According to the invention, the problem of low safety of medical scanning examination is solved.

Description

Technical Field

[0001] This application relates to the technical field of medical scanning, and particularly to an interaction device, a medical scanning system, and a control method for the interaction device. Background Art

[0002] Medical scanning examinations such as Magnetic Resonance Imaging (MRI) have been increasingly widely used because they are non-invasive to the object to be examined, can detect lesions at an early stage, accurately detect the size and scope of the lesions, and have a high accuracy rate in qualitative diagnosis. Taking magnetic resonance examination as an example, during the magnetic resonance examination, since the object to be examined is not clear about the magnetic resonance examination process, it often happens that the object to be examined pushes a metal wheelchair into the scanning room or carries metal implants, etc., which affects the safety of the examination during medical scanning processes such as magnetic resonance scanning.

[0003] Currently, for the problem of low safety of medical scanning examinations in related technologies, no effective solution has been proposed. Summary of the Invention

[0004] Embodiments of this application provide an interaction device, a medical scanning system, and a control method for the interaction device to at least solve the problem of low safety of magnetic resonance examinations in related technologies.

[0005] In a first aspect, embodiments of this application provide an interaction device applied to a medical scanning system. The device includes: an interaction terminal device and a processing device;

[0006] The interaction terminal device is used to collect inquiry information of the object to be examined;

[0007] The processing device is connected to the interaction terminal device and is used to input the inquiry information into a trained and complete target language model for recognition processing and output a first language recognition result; the target language model is trained based on a preset medical scanning knowledge database;

[0008] The processing device is further used to perform a first retrieval process in the medical scanning knowledge database based on the first language recognition result to obtain a first retrieval result, and generate a target reply result for the object to be examined according to the first retrieval result.

[0009] In some of these embodiments, the processing device is further used to use the target language model to perform emotion recognition processing on the inquiry information to obtain a first emotion recognition result;

[0010] The processing device is further configured to perform a second retrieval process based on the first emotion recognition result from the medical scan knowledge database to obtain a second retrieval result, and generate the target reply result according to the first retrieval result and the second retrieval result.

[0011] In some embodiments, the device further includes an image acquisition device;

[0012] The image acquisition device is configured to acquire the face image information of the object to be queried;

[0013] The processing device is further configured to perform emotion recognition processing based on the face image information to obtain a second emotion recognition result, and generate a target emotion recognition result according to the first emotion recognition result and the second emotion recognition result;

[0014] The processing device is further configured to perform a third retrieval process based on the target emotion recognition result from the medical scan knowledge database to obtain a third retrieval result, and generate the target reply result according to the first retrieval result and the third retrieval result.

[0015] In some embodiments, the interactive terminal device is further configured to acquire the query information of the object to be queried;

[0016] The processing device is further configured to input the query information into a pre-stage language model in the target language model for recognition processing, and output a second language recognition result;

[0017] The processing device is further configured to, in response to detecting the second language recognition result, input the acquired multimodal medical scan information into a post-stage sub-model in the target language model for information parsing processing to obtain a parsing result for the multimodal medical scan information; wherein, the pre-stage language model is connected to the post-stage sub-model;

[0018] The processing device is further configured to generate a diagnostic reply result for the object to be queried based on the parsing result.

[0019] In some embodiments, the interactive terminal device is further configured to acquire the login information of the object to be queried;

[0020] The processing device includes an encryption unit; the encryption unit is configured to determine the encryption information of the object to be queried according to the login information, and perform desensitization processing on the target reply result according to the encryption information to obtain a first reply result; and / or,

[0021] The encryption unit of the processing device is configured to acquire the function permission information for the medical scan system, and perform filtering processing on the target reply result according to the function permission information to obtain a second reply result.

[0022] In some of these embodiments, the encryption unit is further configured to obtain the source information of the medical scan knowledge database and publish the target language model to the medical institution device corresponding to the source information.

[0023] In some of these embodiments, the processing device includes an authentication unit and an authorization unit;

[0024] The authentication unit is configured to perform gateway authentication based on the login information of the object to be queried to obtain a first authentication result corresponding to the object to be queried;

[0025] The authorization unit is configured to receive the query information of the object to be queried when the first authentication result indicates that the identity of the object to be queried passes; and / or,

[0026] The authentication unit is further configured to perform gateway authentication based on the login information of the operator to obtain a second authentication result corresponding to the operator;

[0027] The authorization unit is further configured to determine the authorization result of the operator based on the second authentication result.

[0028] In some of these embodiments, the processing device further includes a training unit;

[0029] The training unit is further configured to obtain the preset medical scan knowledge data in the medical scan knowledge database and the annotation information carried by the preset medical scan knowledge data, and perform prompt engineering processing on the initial language model according to the preset medical scan knowledge data and the annotation information to obtain the target language model; and / or,

[0030] The training unit is further configured to update the medical scan knowledge database at least according to the first language recognition result to obtain a new medical scan knowledge database.

[0031] In a second aspect, an embodiment of the present application provides a medical scan system, and the system includes the interaction device as described in the first aspect above.

[0032] In a third aspect, an embodiment of the present application provides a control method for an interaction device. The interaction device is applied to a medical scan system, and the device includes an interaction terminal device and a processing device; the method includes:

[0033] The interaction terminal device collects the query information of the object to be queried;

[0034] The processing device inputs the inquiry information into a trained and complete target language model for recognition processing, and outputs a first language recognition result; wherein, the target language model is trained and generated based on the obtained medical scan knowledge database.

[0035] The processing device performs a retrieval process in the medical scan knowledge database based on the first language recognition result, and generates a target reply result for the object to be queried according to the retrieval result.

[0036] Compared with the related art, the embodiments of the present application provide an interaction device, a medical scan system, and a control method for the interaction device. The interaction device includes: an interaction terminal device and a processing device; the interaction terminal device is used to collect inquiry information of the object to be queried; the processing device is connected to the interaction terminal device and is used to input the inquiry information into a trained and complete target language model for recognition processing, and output a first language recognition result; the target language model is trained based on a preset medical scan knowledge database; the processing device is further used to perform a first retrieval process in the medical scan knowledge database based on the first language recognition result to obtain a first retrieval result, and generate a target reply result for the object to be queried according to the first retrieval result, solving the problem of low safety in medical scan examinations.

[0037] Details of one or more embodiments of the present application are set forth in the following drawings and description, so that other features, objects, and advantages of the present application will become more clearly understood. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The illustrative embodiments and descriptions thereof of the present application are used to explain the present application, and do not constitute an improper limitation of the present application. In the drawings:

[0039] Figure 1 is a block diagram of the hardware structure of a terminal of a control method for an interaction device according to an embodiment of the present application;

[0040] Figure 2 is a block diagram of the structure of an interaction device according to an embodiment of the present application;

[0041] Figure 3 is a schematic diagram of an interactive training platform according to an embodiment of the present application;

[0042] Figure 4 is a flowchart of a control method for an interaction device according to an embodiment of the present application;

[0043] Figure 5 is a block diagram of the internal structure of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be described and explained below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments provided in the present application without making creative efforts fall within the scope of protection of the present application. In addition, it can also be understood that although the efforts made in such a development process may be complex and lengthy, for those of ordinary skill in the art related to the content disclosed in the present application, some design, manufacturing or production changes based on the technical content disclosed in the present application are only conventional technical means and should not be understood as the content disclosed in the present application being insufficient.

[0045] Referring to "embodiments" in the present application means that the specific features, structures or characteristics described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those of ordinary skill in the art explicitly and implicitly understand that the embodiments described in the present application can be combined with other embodiments without conflict.

[0046] Unless otherwise defined, the technical terms or scientific terms involved in the present application should be of the ordinary meaning understood by those with ordinary skills in the technical field to which the present application belongs. The terms "a", "an", "one", "the" and similar words involved in the present application do not indicate a limitation in quantity and may represent a singular or plural number. The terms "including", "comprising", "having" and any variations thereof involved in the present application are intended to cover non-exclusive inclusion; for example, a process, method, system, product or device including a series of steps or modules (units) is not limited to the listed steps or units, but may further include unlisted steps or units, or may further include other steps or units inherent to these processes, methods, products or devices. The terms "connected", "coupled" and similar words involved in the present application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in the present application means greater than or equal to two. "And / or" describes the association relationship of associated objects and indicates that three relationships may exist. For example, "A and / or B" may represent: A exists alone, A and B exist simultaneously, and B exists alone. The terms "first", "second", "third", etc. involved in the present application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0047] The method embodiments provided in this embodiment can be executed on a terminal, a computer, or a similar computing device. Taking running on a terminal as an example, Figure 1 is a hardware structure block diagram of a terminal for a control method of an interaction device according to an embodiment of the present application. As Figure 1 shown, the terminal may include one or more ( Figure 1 only one is shown in the figure) processors 102 (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the above terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above terminal. For example, the terminal may further include more or fewer components than Figure 1 shown in the figure, or have a different configuration from Figure 1 shown in the figure.

[0048] The memory 104 can be used to store computer programs. For example, software programs and modules of application software, such as the computer program corresponding to the control method of the interaction device in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implements the above method. The memory 104 may include a high-speed random access memory, and may further include a non-volatile memory, such as one or more magnetic storage devices, a flash memory, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely provided with respect to the processor 102, and these remote memories may be connected to the terminal through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0049] The transmission device 106 is used to receive or send data via a network. Specific examples of the above network may include a wireless network provided by a communication provider of the terminal. In one instance, the transmission device 106 includes a network adapter (Network Interface Controller, abbreviated as NIC), which can be connected to other network devices through a base station and thus can communicate with the Internet. In one instance, the transmission device 106 may be a radio frequency (Radio Frequency, abbreviated as RF) module, which is used to communicate with the Internet wirelessly.

[0050] This embodiment provides an interaction device, which is applied to a medical scanning system; Figure 2 is a structure block diagram of an interaction device according to an embodiment of the present application. As Figure 2 shown, the device includes an interaction terminal device 22 and a processing device 24.

[0051] The above-mentioned interactive terminal device 22 is used to collect the inquiry information of the object to be examined. Among them, the above-mentioned medical scanning system is used to perform medical scanning and detection on the object to be examined. Taking the example that the medical scanning system is a magnetic resonance system, the magnetic resonance system has completed abdominal magnetic resonance examination by performing magnetic resonance scanning on the abdomen of a certain object to be examined; since the object to be examined may not understand the process and precautions of magnetic resonance scanning before the magnetic resonance examination, the object to be examined can interact and inquire with the interactive terminal device 22. Specifically, the interactive terminal device 22 can be a hardware device such as a self-service terminal, a mobile phone or a personal computer deployed in the medical scanning preparation room, on which a software program for interacting with the object to be examined is deployed; then the object to be examined inputs a voice command or a text command to the interactive terminal device to inquire about questions related to medical scanning, and then the interactive terminal device collects the inquiry information of the object to be examined. In another embodiment, while collecting the inquiry information, the above-mentioned interactive terminal device 22 can also collect other modal data inputs, such as physiological signal detection, images, etc., so as to realize an interactive processing method integrating multi-modal data and provide more comprehensive health management for the object to be examined.

[0052] The above-mentioned processing device 24 is connected to the above-mentioned interactive terminal device 22 and is used to input the inquiry information into a trained target language model for recognition processing and output a first language recognition result; the target language model is trained based on a preset medical scanning knowledge base. Among them, the processing device 24 includes but is not limited to various single-chip microcomputers, main control chips or servers and other hardware devices for processing the interactive information of the above-mentioned object to be examined. The processing device 24 can be integrated on the above-mentioned interactive terminal device 22; or, the processing device 24 can also be installed independently and communicatively connected to the interactive terminal device 22. Specifically, the processing device 24 inputs the inquiry information collected by the above-mentioned interactive terminal device 22 into the target language model for keyword disassembling and recognition, and obtains the keywords related to medical scanning knowledge obtained by disassembling the voice or text information of the object to be examined's question output by the target language model, that is, the above-mentioned first language recognition result. It should be noted that in order to ensure the accuracy of the interactive device's recognition of the object to be examined's questions about medical scanning-related knowledge, in this embodiment, the above-mentioned processing device 24 can first perform prompt engineering optimization processing on the language model based on a medical scanning knowledge base pre-stored with relevant knowledge, so that when it comes to professional fields, such as magnetic resonance scanning-related knowledge, language recognition can be performed through the optimized and trained target language model combined with in-depth prompt engineering in this field, so as to realize a deeper interaction between the interactive device and the corresponding object to be examined.

[0053] The above processing device 24 is further configured to perform a first retrieval process on the medical scan knowledge database based on the first language recognition result, obtain a first retrieval result, and generate a target reply result for the object to be queried according to the first retrieval result. Among them, the data stored in the above medical scan knowledge base may include medical scan-related data, clinical knowledge, or historical data collected during actual application, etc., related keyword data, as well as standard answer data corresponding to various types of related keyword data; various keywords and standard answer data can be stored in the medical scan knowledge base through data structures such as key-value pairs. Then, in actual application, the processing device 24 traverses and retrieves from the stored data in the medical scan knowledge database based on the keywords included in the first language recognition result adaptively recognized and output by the target language model, and uses the stored keywords retrieved from the database and matching the keyword to be recognized as the first retrieval result for the first language recognition result. Then, the standard answer data corresponding to the first retrieval result in the medical scan knowledge database is the above-mentioned target reply result. At this time, the above processing device 24 can feedback the target reply result to the above interactive terminal device 22, and the target reply result is voice broadcast or text displayed through the interactive terminal device 22, thereby completing the adaptive real-time interaction with the above object to be queried. It should be noted that during the above interaction process, the processing device 24 in the interaction device can also analyze the risk level of the object to be queried in real time according to the conversation and interaction process with the object to be queried, and provide a timely warning notice; at the same time, the processing device can also track the health status of the object to be queried according to the above analysis data and generate a customized health report.

[0054] This application provides an interactive device applied to a magnetic resonance system. Through the processing device 24 in the interactive device, the inquiry information of the object to be queried collected is input into the target language model for recognition, and the magnetic resonance knowledge database is retrieved based on the recognized result to obtain a targeted target reply result, so as to be able to efficiently and accurately adaptively reply to the inquiry information of the object to be queried regarding the magnetic resonance examination process, effectively avoiding the phenomenon that the operation error of the object to be queried affects the magnetic resonance examination, and solving the problem of low safety in medical scan examinations; at the same time, through the above device to achieve intelligent interaction with the object to be queried, it also avoids the phenomenon that the operator needs to repeatedly spend a lot of time repeating the same reply to different objects to be queried, effectively improving the interaction efficiency with the object to be queried, and further improving the medical scan examination efficiency.

[0055] In some of these embodiments, the above processing device is further configured to use the target language model to perform emotion recognition processing on the language information to obtain a first emotion recognition result. It should be noted that the above target language model is trained and generated based on a medical scan knowledge base. Therefore, in this embodiment, the medical scan knowledge database can be combined, and the above target language model can be used to perform emotion recognition processing on the language related to medical scans to obtain the above first emotion recognition result. The first emotion recognition result includes the emotional state of the object to be examined recognized based on the language information of the object to be examined, such as states of anxiety, fear, etc.

[0056] The above processing device is further configured to perform a second retrieval process based on the first emotion recognition result from the medical scan knowledge database to obtain a second retrieval result, and generate the target reply result according to the first retrieval result and the second retrieval result. Specifically, the processing device performs keyword retrieval on the medical scan knowledge database according to the above first emotion recognition result. When an emotion keyword matching the first emotion recognition result is retrieved from the database, that is, when the above second retrieval result is obtained, based on the reply information corresponding to the second retrieval result in the database and the reply information corresponding to the above first retrieval result, the above target reply result is comprehensively generated; alternatively, the processing device can also screen out the reply data corresponding to the second retrieval result from the standard reply data corresponding to the first retrieval result as the above target reply result. For example, the processing device, through the above target language model, recognizes that the object to be examined is currently in a state of anxiety based on the inquiry information of the object to be examined. At this time, corresponding communication strategies and explanation methods can be retrieved from the database based on this emotion recognition result to help the object to be examined better cope with and eliminate psychological pressure, thereby facilitating the improvement of the interaction efficiency between the interaction device and the object to be examined.

[0057] In some of these embodiments, the above device further includes an image acquisition device; the image acquisition device is configured to acquire the face image information of the object to be examined. Among them, the image acquisition device includes, but is not limited to, various cameras, video recorders, or other hardware devices for acquiring the face images and videos of the object to be examined. The image acquisition device can be installed and deployed near the actual application site of the interaction device and communicate with the above processing device; alternatively, the image acquisition device can also be integrally installed on the above interaction device.

[0058] The above processing device is also used to perform emotion recognition processing based on the face image information to obtain a second emotion recognition result, and generate a target emotion recognition result based on the first emotion recognition result and the second emotion recognition result. Specifically, the processing device can perform expression recognition and classification on the above face image information through a neural network model to obtain the above second emotion recognition result, and then conduct comprehensive analysis with the first emotion recognition result recognized by the above target language model to finally obtain the target emotion recognition result. For example, if the processing device detects through deep learning technology that the facial expression in the current face image information is frowning, and based on the target language model, it recognizes that the emotion corresponding to the inquiry language information is anxiety, then it can comprehensively analyze to obtain a target emotion recognition result that the emotion state of the object to be investigated is anxiety. In another embodiment, the processing device can also pre-assign corresponding weight values to various emotion recognition results, and perform weighted calculation on the first emotion recognition result and the second emotion recognition result based on the above weight values to obtain the above target emotion result, so as to further improve the accuracy of the comprehensive analysis and recognition of the emotion of the object to be investigated.

[0059] The processing device is also used to perform a third retrieval process based on the target emotion recognition result from the medical scan knowledge database to obtain a third retrieval result, and generate the target reply result based on the first retrieval result and the third retrieval result. Similarly, the processing device retrieves the medical scan knowledge database based on the target emotion recognition result obtained from the above comprehensive analysis. When an emotion keyword matching the target emotion recognition result is retrieved from the database, that is, when the above third retrieval result is obtained, the standard reply data matching the third retrieval result and the first retrieval result is queried based on the medical scan knowledge database, and finally the above target reply result is generated.

[0060] It should be noted that in the interactive device of the present application, the above image acquisition device can be used to have a video conversation with the object to be investigated, rather than just a text-based answer. Through the above interactive video conversation technology, non-verbal cues of the facial expression and voice of the object to be investigated can be analyzed to provide a more personalized service. Video interaction can help capture the emotions and reactions of the object to be investigated more accurately, which may be crucial for diagnosis in some cases.

[0061] Through the above embodiments, the processing device performs auxiliary emotion recognition processing on the face image information of the object to be investigated collected, and conducts comprehensive analysis processing based on the emotion recognition results obtained from face image recognition and language analysis respectively to obtain the final emotion recognition result. Thus, compared with a single emotion recognition method, it can effectively improve the accuracy of emotion recognition, and further improve the accuracy of interaction with the object to be investigated during the medical scan process.

[0062] In some of these embodiments, the above interactive terminal device is further configured to collect query information of the object to be queried. It should be noted that after the above medical scanning system finishes the medical scanning of the object to be queried, there may be a situation where the object to be queried needs to query the medical scanning results. At this time, the object to be queried can input query information about the relevant medical scanning results to be queried to the interactive terminal device, and then the interactive terminal device can collect it for subsequent processing.

[0063] The processing device is further configured to input the query information into a pre-stage language model in the target language model for recognition processing and output a second language recognition result. It can be understood that, similar to the recognition processing of the inquiry information above, in this embodiment, the processing device inputs the above query information into the target language model, and the target language model is used to first extract the keywords related to medical scanning in the query voice result, that is, the second language recognition result is obtained.

[0064] The processing device is further configured to, in response to the detected second language recognition result, input the obtained multi-modal medical scanning information into a post-stage sub-model in the target language model for information parsing processing to obtain a parsing result for the multi-modal medical scanning information; wherein, the pre-stage language model is connected to the post-stage sub-model. The above multi-modal medical scanning information refers to scanning information of at least one modality adaptively scanned by the medical scanning system for the object to be queried. Specifically, in this embodiment, at least one post-stage sub-model can be added to the target language model, so as to realize the recognition and extraction of keywords in the query information by the pre-stage language model, and determine the processing requirements based on the recognized keywords, and the corresponding post-stage sub-model of the processing requirements performs corresponding parsing processing on the above multi-modal medical scanning information. Exemplarily, the above multi-modal medical scanning information may be a magnetic resonance scan image, and then the magnetic resonance scan image can be analyzed and interpreted based on the above post-stage sub-model, and the obtained interpretation results may include image parts, composition, image quality (whether there are artifacts), diagnosis results, and imaging analysis results, etc. Finally, the processing device generates a diagnostic reply result for the object to be queried based on the parsing result.

[0065] Through the above embodiments, after the medical scanning of the object to be queried is completed, the above interactive device can, according to the needs of the object to be queried, answer its doubts about medical terms in the diagnostic report and help the object to be queried better understand its own medical scanning situation, thereby effectively improving the intelligence level of the interactive device in the actual application of medical scanning.

[0066] In some of these embodiments, the above interactive terminal device is further configured to obtain the login information of the object to be queried. The above interactive terminal device can provide a login channel for the object to be queried; among them, when the object to be queried logs in for the first time, registration can be performed first. For example, binding can be completed according to the mobile phone number, WeChat number, cloud platform account, etc. to complete registration. Each object to be queried logs in to the station through the login channel provided by the interactive terminal device. If the login is successful, the current login information is obtained by the interactive terminal device and interacts with the logged-in object to be queried; if the login fails, the interactive terminal device can display a login failure reminder message to remind the object to be queried to log in again until the login is successful.

[0067] The above processing device includes an encryption unit; the encryption unit is configured to determine the encryption information of the object to be queried according to the login information, and perform desensitization processing on the target reply result according to the encryption information to obtain a first reply result. Specifically, the encrypted user determines the privacy information related to the object to be queried based on the above login information, such as age, height, etc. The privacy information can be set as the encryption information, the reply content in the target reply result that may expose the relevant privacy information is deleted, and the reply data after deletion and optimization is used as the above first reply result. Through the above embodiments, desensitization encryption is performed by the encryption unit in the processing device, thereby implementing strict data protection measures, ensuring the privacy and security of patient personal information and diagnosis results, and effectively improving the privacy and security of the interactive device.

[0068] In another embodiment, the encryption unit of the above processing device is configured to obtain the function permission information for the medical scanning system, and perform filtering processing on the target reply result according to the function permission information to obtain a second reply result. The function permission information refers to functions such as the identified market value and additional medical scanning processing functions of the medical scanning system; the above target language model's possible replies are filtered through a license, thereby realizing reply optimization based on the function permission information.

[0069] In some of these embodiments, the encryption unit is further configured to obtain the source information of the medical scan knowledge database and publish the target language model to the medical institution device corresponding to the source information. The above source information refers to the identification information corresponding to the medical institution that provides the stored data in the medical scan knowledge base; the source information can be pre-stored in the medical scan knowledge database. Through the source information detected by the above encryption unit, the target language model established based on the medical scan knowledge database is published to the corresponding medical institution device, so that the data retrieved from Hospital A is not allowed to be published to other hospitals, and the network trained based on this data will only be deployed to the intelligent devices used by Hospital A, thereby enabling seamless integration between the interactive terminal and the devices of medical institutions such as hospitals and clinics, achieving rapid access and analysis of medical resources such as diagnostic reports and scan images, and at the same time ensuring that there is no privacy leakage between medical institutions, which is beneficial to improving the data privacy of the interactive device.

[0070] Through the above embodiments, the data privacy and security are enhanced based on the above encryption unit, which can ensure the secure transmission and storage of patient data and the protection of the privacy of the object to be examined, especially when processing video data and personal health records, thus realizing the strengthening of the data privacy and security of the interactive device.

[0071] In some of these embodiments, the processing device includes an authentication unit and an authorization unit; the authentication unit is configured to perform gateway authentication according to the login information of the object to be examined to obtain the first authentication result corresponding to the object to be examined; the authorization unit is configured to receive the inquiry information of the object to be examined when the first authentication result indicates that the identity of the object to be examined passes. Further, based on the above first authentication result, the above interactive device only provides the function of providing adaptive professional answers to the object to be examined, and for other functions, such as viewing sensitive data in the background, etc., they are not open to the object to be examined, thereby further ensuring the privacy and security of the interactive device.

[0072] And / or, the above authentication unit is further configured to perform gateway authentication according to the login information of the operator to obtain the second authentication result corresponding to the operator; the authorization unit is further configured to determine the authorization result of the operator based on the second authentication result. Specifically, in order to ensure the regularity of the operator's operation of the medical scan system, the above authentication unit can first verify the operator's permissions. If it is detected based on the second authentication result that the operator does not have the permission to operate some medical scan functions, then these functions will not be open to the operator.

[0073] In some of these embodiments, the above processing device further includes a training unit; the training unit is further configured to obtain preset medical scan knowledge data in the medical scan knowledge database, as well as annotation information carried by the preset medical scan knowledge data, and perform prompt engineering processing on the initial language model according to the preset medical scan knowledge data and the annotation information to obtain the target language model. Among them, the sources of the above preset medical scan knowledge data may include: 1. Medical scan-related books and web page knowledge, clinical knowledge, etc.; 2. Problem data actually generated during the application process of the interaction device; 3. Manual annotation information of the staff on the data collected in points 1 and 2. Then, during the process of optimizing the language model after pre-training the initial language model, the above artificial annotation information can be used to optimize or hypnotize the network. For example, for political issues, annotation information on issues that need to be avoided is provided, so that when the optimized target language model recognizes keywords related to politics, it will provide a reminder that the question has nothing to do with medical scans and a reply result. Through the above embodiments, prompt engineering processing is performed on the initial language model through annotation information, thereby effectively improving the accuracy of language model recognition.

[0074] And / or, the above training unit is further configured to update the medical scan knowledge database at least according to the first language recognition result to obtain a new medical scan knowledge database. Specifically, the training unit adds data including keywords and reply data to the medical scan knowledge database by obtaining historical data such as the first language recognition result copied or collected online and returned by the staff, and then iteratively upgrades the medical scan knowledge database, realizing the real-time iterative update of the medical scan knowledge database. Through the above embodiments, a context-aware and adaptive learning module that can learn and adapt to the conversation history of each patient is provided. It can not only provide information according to the current conversation, but also adjust the subsequent conversation process according to the previous conversation content, providing a more personalized question-and-answer experience.

[0075] The embodiments of the present application will be described in detail below in combination with actual application scenarios. Taking the medical scan system as a magnetic resonance system as an example, please refer to Figure 3 . A certain object to be examined needs to undergo an abdominal magnetic resonance examination. Before the examination, the object to be examined knows nothing about the process and precautions of magnetic resonance scanning. In the related art, the object to be examined needs to manually complete a series of preliminary preparations, including filling out a health questionnaire, receiving guidance from the operator for positioning, and remaining as still as possible during the scanning process, resulting in low interaction efficiency and possible problems such as incorrect positioning of the object to be examined during the scanning process, leading to low efficiency and accuracy of magnetic resonance scanning.

[0076] To address the above issues, in this embodiment, an interactive training platform based on a large language model is provided for the above-mentioned interactive device to enable the object to be examined to understand relevant information about magnetic resonance examinations. Before the object to be examined enters the scanning room for the examination, the object can log in to the platform via a mobile phone and start communicating with the large language model. The platform identifies the person who has made an appointment through the camera, and automatically obtains their pathology and the pre-determined scanning protocol from the medical imaging system. The platform recognizes that the object to be examined is a novice user and provides targeted answers based on the inquiry information input by the object to be examined. For example, it explains the principle of magnetic resonance scanning, the differences from CT / XR, and the safety of magnetic resonance scanning to the object to be examined. During the above process, the platform can use a voice interaction module such as a speaker to introduce the scanning process and precautions to the object to be examined in their native language and ask if there are any discomforts or questions, thereby reducing misunderstandings and unnecessary tension. In addition, the platform also tells the object to be examined about the precautions to be followed before the examination, such as removing metal items on the body and informing the operator about metal implants in advance, in order to help identify whether the object to be examined has metallic objects on the body, a cardiac pacemaker, has undergone surgery, or has dentures through conversations with the object to be examined. At the same time, a millimeter-wave radar installed outside the scanning room can assist the object to be examined in performing breath-holding training. During the communication process between the object to be examined and the platform, the object learns how to hold their breath and cooperate with the examination during the scanning process to avoid motion artifacts. During the preliminary preparation process, the platform can also analyze the facial expressions and body postures of the object to be examined through the camera, and combine the emotional health data in their medical records to judge the tension level of the object to be examined and adjust the interaction strategy. For example, based on the answers and expression feedback of the patient, the platform adjusts the response tone and explanatory details to ensure that the information is conveyed appropriately.

[0077] On the day of the actual MRI examination, the subject to be examined communicates with the system again in the MRI scanning preparation room, that is, interacts and answers questions on the platform display interface to confirm that the pre-examination preparation work has been completed. Among them, after the subject to be examined enters the scanning room, the platform can combine multi-modal information to obtain the status of the subject to be examined and then evaluate from multiple dimensions what kind of feedback should be given to the subject to be examined; the multi-modal information can be the patient's facial features collected by the in-bore camera, the patient's motion information obtained by the millimeter-wave radar, the feature points obtained by the sky-eye camera, physiological signal information (such as electrocardiogram signals can be obtained by electrode patches, and respiratory status can be obtained by respiratory straps), voice information collected by the in-bore microphone, etc. Specifically, after the subject logs in, the platform performs identity authentication and authorization through the gateway, and conducts intelligent training on the subject through the large language model and knowledge database. Before the actual scanning starts, the platform can also automatically determine whether the subject to be examined is correctly positioned through the camera installed above the scanning bed and the pressure sensor on the bed board, and at the same time confirm again whether any metal objects enter the scanning room with the patient to prevent possible safety risks. Moreover, the platform can select the most suitable scanning protocol using the previously collected health information and scanning purpose, and automatically adjust it to optimize the image quality and reduce the scanning time, thus realizing adaptive positioning and scanning protocol selection.

[0078] During the actual scanning, the platform analyzes the scanned medical image data in real time to identify possible motion artifacts or other quality problems. If it is detected that the current image quality is lower than the preset threshold, the subject to be examined will be prompted by voice to keep still or adjust the position, and the scanning parameters can be automatically adjusted to repeat scanning the problem area, thus realizing image quality monitoring.

[0079] During the scanning interval, the platform can provide relaxing visual content or music through projection or display screen to soothe the subject to be examined, reduce tension, and enhance the experience. If the patient indicates a need for help or has other requests, the platform can understand and respond, and notify the operator for manual intervention when necessary, thus realizing the improvement of the experience for the subject to be examined.

[0080] Moreover, after the scan is completed, the platform automatically processes the images, applies advanced image recognition algorithms for preliminary diagnostic suggestions, and sends the report to the doctor for review. The platform can also guide the subject to be examined through the subsequent steps, such as providing a preview of the scan results, explaining the next process, or directly scheduling a follow-up examination. In addition, after the scan is completed, the subject to be examined can log in to the platform again to interact with the platform regarding the issues in the diagnostic report generated by the scan. In response to the questions of the subject to be examined, the platform explains the medical terms in the diagnostic report and provides corresponding treatment suggestions through large language models and knowledge databases, enabling the subject to be examined to promptly understand their scan situation and treatment plan. Further, the platform can also combine large language models with the medical history records of the subject to be examined to provide customized health advice and lifestyle adjustment suggestions; it should be noted that such recommendations are provided not only based on medical knowledge but also in combination with information such as the personal preferences and psychosocial factors of the subject to be examined obtained; the information such as the preferences of the subject to be examined can be obtained through questionnaires provided by the platform. More specifically, the platform is equipped with a dynamic questionnaire generation engine. Compared with using static questionnaire templates, this engine can use machine learning algorithms to generate personalized questionnaires in real time based on the conversation content with the subject to be examined, making the questionnaire for each subject to be examined or each magnetic resonance examination unique and avoiding the problem that the information collected by the questionnaire using a fixed questionnaire template may be inaccurate.

[0081] Throughout the process, the operator no longer needs to spend a lot of time repeating the precautions, making the communication between the subject to be examined and the operator more efficient and smooth. In another embodiment, the above platform can also utilize VR or AR technology to provide a more interactive and immersive user experience, conduct consultations through virtual reality-simulated medical scenarios, or provide augmented reality-assisted body part indications, enabling the subject to be examined to more intuitively understand the problems and the required medical actions.

[0082] This embodiment provides a medical scanning system, which includes the interaction device described in any of the above embodiments. Among them, the medical scanning system includes one or a combination of a magnetic resonance scanning system, a PET scanning system, and a CT scanning system.

[0083] This embodiment provides a control method for an interaction device, which is applied to a medical scanning system, and the interaction device includes an interaction terminal device and a processing device. Figure 4 is a flowchart of a control method for an interaction device according to an embodiment of the present application, as Figure 4 shown, this process includes the following steps:

[0084] Step S410, the interaction terminal device collects the inquiry information of the subject to be examined.

[0085] Step S420: The processing device inputs the inquiry information into a trained target language model for recognition processing and outputs a first language recognition result. The target language model is trained and generated based on the obtained medical scan knowledge database.

[0086] Step S430: The processing device retrieves from the medical scan knowledge database based on the first language recognition result and generates a target reply result for the object to be queried according to the retrieval result.

[0087] Through the above steps, the processing device inputs the inquiry information of the object to be queried collected into the target language model for recognition, retrieves the magnetic resonance knowledge database based on the recognized result, and obtains a targeted target reply result. Thus, it can efficiently and accurately adaptively reply to the inquiry information of the object to be queried regarding the magnetic resonance examination process, effectively avoiding the phenomenon that the operation error of the object to be queried affects the magnetic resonance examination, and solving the problem of low safety in medical scan examinations. At the same time, through the above method to achieve intelligent interaction with the object to be queried, it also avoids the phenomenon that the operator needs to repeatedly spend a lot of time repeating the same reply to different objects to be queried, effectively improving the interaction efficiency with the object to be queried, and further improving the medical scan examination efficiency.

[0088] This embodiment also provides a computer device, which may be a server. Figure 5 It is a structural diagram inside a computer device according to an embodiment of the present application, as Figure 5 shown. The computer device includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the target reply result. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it realizes the control method of the above-mentioned interaction device.

[0089] Those skilled in the art can understand that Figure 5 the structure shown in

[0090] It should be noted that the steps shown in the above process or the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0091] This embodiment also provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0092] Optionally, the above electronic device may further include a transmission device and an input / output device. Among them, the transmission device is connected to the above processor, and the input / output device is connected to the above processor.

[0093] Optionally, in this embodiment, the above processor may be configured to execute the following steps through a computer program:

[0094] S1, the interactive terminal device collects the inquiry information of the object to be queried.

[0095] S2, the processing device inputs the inquiry information into a trained target language model for recognition processing and outputs a first language recognition result; wherein, the target language model is trained and generated based on the obtained medical scan knowledge database.

[0096] S3, the processing device performs a retrieval process from the medical scan knowledge database based on the first language recognition result, and generates a target reply result for the object to be queried according to the retrieval result.

[0097] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation manners, and will not be repeated here.

[0098] In addition, in combination with the control method of the interactive device in the above embodiments, an embodiment of the present application can be implemented by providing a storage medium. A computer program is stored on the storage medium; when the computer program is executed by a processor, it implements any one of the control methods of the interactive device in the above embodiments.

[0099] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. Such a computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0100] Those skilled in the art should understand that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0101] The above embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. An interactive device, characterized in that, applied to a medical scanning system, the device comprising: an interactive terminal device and a processing device; the interactive terminal device for collecting inquiry information of an object to be examined; the processing device, connected to the interactive terminal device, for inputting the inquiry information into a trained target language model for recognition processing and outputting a first language recognition result; the target language model is trained based on a preset medical scanning knowledge database; the processing device is further configured to perform a first retrieval process from the medical scanning knowledge database based on the first language recognition result to obtain a first retrieval result, and generate a target reply result for the object to be examined according to the first retrieval result.

2. The interactive device according to claim 1, characterized in that, the processing device is further configured to use the target language model to perform emotion recognition processing on the inquiry information to obtain a first emotion recognition result; the processing device is further configured to perform a second retrieval process from the medical scanning knowledge database based on the first emotion recognition result to obtain a second retrieval result, and generate the target reply result according to the first retrieval result and the second retrieval result.

3. The interactive device according to claim 2, characterized in that, the device further comprises an image acquisition device; the image acquisition device for collecting face image information of the object to be examined; the processing device is further configured to perform emotion recognition processing according to the face image information to obtain a second emotion recognition result, and generate a target emotion recognition result according to the first emotion recognition result and the second emotion recognition result; the processing device is further configured to perform a third retrieval process from the medical scanning knowledge database based on the target emotion recognition result to obtain a third retrieval result, and generate the target reply result according to the first retrieval result and the third retrieval result.

4. The interactive device according to claim 1, characterized in that, the interactive terminal device is further configured to collect query information of the object to be examined; the processing device is further configured to input the query information into a pre-stage language model in the target language model for recognition processing and output a second language recognition result; the processing device is further configured to, in response to detecting the second language recognition result, input the obtained multimodal medical scanning information into a post-stage sub-model in the target language model for information parsing processing to obtain a parsing result for the multimodal medical scanning information; wherein, the pre-stage language model is connected to the post-stage sub-model; the processing device is further configured to generate a diagnostic reply result for the object to be examined based on the parsing result.

5. The interactive device according to claim 1, characterized in that, the interactive terminal device is further configured to obtain login information of the object to be examined; the processing device includes an encryption unit; the encryption unit is configured to determine encryption information of the object to be examined according to the login information, and perform desensitization processing on the target reply result according to the encryption information to obtain a first reply result; and / or, The encryption unit of the processing device is used to obtain function license information for the medical scanning system, and filter the target reply result according to the function license information to obtain a second reply result.

6. The interactive device according to claim 5, wherein, the encryption unit is further used to obtain the source information of the medical scanning knowledge database, and publish the target language model to the medical institution device corresponding to the source information.

7. The interactive device according to claim 1, wherein, the processing device includes an authentication unit and an authorization unit; the authentication unit is used to perform gateway authentication according to the login information of the object to be queried, and obtain a first authentication result corresponding to the object to be queried; the authorization unit is used to receive the query information of the object to be queried when the first authentication result indicates that the identity of the object to be queried passes; and / or, the authentication unit is further used to perform gateway authentication according to the login information of the operator, and obtain a second authentication result corresponding to the operator; the authorization unit is further used to determine the authorization result of the operator based on the second authentication result.

8. The interactive device according to any one of claims 1 to 7, wherein, the processing device further includes a training unit; the training unit is further used to obtain preset medical scanning knowledge data in the medical scanning knowledge database, and annotation information carried by the preset medical scanning knowledge data, and perform prompt engineering processing on the initial language model according to the preset medical scanning knowledge data and the annotation information to obtain the target language model; and / or, the training unit is further used to update the medical scanning knowledge database at least according to the first language recognition result to obtain a new medical scanning knowledge database.

9. A medical scanning system, wherein, the system includes the interactive device according to any one of claims 1 to 8.

10. A control method for an interactive device, wherein, the interactive device is applied to a medical scanning system, and the device includes an interactive terminal device and a processing device; the method includes: the interactive terminal device collects query information of the object to be queried; the processing device inputs the query information into a trained target language model for recognition processing, and outputs a first language recognition result; wherein, the target language model is trained and generated based on the obtained medical scanning knowledge database; the processing device performs a retrieval process in the medical scanning knowledge database based on the first language recognition result, and generates a target reply result for the object to be queried according to the retrieval result.

Images