Medical device control method and system

Abstract

The embodiment of the application provides a medical equipment control method and system, the method comprises the following steps: obtaining a user intention instruction; generating an ordered operation task sequence for realizing a target intention according to the user intention instruction; and converting the ordered operation task sequence into an operation instruction sequence of an input device of the medical equipment.

Description

Technical Field

[0001] This application relates to the field of information technology, and in particular to a medical device control method and system. Background Technology

[0002] With the continuous development of medical imaging technology, equipment such as computed tomography (CT) and magnetic resonance imaging (MR) are increasingly used in clinical diagnosis. However, current medical equipment control mainly relies on manual operation near the control panel, where operators complete scanning protocol settings, parameter configurations, and task execution through complex graphical interfaces. This method is not only inefficient but also requires a high level of professional skill and operational proficiency from the operators.

[0003] Especially in scenarios like telemedicine, operators and equipment may be in different locations, making direct control of the equipment difficult. Furthermore, during operation, doctors or technicians may face issues such as unclear knowledge, ambiguous operating procedures, or insufficient ability to handle equipment errors, further impacting the ease of use and safety of the equipment. When operators encounter problems during equipment operation, they often need to spend a significant amount of time consulting operation manuals or professional personnel to find solutions, which seriously affects the efficiency of equipment use. Summary of the Invention

[0004] To address at least one of the aforementioned problems, embodiments of this application provide a medical device control method and system. This enables automatic control of the medical device from user input commands to the execution of operations, significantly improving the intelligence, remote control, and automation level of medical device operation.

[0005] According to one aspect of the embodiments of this application, a medical device control method is provided, comprising:

[0006] Obtain user intent commands;

[0007] Generate an ordered sequence of operation tasks to achieve the target intent based on the user intent instruction;

[0008] The ordered sequence of operation tasks is converted into a sequence of operation instructions for the input devices of the medical equipment.

[0009] According to one aspect of the embodiments of this application, a medical device control system is provided, comprising:

[0010] The acquisition unit acquires the user's intent command;

[0011] The generation unit generates an ordered sequence of operation tasks to achieve the target intent based on the user intent instruction;

[0012] The conversion unit converts the ordered sequence of operation tasks into a sequence of operation instructions for the input device of the medical device.

[0013] Specific embodiments of this application are disclosed in detail with reference to the following description and accompanying drawings, indicating how the principles of this application can be adopted. It should be understood that the embodiments of this application are not limited in scope. Within the scope of the appended claims, embodiments of this application include many changes, modifications, and equivalents.

[0014] Features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, combined with features in other embodiments, or substituted for features in other embodiments. Attached Figure Description

[0015] The accompanying drawings, which form part of the specification, are used to provide a further understanding of the embodiments of this application and illustrate the implementation methods of this application, together with the textual description, to explain the principles of this application. Obviously, the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other implementation methods based on these drawings without creative effort. In the drawings:

[0016] Figure 1 This is a schematic diagram of a medical device control method according to an embodiment of this application;

[0017] Figure 2 This is a schematic diagram illustrating step 102 of an embodiment of this application;

[0018] Figure 3 This is a schematic diagram of the state machine model according to an embodiment of this application;

[0019] Figure 4 This is a flowchart of step 102 in an embodiment of this application;

[0020] Figure 5 This is a flowchart illustrating the sequence of operation instructions for the input device of a medical device generated according to an embodiment of this application;

[0021] Figure 6 This is a schematic diagram of a medical device control system according to an embodiment of this application;

[0022] Figure 7 This is a schematic diagram of a medical device control system according to an embodiment of this application;

[0023] Figure 8 This is a schematic diagram of an electronic device according to an embodiment of this application;

[0024] Figure 9 This is a schematic diagram of the user interface of the interactive module in an embodiment of this application. Detailed Implementation

[0025] Referring to the accompanying drawings, the foregoing and other features of the embodiments of this application will become apparent from the following description. Specific embodiments of this application are specifically disclosed in the description and drawings, illustrating partial implementations in which the principles of the embodiments of this application can be adopted. It should be understood that this application is not limited to the described embodiments; rather, the embodiments of this application include all modifications, variations, and equivalents falling within the scope of the appended claims.

[0026] In the embodiments of this application, the terms "first," "second," etc., are used to distinguish different elements by name, but do not indicate the spatial arrangement or temporal order of these elements, and these elements should not be limited by these terms. The term "and / or" includes any one or more of the terms listed in association and all combinations thereof. The terms "comprising," "including," "having," etc., refer to the presence of the stated features, elements, components, or assemblies, but do not exclude the presence or addition of one or more other features, elements, components, or assemblies. In the embodiments of this application, terms such as "connected," "linked," and "coupled" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0027] In the embodiments of this application, the singular forms "a," "the," etc., including the plural forms, should be broadly understood as "a kind" or "a class" rather than limited to the meaning of "an." Furthermore, the term "the" should be understood to include both the singular and plural forms, unless the context explicitly indicates otherwise. Additionally, the term "according to" should be understood as "at least partially based on…," and the term "based on" should be understood as "at least partially based on…," unless the context explicitly indicates otherwise.

[0028] Features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, combined with features in other embodiments, or substituted for features in other embodiments. The term "comprising / including" as used herein means the presence of a feature, integral, step, or component, but does not exclude the presence or addition of one or more other features, integrals, steps, or components.

[0029] This application provides a method for controlling medical devices.

[0030] Figure 1 This is a schematic diagram of a medical device control method according to an embodiment of this application, such as... Figure 1 As shown, the method includes:

[0031] 101, Obtain user intent commands;

[0032] 102. Generate an ordered sequence of operation tasks to achieve the target intent based on the user intent instruction;

[0033] 103. The ordered sequence of operation tasks is converted into a sequence of operation instructions for the input device of the medical device.

[0034] The medical devices described in this application primarily refer to devices used for medical examination, diagnosis, or treatment, including but not limited to computed tomography (CT) equipment, magnetic resonance imaging (MRI) equipment, C-arm imaging equipment, positron emission tomography (PET) equipment, single-photon emission computed tomography (SPECT) equipment, ultrasound equipment, X-ray imaging equipment, or any other suitable medical devices. The medical devices in this application have a graphical user interface (or control interface) for performing operations such as image acquisition, protocol setting, parameter adjustment, task management, and result browsing.

[0035] The medical device in this application embodiment can perform human-computer interaction through an input device, which can be a keyboard, mouse, touchpad, touch screen, trackball, function keys, or other interactive components. The function of the input device is to receive operation signals input by the user and transmit them to the control module of the medical device, thereby realizing the issuance and execution of operation commands.

[0036] In existing medical device operation procedures, operators need to complete a series of operations through the aforementioned input devices, such as selecting a scanning protocol, inputting or adjusting scanning parameters (e.g., slice thickness, voltage, current), browsing or confirming the work list, and responding to device prompts or error messages. However, this method, relying on manual step-by-step operations, is inefficient in scenarios such as remote operation and requires a high level of professional skill and operational proficiency from the operator. Therefore, this application proposes a medical device control method that can generate an ordered sequence of operation tasks to achieve the target intent based on the acquired user intent instructions, and convert the ordered sequence of operation tasks into a sequence of operation instructions for the medical device's input devices, so that the medical device executes the corresponding sequence of operation instructions. This enables automatic control from user input instructions to medical device execution, significantly improving the intelligence, remoteness, and automation level of medical device operation. The following describes this medical device control method.

[0037] In some embodiments, at 101, a user intent instruction is obtained. This user intent instruction refers to the request content expressed by the user when they want the medical device to perform a certain operation or obtain certain information. The intent instruction may include at least control instructions or question instructions. Control instructions may be used to directly express a specific operation to be performed, such as "Please select an abdominal scan protocol," or "Reset scan parameters - slice thickness 2.5mm." Question instructions are used to express questions or needs based on the current device status, such as "An error occurred on the scan interface, what should I do?", "It is necessary to assess the patient's cerebral vascular malformation, please select an appropriate protocol," or "When performing an abdominal CT scan on a patient with metal implants, how can metal artifacts be reduced?" Examples are not provided here.

[0038] In some embodiments, users can obtain intent commands through interaction modules deployed on devices such as mobile apps, web clients, and voice assistant terminals. These intent commands can be expressed in media formats such as text, voice, and images. For example, users can input intent commands (text) via a keyboard, or input intent commands (voice) via a microphone, or input intent commands (images) by uploading an image or capturing a screenshot of the screen using a camera. Examples of these methods will not be listed here.

[0039] In some embodiments, after obtaining the user's input intent command, the user intent command can be converted into a natural language text string. For example, when the intent command is speech or image, it can be converted into a processable natural language text string using Automatic Speech Recognition (ASR) or Optical Character Recognition (OCR) technologies.

[0040] In some embodiments, after obtaining the user's intent instruction, the intent instruction can be parsed, understood, and converted into a sequence of operation instructions that can be executed by the medical device. In step 102, an ordered sequence of operation tasks for achieving the target intent is first generated based on the user's intent instruction. In step 103, the ordered sequence of operation tasks is then converted into a sequence of operation instructions for the input device of the medical device. This will be described in detail below.

[0041] Figure 2 This is a schematic diagram of the implementation method of step 102 in the embodiment of this application, as shown below. Figure 2 As shown, step 102 includes:

[0042] 201. Search for a first function in the preset first function library that matches the user intent instruction;

[0043] 202. Call the first matching function to obtain the intent instruction parsing result;

[0044] 203. Generate the ordered operation task sequence based on the intent instruction parsing result.

[0045] In some embodiments, a first function library can be pre-configured, containing various task classification functions related to medical device control to cover common user intent scenarios during operation. This first function library includes, but is not limited to, at least one of the following categories (names are for illustrative purposes only and not as limitations): scan protocol query function (query_protocol), patient-based scan parameter optimization function (scan_param_optimization_specific_patient), image-based scan parameter optimization function (scan_param_optimization_image_quality), artifact reduction function (artifact_reduction), device operation query function (console_warning), and manual query function (manual_query). The definitions of these functions include function names, descriptions, and input parameter requirements, and can be defined in JSON format; however, this embodiment does not limit this definition. Examples of various first function definitions can be found in Table 1 below:

[0046] Table 1

[0047]

[0048]

[0049] The above input parameters can be defined as shown in Table 2 below:

[0050] Table 2

[0051]

[0052] The definitions of the various first function and parameters above are merely examples, and the embodiments of this application are not intended to limit them. They will not be listed here one by one.

[0053] In some embodiments, in step 201, after the user's intent instruction is converted into a natural language text string, a Large Language Model (LLM) can be used to identify the user's operational intent and match the most relevant first function in a preset first function library. That is, after receiving the input natural language text string, the Large Language Model parses its semantics, and the LLM searches for a matching first function in the aforementioned first function library (input to the LLM as part of the prompt message). For example, for the natural language text string "Please query the scanning protocol for cerebrovascular malformations," the LLM will match the scanning protocol query function (query_protocol); for the natural language text string "How to reduce metal artifacts during abdominal CT scans of patients with metal implants," the LLM will match the artifact reduction function (artifact_reduction). The above LLM technology is merely illustrative and is not intended to limit the scope of this application.

[0054] In some embodiments, after matching the most relevant first function, in order to correctly execute the call to the first function, the parameters required to call the first function can also be extracted from the user intent command. The definitions of each parameter (e.g., Table 2) can be input into the LLM as part of a prompt. For example, if the LLM matches the scan protocol query function (query_protocol), it will also extract the necessary parameters required for the first function from the user intent command "Please query the scan protocol for cerebral vascular malformations": examination site "brain", disease name "vascular malformation"; or, if the LLM matches the artifact reduction function (artifact_reduction), it will also extract the necessary parameters required for the first function from the user intent command "How to reduce metal artifacts when performing an abdominal CT scan on a patient with a metal implant": artifact type "metal", examination site "abdomen".

[0055] In some embodiments, when it is impossible to extract all the necessary parameter data associated with the matching first functional function, or when it detects that the necessary parameters associated with the matching first functional function are missing, the user can be prompted to supplement the associated parameter data. The newly supplemented parameter data and the original intent command are then encapsulated and submitted to LLM analysis again until all necessary parameters are complete. For example, for the natural language text string "Please query the scanning protocol for vascular malformations", LLM will match the scanning protocol query function (query_protocol) and extract the parameter disease name "vascular malformation". However, LLM cannot extract the parameter for the examination site. At this time, the user can be prompted through the aforementioned interactive module: "Please tell me the site that needs to be examined", so that the user can input the examination site "brain" again through the interactive module. When the user supplements the examination site with "brain", this information will be re-encapsulated with the original intent command and sent to LLM. In this case, LLM determines that the parameters required by the scanning protocol query function (query_protocol) are complete. This mechanism ensures the completeness of the user's intent and the accuracy of the function call, avoiding system misunderstanding due to insufficient input.

[0056] In some embodiments, after matching a first function and extracting all necessary parameters, in step 202, the matched first function is invoked to obtain the intent instruction parsing result. This includes: retrieving information associated with the extracted parameter data from a pre-established knowledge base, using the retrieved information as input to a large language model, and generating the intent instruction parsing result based on the output of the large language model. This is described in detail below.

[0057] In some embodiments, a knowledge base can be pre-established to provide useful contextual information and knowledge support for subsequent instruction parsing. For example, the knowledge base may include at least one of the following: a scanning protocol knowledge base, an equipment manual knowledge base, and an imaging parameter knowledge base. These knowledge bases can be constructed based on standard scanning protocol documents used internally by the hospital, industry-recommended guidelines, scanning protocol documents provided by manufacturers, professional information in the field of medical imaging, user manuals, technical specifications, error code tables, training materials, FAQ documents, equipment parameter recommendation documents provided by manufacturers, operation manuals and experience summaries from hospitals or radiology departments, and documents from medical-related professional conferences / papers, etc. First, the original documents are collected, the text is extracted, and then structured according to a predetermined format. Natural language processing functions are used for segmentation and cleaning to generate clean text. The text is then vectorized (e.g., using an embedding model) and stored in a vector database to support subsequent retrieval. Specific processing procedures can be found in relevant technologies and will not be elaborated here.

[0058] In some embodiments, LLM can also combine Retrieval Augmented Generation (RAG) technology to retrieve and extract information related to parameter data to obtain intent command parsing results. When determining the first matching function to be called, the user intent command, the name of the first function to be called, and the extracted parameters can be used as query content. The query content is vectorized (using the same vectorization method as described above) to generate a query vector. The similarity between the query vector and all vectors in the vector database (knowledge base) is calculated, and the N (N is a positive integer) vectors with the closest semantics are found as context information. The retrieved N vectors and the query vector are combined to form a complete prompt message, which is input into LLM for semantic reasoning to obtain intent command parsing results. The following three examples illustrate this.

[0059] Example 1:

[0060] User intent command (problem command): The patient wants to undergo a cerebral vascular malformation examination. How should the scanning protocol be selected?

[0061] The first function for matching: the query protocol function `query_protocol(parameter disease="vascular malformation", parameter bodypart="head")`

[0062] Vector matching was performed in the scanning protocol knowledge base, and the following relevant knowledge was retrieved:

[0063] The Brain Additional Sequence protocol is often used for cerebral vascular malformations.

[0064] We recommend using the head and neck protocol combination scheme under the XX library.

[0065] The retrieved knowledge, the name of the first function, and the extracted parameters are input into LLM again, and the intent resolution result is: Select Brain Vascular and Brain Additional Sequence protocol packets from the XX library, and check the header.

[0066] Example 2:

[0067] User intent command (problem command): How to reduce metal artifacts when performing an abdominal CT scan on a patient with a metal implant?

[0068] The first matching function: artifact reduction function (artifact_reduction) (parameters artifact type="metal", parameter bodypart="abdomen")

[0069] Vector matching is performed in the imaging parameter knowledge base. The matching result is combined with the first function name and the extracted parameters and then input into LLM again to obtain the intent parsing result: It is recommended to adjust the layer thickness to 2.5mm and the tube voltage to 140kVp (these values ​​are only examples and are not actual recommended values).

[0070] Example 3:

[0071] User intent command (problem command): An E120 error appears in the console. How do I fix it?

[0072] The first matching function: Device operation query function (console_warning) (parameter warning_message = "E120")

[0073] Vector matching in the equipment manual knowledge base yielded the following relevant information:

[0074] Error code E120 typically indicates a data stream interruption, commonly seen in overload exceptions during scanning.

[0075] Suggested steps: 1) Check the data link status; 2) Close and restart the console software; 3) Log and report.

[0076] Inputting the retrieved knowledge, the name of the first function, and the extracted parameters back into LLM, the intent resolution result is: Data flow interrupted. It is recommended to check the data link, restart the console, log the information, and contact engineering support.

[0077] The first function called above can be retrieved from one or more corresponding knowledge bases, and this application embodiment does not limit it.

[0078] The above method for parsing user intent commands combines LLM and RAG techniques, but this application does not limit it to these methods. For example, keyword matching, support vector machine (SVM), traditional classification, neural networks, and other methods can also be used, which will not be listed here.

[0079] In some embodiments, at step 203, after obtaining the intent instruction parsing result, an ordered sequence of operation tasks for achieving the target intent can be generated based on the parsing result. This ordered sequence of operation tasks describes each basic operation task required to sequentially complete the target intent, starting from the current state of the medical device interface. Generating the ordered sequence of operation tasks for achieving the target intent based on the parsing result includes: searching for a second function in a preset second function library that matches the intent instruction parsing result; determining a state machine model associated with the matched second function; and generating the ordered sequence of operation tasks based on the state machine model.

[0080] In some embodiments, a second function library can be pre-configured, where each second function represents an operational objective of the medical device. For example, the second function library may include at least one of the following: a protocol selection operation function, a parameter adjustment operation function, a fault handling operation function, etc. The definitions of these functions include function names, descriptions, and input parameter requirements, and can be defined using JSON format; however, this embodiment does not limit this definition. The definitions of various second function types can be found in Table 3 below:

[0081] Table 3

[0082]

[0083] In some embodiments, a second function matching the intent command parsing result is searched in a preset second function library. For example, if the parsing result contains a protocol name and location information, the matching second function is the protocol selection operation function `choose_protocol`; if the parsing result contains scan parameter names and parameter values, the matching second function is the parameter adjustment operation function `modify_scan_parameters`. After determining the matching second function, the parameters required to call the second function are extracted from the intent command parsing result. The above matching process can also be implemented by an LLM, but this embodiment is not intended to limit it, and will not be exemplified here.

[0084] For the aforementioned Example 1, the second matching function is the selection protocol operation function, with the following parameters: "pkg": "Brain Vascular", "bodypart": "HEAD"; "pkg": "Brain Additional Sequence", "bodypart": "HEAD";

[0085] For example two above, the matched second function is a parameter adjustment operation function with the following parameters: "param": "slice thickness", "target": "2.5mm"; "param": "kVp", "target": "140".

[0086] In some embodiments, when a second function cannot be matched, the aforementioned interactive module can prompt the user that the intent command they entered cannot be recognized and executed, and ask them to re-enter it.

[0087] In some embodiments, after determining the matching second functional function, a state machine model associated with the matching second functional function is determined. Each second functional function predefines one or more associated state machine models to describe the various interface state nodes, operation task nodes, and execution paths required to achieve the operation objective corresponding to the second functional function. The state machine model may consist of multiple ordered nodes, each node corresponding to an operation task. Each operation task represents a user interface interaction behavior, including: clicking (selecting, confirming, or checking) a button or menu, inputting data, deleting data, etc. Each operation task may or may not contain parameters to be filled.

[0088] In some embodiments, after determining the associated state machine model, the extracted parameter data is bound to the operation task corresponding to the state machine model. For example, if an operation task node is click_pkg(pkg_name), pkg_name can be replaced with a specific protocol name based on the protocol field in the intent instruction parsing result, such as click_pkg("BrainVascular") or click_pkg("Brain Additional Sequence").

[0089] In some embodiments, after binding parameters, generating the ordered operation task sequence based on the state machine model includes: starting from the starting node of the associated state machine model, sequentially visiting each operation task node (replacing parameters) along the state transition path to generate the ordered operation task sequence. The ordered operation task sequence records information such as task execution order, dependencies, and operation tasks (operation steps).

[0090] In some embodiments, the state machine supports parameter traversal processing logic. For example, when the parsing result contains multiple protocols, multiple selection tasks can be generated cyclically in the choose_proto_package node.

[0091] Taking the second matching function, choose_protocol, as an example, its target intent is to complete the selection operation of the protocol described in the user intent parsing result. Figure 3 This is a schematic diagram of the state machine model associated with the second function choose_protocol in the embodiments of this application, as shown below. Figure 3As shown, each triangular node represents an operation task, and each elliptical node represents the interactive interface state after executing the operation task (or the output state of the operation task). The state transition path of this state machine includes: enter_proto_selector (enter the protocol selection interface) → choose_proto_bodypart (select the inspection part) → choose_proto_package (check the protocol package) → user_confirm_proto_candidate (user confirms the candidate protocol) → accept_proto (accept the protocol and return) / abort_mission (abort the task). Combining the parameters bodypart="HEAD", pkg="Brain Vascular", and pkg="Brain Additional Sequence", the ordered operation task sequence generated for Example 1 includes: 1. Enter the protocol selection page; 2. Select the inspection part HEAD; 3. Check the protocol packages Brain Vascular and Brain Additional Sequence; 4. Click "Confirm"; 5. User accepts.

[0092] The second function and state machine model described in the embodiments of this application are merely illustrative examples and are not intended to limit the scope of this application.

[0093] Figure 4 This is a flowchart of step 102 in an embodiment of this application, as follows: Figure 4 As shown, step 102 includes:

[0094] 401. Obtain the user intent command, encapsulate the user intent command and the contents of the first function library into a prompt message and input it into the LLM;

[0095] 402. The LLM searches for a first function that matches the user intent instruction in the first function library. When a first function is matched, it extracts the necessary parameters from the first function from the user intent instruction.

[0096] 403 checks if a necessary parameter is missing. If it is missing, execute 404; otherwise, execute 405.

[0097] 404 prompts the user to re-enter the missing parameters; after receiving the user's supplementary input parameters, it returns to 401, and encapsulates the supplementary input parameters, the aforementioned user intent command, and the contents of the first function library into a prompt message and inputs it into the LLM;

[0098] 405, invoke the first function parameter, encapsulate the successful invocation result into the aforementioned prompt and input it into the LLM;

[0099] 406, LLM outputs the intent instruction parsing result;

[0100] 407. The intent command parsing result and the contents of the second function library are encapsulated into a prompt message and input into the LLM;

[0101] 408. The LLM searches for a second function in the second function library that matches the result of the intent instruction parsing. If a second function is matched, execute 409-410; otherwise, execute 411.

[0102] 409. LLM extracts the necessary parameters from the second function from the intent instruction parsing result;

[0103] 410. Call the second function to determine the state machine model associated with the matched second function, and generate the ordered operation task sequence according to the state machine model.

[0104] 411, The prompt interaction module displays a prompt to the user that the intended command entered by the user cannot be recognized and executed, please re-enter.

[0105] The above steps 401, 404-405, 407, 410-411 can be executed by the processor on the server. The LLM can be deployed on the server, or on other devices connected to the server or in the cloud. This application embodiment is not intended to limit it. The specific deployment method will be described in the following embodiments.

[0106] In some embodiments, in step 103, after generating an ordered sequence of operation tasks to achieve the target intent, the task sequence can be converted into a series of operation instructions (ordered operation instruction sequence) for the medical device to execute, driving the medical device to complete the target intent. These input device operation instructions control the specific interactive actions of the medical device input device on the user's graphical interface. The operation instructions include operation instruction actions and operation instruction objects. For example, mouse operation instructions include hover, click, double-click (dblclick), mousedown, mouseup, and mousemove, etc., and operation instruction objects may include the horizontal coordinate (screenX) of the mouse pointer on the screen, the vertical coordinate (screenY) of the mouse pointer on the screen, the horizontal coordinate (clientX) of the mouse pointer in the visible area of ​​the current browser window, and the horizontal coordinate (clientY) of the mouse pointer in the visible area of ​​the current browser window, etc. Keyboard operation instructions may include: keydown, keypress, etc., and operation instruction objects include key codes (keyCode) or input strings composed of key codes. Examples are not provided here.

[0107] The above-mentioned operation instructions can all be completed through the device's input device. However, in actual execution, the operation instructions are not executed by the operator controlling the input device according to the operation instructions, but are automatically executed by the medical device's control module.

[0108] In some embodiments, converting the ordered operation task sequence into an operation instruction sequence for the input device of a medical device includes: determining the coordinates of an operation position associated with the ordered operation task sequence based on the control interface (graphical user interface) of the medical device; and generating the operation instruction sequence based on the coordinates of the operation position and the ordered operation task sequence. The operation position can be an element or control, including an icon or text, as described in detail below.

[0109] In some embodiments, to support the above-described coordinate positioning operation, a mapping table of interface element (control) positions can be pre-established. This mapping table includes the size, icon image (if it is an icon), whether it is a fixed element, and the position coordinates of the fixed element (if it is a fixed element) for various elements in the medical device control interface. When an operation task in an ordered sequence of tasks matches an element (control) in the interface element mapping table, the position coordinates of that element (control) can be read from the mapping table. The above coordinates can be the element's center coordinates or other position coordinates, and this embodiment of the application does not limit them.

[0110] In some embodiments, when the position of the element matched by the operation task is not a fixed coordinate (e.g., its position changes depending on the device model, resolution, or system state), the method may further include: acquiring an image of the current control interface of the medical device; and identifying (e.g., template matching, OCR recognition, target detection, etc.) the coordinates of the operation position (matched element) in the image.

[0111] In some embodiments, after determining the coordinates of the operation position (element / control) matching the operation task, the coordinates are bound to the corresponding operation instructions to generate an operation instruction sequence. Each operation task in the ordered operation task sequence corresponds to one or a series of operation instructions. For each operation task sequentially, the set of operation instructions corresponding to the operation task can be determined in a mapping table. Combining the aforementioned coordinates, the parameters of the operation task, and the determined set of operation instructions, a complete operation instruction sequence is generated.

[0112] In some embodiments, a mapping table between various operation tasks and their corresponding sets of operation instructions can be pre-defined. For example, if the operation task is to click a control, the corresponding set of operation instructions includes {mouse: move [parameter: coordinates] -> press -> release}. If the operation task is to input text, the corresponding set of operation instructions includes {mouse move [parameter: coordinates] -> mouse press -> keyboard press [parameter: string]}. If the operation task is to drag a slider, the corresponding set of operation instructions includes {mouse move [parameter: starting coordinates] -> mouse press -> mouse move [parameter: ending coordinates] -> mouse release}. Examples of these will not be provided here.

[0113] For example, for the operation task "Select protocol package (choose_proto_package) Brain Vascular" in an ordered operation task sequence, the position coordinates (100, 305) of the Brain Vascular protocol checkbox control can be determined in the control interface, and the following operation instruction sequence can be generated: 1. Move the mouse to the specified coordinates MOVE MOUSE TO POS(100, 305); 2. Press the left mouse button PRESS LEFT MOUSE; 3. Release the left mouse button RELEASE LEFT MOUSE.

[0114] For example, if the position coordinates of the Brain Vascular checkbox control are not fixed, then after acquiring the interface image, the position coordinates of the checkbox control containing the text "Brain Vascular" are identified, and then the above operation instruction sequence is generated.

[0115] The above operation instructions are merely illustrative examples and are not intended to limit the scope of this application. The specific implementation details are related to the graphical user interface design of each medical device and are not intended to limit the scope of this application.

[0116] Figure 5 This is a flowchart illustrating the sequence of operation instructions for the input device of a medical device generated according to an embodiment of this application, such as... Figure 5 As shown, for an ordered sequence of operation tasks, including J operation tasks, the method includes:

[0117] 501. For the i-th operation task, determine whether the position of the element matching the operation task is fixed. If it is fixed, execute 502; otherwise, execute 503. The initial value of i is 1.

[0118] 502, Determine the coordinates of the element's position from the position mapping table;

[0119] 503, Obtain the current control interface image, and identify the coordinates of the element's position in the image;

[0120] 504. Based on the coordinates and the task, a series of operation instructions are generated.

[0121] 505. Check if i is equal to J. If i is equal to J, end the operation; otherwise, i = i + 1 and return 501.

[0122] The series of operation instructions corresponding to J operation tasks are sequentially combined to form an operation instruction sequence that achieves the target intention.

[0123] In some embodiments, the method may further include: sending the sequence of operation instructions obtained in step 103 to a medical device, so that the medical device executes the sequence of operation instructions. The above-mentioned sequence of operation instructions is non-invasively controlled by input signals from an analog input device; specific implementation details can be found in the embodiments described later.

[0124] The library in the above embodiments can be replaced by a lookup table (LUT) or a pre-trained neural network model, which can be stored in the memory of the corresponding module or in a separate memory connected to the module. This application embodiment does not limit it in this way.

[0125] The above embodiments are merely illustrative examples of the embodiments of this application, but this application is not limited thereto, and appropriate modifications can be made based on the above embodiments. The above embodiments can be used individually, or one or more of the above embodiments can be combined. For example, 101-102 or 103 can be executed individually, and the embodiments of this application are not intended to be limited thereto.

[0126] Figure 9 This is a schematic diagram of the user interface of the user interaction module in an embodiment of this application, such as... Figure 9 As shown, the user-input intent command 901 can be displayed on the user interface 900, along with the generated ordered sequence of operation tasks 902, and the completion progress indicator 903 for each operation task (e.g., different indicators are displayed for tasks not yet performed, completed, and in progress). Optionally, the user can be prompted for confirmation 904. For example, for the operation task "4. Click 'Confirm'" in the aforementioned example, select the button YES (Accept) or NO (Abort). Optionally (not shown), when necessary parameter data is missing, the user interface 900 can also display a prompt for the user to supplement the missing necessary parameters, and display the supplemented necessary parameters. This is only an illustrative example, and the embodiments of this application are not intended to be limiting.

[0127] One of the beneficial effects of this application's embodiments is that it can generate an ordered sequence of operation tasks to achieve the target intent based on the acquired user intent command, and convert the ordered sequence of operation tasks into an operation command sequence for the medical device's input device, so that the medical device executes the corresponding operation command sequence. Therefore, it can realize automatic control from user input commands to medical device execution, significantly improving the intelligence, remote control, and automation level of medical device operation.

[0128] Furthermore, unlike existing technologies that embed neural network models within medical devices, the embodiments of this application do not require modification of the medical device itself. Moreover, the method in step 102 can be reused for different medical devices; only specific modifications to the control interface of different medical devices are needed to generate the input device's operation instruction sequence, greatly improving the system's reusability. This facilitates cross-vendor and cross-system deployment.

[0129] In addition, existing technologies require embedding the intention command input terminal into the medical device, which requires the user and the medical device to be in the same space and close to each other. The embodiments of this application eliminate the physical space limitation between the user and the device.

[0130] In addition, this application can automatically parse user intent commands by combining RAG and LLM technologies, thereby improving understanding accuracy and semantic matching capabilities.

[0131] In addition, introducing a state machine model to transform intent parsing instructions into a sequence of operation tasks can enhance the system's versatility and reusability.

[0132] In addition, by combining element mapping tables with image recognition to determine element positions and then converting them into operation instructions, the reliability of execution on different devices and interfaces can be guaranteed.

[0133] This application also provides a medical device control system, which corresponds to the medical device control method in the foregoing embodiments.

[0134] Figure 6 This is a schematic diagram of a medical device control system according to an embodiment of this application, such as... Figure 6 As shown, the medical device control system 600 includes:

[0135] Acquisition unit 601 acquires user intent commands;

[0136] The generation unit 602 generates an ordered sequence of operation tasks to achieve the target intent based on the user intent instruction;

[0137] The conversion unit 603 converts the ordered sequence of operation tasks into a sequence of operation instructions for the input device of the medical device.

[0138] The implementation methods of the acquisition unit 601, the generation unit 602, and the conversion unit 603 can be found in 101 to 103, and will not be repeated here.

[0139] In some embodiments, the functional units may be deployed in the same electronic device or on different electronic devices, depending on deployment requirements.

[0140] In some embodiments, the acquisition unit 601 can be deployed in an electronic device closest to the user, such as a voice assistant, mobile terminal, or network platform. The acquisition unit 601 and the generation unit 602 can communicate via wired or wireless network, with the acquisition unit 601 sending the user's input intent command to the generation unit 602. Alternatively, the acquisition unit 601 and the generation unit can be deployed in the same electronic device, communicating with the user interaction device via wired or wireless network to acquire the user's input intent command.

[0141] In some embodiments, the generation unit 602 may be deployed on a local or cloud server, including a memory that stores the aforementioned function library, knowledge base, state machine model, etc. It may be embedded in an LLM model or may be connected to an external LLM model. This application embodiment is not intended to limit it.

[0142] In some embodiments, the conversion unit 603 can be deployed on an edge server in the hospital where the medical device is located, communicating with the generation unit 602 via wired or wireless network. The conversion unit 603 can connect to the medical device, for example, via a Universal Serial Bus (USB) interface (not limited to this), to send simulated input device operation command sequences to the medical device. Alternatively, the conversion unit 603 can be considered a USB peripheral, simulating input device behavior according to the operation command sequence using the standard Human Interface Device (HID) protocol, with the medical device treating it as an input device through the interface. Alternatively, the conversion unit 603 can send the operation command sequence to the control module of the medical device via a communication module, so that the medical device can execute the operation command sequence. Furthermore, the conversion unit 603 can also communicate with the medical device (e.g., via a High-Definition Multimedia Interface (HDMI) interface), acquiring real-time control interface (graphical user interface) images of the medical device through a capture card, etc., but this embodiment is not intended to limit this. Alternatively, the functions of the aforementioned conversion unit 603 can be integrated into the electronic device deployed with the generation unit 602, which will not be exemplified here.

[0143] Figure 7 This is a schematic diagram of the medical device control system 600 according to an embodiment of this application, as shown below. Figure 7As shown, the control system 600 includes: an interactive client 701, a parsing server 702, and an edge server 703. The interactive client 701 acquires user intent commands, the parsing server 702 integrates the functions of a generation unit 602, and the edge server 703 integrates the functions of a conversion unit 603. The interactive client 701, parsing server 702, and edge server 703 communicate with each other via an external network, while the edge server 703 communicates with medical devices via the hospital's intranet. Specific implementation details can be found in the aforementioned embodiments and will not be repeated here.

[0144] In some embodiments, the control system 600 can be deployed on a standalone intelligent control terminal, server, or workstation. The acquisition unit 601 can be an input module for user interaction, or it can be a communication module connected to a user interaction device; however, this embodiment is not intended to limit the scope. The functions of the generation unit 602 and the conversion unit 603 can be integrated into the processor. The conversion unit 603 can send an operation instruction sequence to the medical device so that the control module of the medical device can execute the operation instruction sequence to achieve the target intent. The conversion unit 603 can also acquire real-time control interface images from the medical device. The implementation method is as described above and will not be repeated here.

[0145] The above embodiments are merely illustrative examples of embodiments of this application, but this application is not limited thereto, and appropriate modifications can be made based on the above embodiments. For example, the above embodiments can be used alone, or one or more of the above embodiments can be combined.

[0146] This application also provides an electronic device. Figure 8 This is a schematic diagram of an electronic device according to an embodiment of this application, such as... Figure 8 As shown, the electronic device 800 may include: at least one interface (not shown), a processor (e.g., a central processing unit (CPU)) 801, and a memory 802; the memory 802 is coupled to the processor 801.

[0147] In some implementations, the processor 801 may be configured to: acquire user intent instructions; and generate an ordered sequence of operation tasks to achieve the target intent based on the user intent instructions.

[0148] In some embodiments, the processor 801 may be configured to: acquire an ordered sequence of operation tasks for achieving a target intent; and convert the ordered sequence of operation tasks into a sequence of operation instructions for the input device of the medical device.

[0149] The above implementation methods can be carried out individually or in combination with the embodiments, and the embodiments of this application are not intended to limit the scope of implementation.

[0150] It is worth noting that the electronic device 800 may also include Figure 8 For components not shown, please refer to existing technologies.

[0151] In the embodiments of this application, the processor 801 is sometimes also referred to as a controller or operation control, and may include a microprocessor or other processor device and / or logic device.

[0152] In some embodiments, memory 802 may store various types of data; it may also store medical device control program 803, real-time data, and pre-set libraries or tables, or pre-trained neural network models, and execute program 803 under the control of processor 801.

[0153] In some embodiments, the electronic device may be, for example, a personal computer, a server, a cloud computer, a workstation, a laptop computer, a mobile terminal, etc.; however, the embodiments of this application are not limited thereto.

[0154] This application also provides a computer-readable program, wherein when the program is executed in an electronic device, the program causes the electronic device to acquire a user intent instruction; and generates an ordered sequence of operation tasks to achieve the target intent based on the user intent instruction.

[0155] This application also provides a storage medium storing a computer-readable program, wherein the computer-readable program enables a computer to acquire user intent instructions and generate an ordered sequence of operation tasks to achieve the target intent based on the user intent instructions.

[0156] This application also provides a computer-readable program, wherein when the program is executed in an electronic device, the program causes the electronic device to acquire an ordered sequence of operation tasks for achieving a target intent; and converts the ordered sequence of operation tasks into a sequence of operation instructions for the input device of a medical device.

[0157] This application also provides a storage medium storing a computer-readable program, wherein the computer-readable program enables a computer to acquire an ordered sequence of operation tasks for achieving a target intent; and converts the ordered sequence of operation tasks into a sequence of operation instructions for an input device of a medical device.

[0158] The above implementation methods can be carried out individually or in combination with the embodiments, and the embodiments of this application are not intended to limit the scope of implementation.

[0159] The apparatus and methods described above in this application can be implemented in hardware or in combination with software. This application relates to a computer-readable program that, when executed by a logic component, enables the logic component to implement the apparatus or components described above, or to implement the various methods or steps described above. This application also relates to storage media for storing the above programs, such as hard disks, magnetic disks, optical disks, DVDs, flash memory, etc.

[0160] The methods / apparatus described in conjunction with the embodiments of this application can be directly embodied in hardware, software modules executed by a processor, or a combination of both. For example, one or more and / or combinations of one or more functional block diagrams shown in the figures can correspond to various software modules in a computer program flow, or to various hardware modules. These software modules can correspond to the various steps shown in the figures, respectively. These hardware modules can be implemented, for example, using a field-programmable gate array (FPGA) to embed these software modules.

[0161] The software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. A storage medium can be coupled to the processor, enabling the processor to read information from and write information to the storage medium; or the storage medium can be an integral part of the processor. The processor and storage medium can reside in an ASIC. The software module can be stored in the memory of a mobile terminal or in a memory card that can be inserted into the mobile terminal. For example, if the device (such as a mobile terminal) uses a high-capacity MEGA-SIM card or a high-capacity flash memory device, the software module can be stored in the MEGA-SIM card or the high-capacity flash memory device.

[0162] One or more and / or one or more combinations of functional blocks described in the accompanying drawings can be implemented as a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, or any suitable combination thereof for performing the functions described herein. One or more and / or one or more combinations of functional blocks described in the accompanying drawings can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in communication with a DSP, or any other such configuration.

[0163] The present application has been described above with reference to specific embodiments. However, those skilled in the art should understand that these descriptions are exemplary and not intended to limit the scope of protection of the present application. Those skilled in the art can make various modifications and variations to the present application based on the principles thereof, and these modifications and variations are also within the scope of the present application.

[0164] Postscript

[0165] 1. A method for controlling a medical device, characterized in that the method comprises:

[0166] Obtain user intent commands;

[0167] Generate an ordered sequence of operation tasks to achieve the target intent based on the user intent instruction.

[0168] 2. A method for controlling a medical device, characterized in that the method comprises:

[0169] Obtain an ordered sequence of operational tasks to achieve the intended goal;

[0170] The ordered sequence of operation tasks is converted into a sequence of operation instructions for the input devices of the medical equipment.

[0171] 3. An electronic device comprising a processor configured to perform the method described in Appendix 1 or 2.

Claims

1. A method for controlling a medical device, characterized in that, The method includes: Obtain user intent commands; Generate an ordered sequence of operation tasks to achieve the target intent based on the user intent instruction; The ordered sequence of operation tasks is converted into a sequence of operation instructions for the input devices of the medical equipment.

2. The method according to claim 1, wherein, The user intent instructions include question instructions or control instructions.

3. The method according to claim 1, wherein, Generating an ordered sequence of operation tasks to achieve the target intent based on the user intent instruction includes: Search for a first function in a preset first function library that matches the user's intent instruction; Call the first matching function to obtain the intent instruction parsing result; The ordered sequence of operation tasks is generated based on the result of the intent instruction parsing.

4. The method according to claim 3, wherein, The first function library includes at least one of the following functions: scan protocol query function, patient-based scan parameter optimization function, image-based scan parameter optimization function, artifact reduction function, device operation query function, and technical manual query function.

5. The method according to claim 3, wherein, Calling the first matching function to obtain the intent instruction parsing result includes: Retrieve information associated with parameter data from a pre-established knowledge base, wherein the parameter data is parameter data associated with a matching first functional function extracted from the user intent instruction; The retrieved information is used as input to a large language model, and the intent instruction parsing result is generated based on the output of the large language model.

6. The method according to claim 3, wherein, The method further includes: If the parameter data associated with the matching first function cannot be extracted, the user is prompted to input additional associated parameter data.

7. The method according to claim 5, wherein, The knowledge base includes at least one of the following: a scanning protocol knowledge base, an equipment manual knowledge base, and an imaging parameter knowledge base.

8. The method according to claim 3, wherein, Generating the ordered operation task sequence based on the intent instruction parsing result includes: Search the preset second function library for a second function that matches the result of the intent instruction parsing; Determine the state machine model associated with the matched second function, and generate the ordered sequence of operation tasks based on the state machine model.

9. The method according to claim 8, wherein, The second function library includes at least one of the following functions: protocol selection operation function, parameter adjustment operation function, and fault handling operation function.

10. The method according to claim 1, wherein, Converting the ordered sequence of operation tasks into a sequence of operation instructions for the input device of a medical device further includes: Based on the control interface of the medical device, determine the coordinates of the operation position related to the operation task in the ordered operation task sequence; The operation instruction sequence is generated based on the coordinates of the operation location and the ordered operation task sequence.

11. The method according to claim 10, wherein, When the coordinates of the operating position are not fixed, the method further includes: Acquire an image of the current control interface of the medical device; Identify the coordinates of the operation location in the image.

12. The method according to claim 1, wherein, The method further includes: The medical device is sent the sequence of operation instructions to cause the medical device to execute the sequence of operation instructions.

13. A medical device control system, characterized in that, The system includes: The acquisition unit acquires the user's intent command; The generation unit generates an ordered sequence of operation tasks to achieve the target intent based on the user intent instruction; The conversion unit converts the ordered sequence of operation tasks into a sequence of operation instructions for the input device of the medical device.

14. A computer program product, comprising at least computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the method described in any one of claims 1 to 12.

Images