A human-computer interaction control method and device, electronic equipment and storage medium

Abstract

The application discloses a human-computer interaction control method and device, electronic equipment and storage medium, which are applied to an electromagnetic navigation angiography system. Specifically, after the electromagnetic navigation angiography system is initialized, a control interface is displayed, and the graphic and text data of a patient is loaded and displayed. Three-dimensional blood vessel model reconstruction is realized based on a blood vessel reconstruction request. The current position and motion track of a guide wire are displayed based on a guide wire display request. Finally, a current control instruction is output to a programmable power supply based on a control instruction input by a user through a joystick, so that the electromagnetic navigation hardware generates a spatial magnetic field to drive the guide wire to move under the drive of the programmable power supply. The application realizes preoperative data management, intraoperative real-time blood vessel three-dimensional reconstruction, AI guide wire accurate tracking, joystick driving navigation and image fusion display through a unified control interface. Compared with the prior art, the application can avoid the disconnection between surgical operation and data checking, thereby improving the surgical efficiency and operation precision.

Description

Technical Field

[0001] This application relates to the field of medical device technology, and more specifically, to a human-computer interaction control method, device, electronic device, and storage medium. Background Technology

[0002] Angioscopy is a common method for diagnosing lung diseases, especially lung tumors. For example, during a lung examination, a doctor inserts a flexible tube equipped with a camera and light source through the patient's mouth or nose into a blood vessel to directly observe the inside of the vessel.

[0003] However, the human vascular tree is complex, with numerous branches and small diameters, posing significant challenges to traditional angiography in several scenarios. Generally, lesions are located in the peripheral lung fields, making it difficult for traditional angiography to reach distal, small vessels; lesions are not directly exposed to the airway surface, making localization impossible based solely on the endoscopic view; and the lack of quantitative guidance methods makes the procedure heavily reliant on the physician's experience. To address these issues, the medical community has developed virtual angiography navigation, which reconstructs a three-dimensional vascular tree model based on computed tomography (CT) data to simulate the angiography path. However, this technology lacks real guidewire position feedback. To overcome this lack of position feedback, electromagnetic navigation angiography has been developed. Building upon the aforementioned three-dimensional model, it introduces electromagnetic positioning technology to track the electromagnetic sensors at the guidewire tip in real time and display their position on the virtual model, guiding the physician along a pre-defined path.

[0004] However, current electromagnetic navigation angiography systems lack comprehensive data management capabilities. Patient information registration, surgical plans (such as lesion marking and path planning), and medical digital imaging and communication (DICOM) file management required for the operation are stored in different medical software, resulting in fragmented data management, disjointed operation processes, and inconvenience in use.

[0005] In existing systems, functions such as patient information registration, surgical planning (e.g., lesion marking and pathway planning), management of medical digital imaging and communication (DICOM) files, and postoperative data archiving often require operators to repeatedly switch between multiple independent software modules or even different hardware devices. This fragmented architecture prevents doctors from quickly accessing patient images and text information on a unified interface, leading to a disconnect between surgical procedures and data verification, directly reducing surgical efficiency and operational accuracy. Summary of the Invention

[0006] In view of this, this application provides a human-computer interaction control method, device, electronic device and storage medium, which are applied to an electromagnetic navigation angiography system to improve surgical efficiency and operational accuracy.

[0007] To achieve the above objectives, the following solution is proposed:

[0008] A human-computer interaction control method is applied to an electromagnetic navigation angiography system, the electromagnetic navigation angiography system including an intelligent analysis module, a detector, a programmable power supply, and electromagnetic navigation hardware, the human-computer interaction control method including the following steps:

[0009] After the electromagnetic navigation angioscope system completes initialization, a control interface is displayed. The control interface is used to import and display the patient's graphic and textual data during the user's import operation.

[0010] In response to a user's request for vascular reconstruction, the system receives multiple sets of two-dimensional perspective images acquired by the detector, and generates and displays the reconstructed three-dimensional vascular model on the control interface based on the multiple sets of two-dimensional perspective images.

[0011] In response to the user's guidewire display request, the system receives real-time images continuously acquired by the detector, generates a set of guidewire position points based on the real-time images, and displays the current position and movement trajectory of the guidewire in the three-dimensional blood vessel model based on the set of position points.

[0012] In response to a user's navigation request, a current control command is output to the programmable power supply based on the user's control command input via a joystick. This causes the programmable power supply to output a drive current to the electromagnetic navigation hardware. Under the drive current, the electromagnetic navigation hardware generates a spatial magnetic field to move the guide wire.

[0013] Optionally, the control interface is a unified integrated interface that supports the entire process of patient information management, DICOM file import, preoperative path planning, intraoperative real-time navigation, angiography control, aspiration / biopsy control, and postoperative data archiving.

[0014] The control interface adopts a simultaneous display mode: the main view area displays the three-dimensional blood vessel model and guidewire image, while the picture-in-picture area synchronously displays the real-time fluoroscopic original image acquired by the detector;

[0015] The control interface also provides a transparency adjustment control for adjusting the overlay and fusion ratio between the real-time perspective original image and the three-dimensional blood vessel model;

[0016] The control interface also integrates a surgical parameter display panel, which displays at least one type of parameter in real time, including injection flow rate, tube voltage, tube current, injection pressure, aspiration negative pressure, fluoroscopy dose, and heating unit.

[0017] When the parameter exceeds the preset threshold, the control interface will display an alarm message and highlight the abnormal item.

[0018] The control interface provides step-by-step guidance controls, which are used to sequentially prompt the user to perform mask acquisition, contrast agent injection, contrast image acquisition, subtraction processing, and three-dimensional vascular reconstruction.

[0019] After reconstruction, the control interface overlays the three-dimensional vascular model onto the airway three-dimensional model reconstructed by preoperative CT as a semi-transparent layer, and provides a switch to show and hide the layer.

[0020] Optionally, the step of responding to a user's vascular reconstruction request by receiving multiple sets of two-dimensional perspective images acquired by the detector, and generating and displaying the reconstructed three-dimensional vascular model on the control interface based on the multiple sets of two-dimensional perspective images, includes the following steps:

[0021] The detector receives the multiple sets of two-dimensional perspective images, which specifically include the mask image collected by the detector before the contrast agent is injected and the contrast image after the contrast agent is injected.

[0022] Digital subtraction is performed on the mask image and the contrast image to remove the bone and soft tissue background, resulting in a blood vessel image sequence;

[0023] The vascular image sequence is reconstructed using a filtered back projection algorithm to obtain the three-dimensional vascular model.

[0024] Optionally, the step of responding to a user's guidewire display request, receiving real-time images continuously acquired by the detector, generating a set of guidewire position points based on the real-time images, and displaying the current position and trajectory of the guidewire in the three-dimensional vascular model based on the set of position points includes the following steps:

[0025] Receive the real-time images continuously acquired by the detector;

[0026] The real-time images are analyzed at the pixel level using an artificial intelligence segmentation model to identify and extract the location point set of the guide wire;

[0027] The coordinates of the location point set are converted into three-dimensional spatial coordinates, and the location point set is rendered in real time in the three-dimensional blood vessel model with a preset color to realize the real-time display of the current position and the movement trajectory.

[0028] Optional,

[0029] The joystick includes a first joystick and a second joystick, and is configured to generate a first current control command and a second current control command based on the user's operation. The first current control command is used to control the forward and backward movement of the programmable power drive wire, and the second current control command is used to control the deflection direction of the programmable power drive wire.

[0030] The control interface displays the virtual control components, the desired direction of the guidewire, the actual posture, and the navigation deviation arrows in real time, and is synchronized with the operation status of the joystick.

[0031] Optional steps may also be included:

[0032] During the navigation operation, real-time images, guidewire movement trajectory, guidewire operation parameters, and electromagnetic navigation data are recorded throughout the process. After the operation, the control interface also provides operation playback, data archiving, and report generation controls for postoperative review and teaching.

[0033] A human-computer interaction control device is applied to an electromagnetic navigation angiography system. The electromagnetic navigation angiography system includes an intelligent analysis module, a detector, a programmable power supply, and electromagnetic navigation hardware. The human-computer interaction control device includes:

[0034] The interface display control module is configured to display a control interface after the electromagnetic navigation angioscope system completes initialization. The control interface is used to import and display the patient's graphic and textual data during the user's import operation.

[0035] The blood vessel display control module is configured to receive multiple sets of two-dimensional perspective images acquired by the detector in response to a user's blood vessel reconstruction request, and generate and display the reconstructed three-dimensional blood vessel model on the control interface based on the multiple sets of two-dimensional perspective images.

[0036] The guidewire display control module is configured to receive real-time images continuously acquired by the detector in response to a user's guidewire display request, generate a set of guidewire position points based on the real-time images, and display the current position and movement trajectory of the guidewire in the three-dimensional blood vessel model based on the set of position points.

[0037] The navigation control module is configured to respond to a user's navigation request by outputting a current control command to the programmable power supply based on the user's control command input via a joystick. This causes the programmable power supply to output a drive current to the electromagnetic navigation hardware, which generates a spatial magnetic field under the drive current to move the guide wire.

[0038] Optional, also includes:

[0039] The recording and playback module is configured to record real-time images, guidewire movement trajectory, guidewire operation parameters, and electromagnetic navigation data throughout the navigation operation. After the operation, the control interface also provides operation playback, data archiving, and report generation controls for postoperative review and teaching.

[0040] An electronic device includes at least one processor and a memory connected to the processor, wherein:

[0041] The memory is used to store computer programs or instructions;

[0042] The processor is used to execute the computer program or instructions to enable the electronic device to implement the human-computer interaction control method as described above.

[0043] A computer-readable storage medium is applied to an electronic device, the storage medium carrying one or more computer programs that can be executed by the electronic device to enable the electronic device to implement the human-computer interaction control method as described above.

[0044] As can be seen from the above technical solution, this application discloses a human-computer interaction control method, device, electronic device, and storage medium. This method and device are applied to an electromagnetic navigation angiography system. Specifically, after the electromagnetic navigation angiography system completes initialization, a control interface is displayed. The control interface is used to import and display the patient's text and image data during the user's import operation; to reconstruct a three-dimensional vascular model based on a vascular reconstruction request; and to display the current position and trajectory of the guidewire based on a guidewire display request. Finally, based on the user's control commands input via a joystick, a current control command is output to the programmable power supply, so that the electromagnetic navigation hardware generates a spatial magnetic field under the drive of the programmable power supply to move the guidewire, thus realizing the surgical operation. This application achieves preoperative data management → real-time intraoperative 3D vascular reconstruction → AI guidewire precise tracking → joystick-based driving navigation → image fusion display through a unified control interface. Compared with existing technologies, this application has an integrated interface, reducing switching and avoiding disconnection between surgical operation and data verification, thereby improving surgical efficiency and operational accuracy. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart of a human-computer interaction control method according to an embodiment of this application;

[0047] Figure 2a This is a schematic diagram of a three-dimensional blood vessel model according to an embodiment of this application;

[0048] Figure 2b This is a schematic diagram illustrating electromagnetic navigation of the guide wire in an embodiment of this application;

[0049] Figure 2cThis is a silhouette of a historical blood vessel image in the embodiments of this application;

[0050] Figure 3 This is a flowchart of another human-computer interaction control method according to an embodiment of this application;

[0051] Figure 4 This is a block diagram of a human-computer interaction control device according to an embodiment of this application;

[0052] Figure 5 This is a block diagram of another human-computer interaction control device according to an embodiment of this application;

[0053] Figure 6 This is a block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0055] A typical electromagnetic navigation angiography system usually includes an electromagnetic positioning device, an electromagnetic field generator, electromagnetic sensors, image processing and navigation software, and manipulators such as guidewires. The electromagnetic positioning device is used for real-time image analysis; the electromagnetic field generator is placed next to the patient's bedside to generate an electromagnetic field with a known spatial distribution; the electromagnetic sensors are installed on positioning probes or at the ends of specialized instruments within the working channel of the angioscope, and can output their three-dimensional coordinates and orientation in the electromagnetic field in real time.

[0056] Image processing and navigation software is used to reconstruct a three-dimensional lung model based on preoperative CT data, including important structures such as vascular tree, lesion location, and blood vessels; automatically plan the optimal navigation path from the main trachea to the lesion; and receive sensor positions in real time and display them graphically on the three-dimensional model.

[0057] The instruments used include electromagnetic positioning catheters, biopsy forceps, aspiration needles, and drug injection catheters.

[0058] Its typical workflow is as follows:

[0059] Preoperative stage: Import patient CT data → 3D reconstruction → Mark lesions → Automatically generate navigation path → Save surgical plan.

[0060] Intraoperative phase: Activate electromagnetic field → Calibration and registration (align the patient's actual position with the 3D model) → Real-time navigation (display guidewire position and path deviation on the screen) → Reach the target point → Perform biopsy, aspiration or drug injection.

[0061] Postoperative stage: Record operation information → Save surgical report. Based on this, this application proposes the following human-computer interaction control scheme.

[0062] Figure 1 This is a flowchart of a human-computer interaction control method according to an embodiment of this application.

[0063] like Figure 1 As shown, the human-computer interaction control method provided in this embodiment is applied to the aforementioned electromagnetic navigation vascular endoscope system. This electromagnetic navigation vascular endoscope system includes at least an intelligent analysis module, a detector, a programmable power supply, and electromagnetic navigation hardware. The human-computer interaction control method of this application is actually applied to the intelligent analysis module for preoperative planning, intraoperative real-time navigation, angiography, three-dimensional vascular reconstruction, AI guidewire tracking, dual-joystick simulated driving electromagnetic control, and postoperative data management in pulmonary vascular interventional surgery. Specifically, it includes the following steps:

[0064] S1. System initialization and unified control interface display.

[0065] Before the surgery begins, the electromagnetic navigation angioscope system performs a power-on startup and initialization process, including hardware communication verification, electromagnetic positioning device calibration, detector exposure parameter calibration, intelligent analysis module loading, high-pressure injector self-test, and programmable power supply self-test. After system initialization is complete, a unified control interface is automatically displayed on the main control screen.

[0066] This control interface is the core interactive platform of this application. It is an integrated graphical user interface that combines patient information, image display, navigation view, parameter adjustment, workflow guidance, equipment control, and alarm prompts into a single window. Doctors can complete all surgical procedures without switching software or interfaces. The system responds to the doctor's import operations within the control interface, automatically reading and loading patient image and text data, including but not limited to basic patient information, medical history, examination and diagnostic information, DICOM format CT image data, preoperative reconstructed 3D airway model, lesion markings, and surgical plans such as the planned pathway. All data is displayed in partitioned sections of the interface, enabling one-stop viewing and management.

[0067] S2. Respond to the vascular reconstruction request, generate and display a three-dimensional vascular model.

[0068] When surgeons need to obtain a realistic vascular structure during surgery to avoid accidental vascular puncture, they trigger a vascular reconstruction request on the unified control interface. A vascular reconstruction request is an instruction initiated by the surgeon via an interface button to start the angiography, image acquisition, digital subtraction, and 3D reconstruction process to obtain a realistic 3D vascular model during surgery. In this case, the following steps are performed:

[0069] 1. Acquire masked images and contrast images.

[0070] The system first controls the X-ray tube and detector to acquire a set of two-dimensional perspective images, called mask images, before the injection of contrast agent. Mask images are perspective images taken before the injection of contrast agent, containing background structures such as bones, soft tissues, and airways, and are used as a "background template" for subsequent subtraction processing.

[0071] Subsequently, the system prompts the doctor to inject the contrast agent through the control interface. After the blood vessels are visualized, the fluoroscopic image is acquired again at the same angle and with the same parameters to obtain the contrast image.

[0072] 2. Perform digital subtraction processing

[0073] The mask image and the contrast image are matched pixel by pixel and the difference is calculated to perform digital subtraction. Digital subtraction is a medical imaging processing technique that subtracts the two images before and after contrast imaging to eliminate background structures such as bones and muscles, retaining only the image of blood vessels filled with contrast agent, making the blood vessels clearly prominent.

[0074] After the subtraction is completed, the system obtains a sequence of vascular images containing only vascular information.

[0075] 3. Reconstructing a three-dimensional blood vessel model based on a filtered back projection algorithm.

[0076] That is, using a filtered back projection algorithm to perform three-dimensional reconstruction of blood vessel image sequences, generating a three-dimensional blood vessel model, such as... Figure 2a As shown, it is automatically loaded into the 3D view of the control interface for display, providing doctors with the real vascular spatial structure during surgery.

[0077] The filtered back projection algorithm is a classic algorithm for medical CT and 3D image reconstruction, which can recover the three-dimensional structure of an object from multi-angle two-dimensional projection images.

[0078] S3. Respond to guidewire display requests and display the guidewire position and movement trajectory in real time.

[0079] During navigation, the doctor triggers a guidewire display request, and the system enters real-time guidewire tracking mode. A guidewire display request is an instruction from the doctor to activate fluoroscopy and AI recognition, enabling the system to display the guidewire's position, orientation, and trajectory in a 3D model in real time. Specifically, it includes the following steps:

[0080] 1. The detector continuously acquires real-time images.

[0081] At this point, the detector is controlled to enter continuous fluoroscopy mode to acquire real-time images during the operation at a fixed frame rate, ensuring that the images are synchronized with the surgical procedure.

[0082] 2. Extract the location point set of the guide wire based on an artificial intelligence segmentation model.

[0083] The intelligent analysis module loads a pre-trained AI segmentation model and performs pixel-level analysis on each frame of real-time image. This allows the model to output a set of guidewire position points, representing a set of three-dimensional coordinates between the guidewire centerline and the tip.

[0084] Artificial intelligence segmentation models are deep learning models that are trained on a large amount of image data. They can automatically identify, segment, and extract guide wire regions in fluoroscopic images, unaffected by tissue, blood vessels, or noise.

[0085] 3. Convert the two-dimensional coordinates of the location point set into three-dimensional spatial coordinates.

[0086] Based on the perspective geometry parameters, detector calibration matrix, and spatial registration relationship, the two-dimensional coordinates of the guidewire's position point set in the image are converted into three-dimensional spatial coordinates consistent with the three-dimensional blood vessel model.

[0087] 4. Real-time rendering of guidewire trajectory in 3D vascular model.

[0088] In the 3D vascular model, the current position, tip posture and motion trajectory of the guidewire are rendered in real time with a highlight color (such as green / yellow) based on the above location point set, so that doctors can intuitively see the relative position of the guidewire with blood vessels and lesions, and achieve precise navigation.

[0089] S4. Respond to navigation requests and perform electromagnetic navigation control based on the joystick.

[0090] When a doctor needs to control the guidewire's advance, retreat, and deflection, a navigation request is triggered on the unified control interface. This means the doctor activates the electromagnetic navigation control mode, inputs their operational intent via a joystick, and the system automatically calculates the current and drives the guidewire movement.

[0091] This application features dual joysticks, enabling the system to acquire the doctor's operational intentions based on the two joysticks: a first joystick and a second joystick. The first joystick controls the guidewire / catheter's forward and backward movement (similar to a car's accelerator); the second joystick controls the direction of the guidewire tip's deflection (similar to a car's steering wheel). This application transforms complex electromagnetic navigation into a "steering wheel + accelerator" operation mode familiar to doctors, significantly reducing the learning curve.

[0092] In practice, the system converts the joystick operation intention into a motion vector, automatically calculates the required current for each electromagnetic coil using a magnetic field calculation model, generates current control commands, and sends them to the programmable power supply. The programmable power supply is a drive device capable of precisely outputting current to each path according to digital commands, providing stable and controllable drive current for the electromagnetic navigation hardware.

[0093] The programmable power supply outputs drive current to the electromagnetic navigation hardware based on current control commands, which in turn generates a synthetic spatial magnetic field through the coordinated action of multiple electromagnetic coils. This electromagnetic navigation hardware consists of multiple sets of electromagnetic coils; by controlling the current in each coil, a magnetic field of arbitrary direction and magnitude can be generated in the surgical area, using magnetic force to drive the guidewire's deflection and movement. Under the influence of the magnetic field, the guidewire moves precisely according to the surgeon's intentions, achieving smooth, driver-like navigation. Figure 2b As shown.

[0094] The control interface displays the attitude and deviation in real time, and the unified control interface synchronously displays the virtual control components, the desired direction of the guidewire, the actual attitude of the guidewire, and the navigation deviation arrow. All displays and joystick operations are linked in real time, forming an intuitive and closed-loop interactive experience.

[0095] In this application, the unified control interface has full-process integration capabilities, enabling patient information management, DICOM file import and viewing, preoperative path planning, lesion marking, and surgical plan saving without switching windows; intraoperative real-time navigation, angiography, subtraction angiography, and 3D reconstruction; fluoroscopy control, injection parameters, and aspiration / biopsy control; and postoperative data archiving, trajectory playback, and surgical report generation.

[0096] The interface employs a simultaneous display mode, with the main view showing a 3D vascular model, airway model, and guidewire trajectory; a picture-in-picture view synchronously displays the real-time fluoroscopic image. This simultaneous display overlays the 3D model with real-time 2D images on the same screen, eliminating the need for doctors to switch between multiple screens and reducing cognitive load. The interface provides transparency controls to adjust the overlay intensity between the fluoroscopic image and the 3D model. The interface also integrates a surgical parameter panel, displaying in real-time injection flow rate, injection pressure, maximum injection pressure; tube voltage, tube current, fluoroscopic dose; aspiration negative pressure, contrast agent heating unit, and remaining dose, etc.

[0097] When parameters exceed the safety threshold, the interface will automatically pop up an alarm prompt and highlight the abnormal items to ensure surgical safety.

[0098] To simplify the doctor's operation, the control interface provides step-by-step guidance controls. These controls are flowchart prompts on the interface, guiding the doctor through the next steps in sequence to avoid errors or omissions. The guided steps are as follows: mask acquisition, contrast agent injection, contrast image acquisition, automatic subtraction processing, and 3D vascular reconstruction. Each step automatically proceeds to the next. After reconstruction, the system overlays the 3D vascular model as a semi-transparent layer onto the preoperative CT airway model, providing a layer switch. The doctor can show / hide the vessels at any time and preview historical silhouettes, such as... Figure 2c As shown.

[0099] As can be seen from the above technical solution, this embodiment provides a human-computer interaction control method. This method is applied to an electromagnetic navigation angiography system. Specifically, after the electromagnetic navigation angiography system completes initialization, a control interface is displayed. The control interface is used to import and display the patient's text and image data during the user's import operation; to reconstruct a three-dimensional vascular model based on a vascular reconstruction request; and to display the current position and movement trajectory of the guidewire based on a guidewire display request. Finally, based on the user's control commands input via a joystick, a current control command is output to the programmable power supply, so that the electromagnetic navigation hardware generates a spatial magnetic field under the drive of the programmable power supply to move the guidewire, thereby realizing the surgical operation. This application realizes preoperative data management → real-time intraoperative 3D vascular reconstruction → AI guidewire precise tracking → joystick-based driving navigation → image fusion display through a unified control interface. Compared with the prior art, this application has an integrated interface, reducing switching and thus avoiding the disconnect between surgical operation and data verification, improving surgical efficiency and operational accuracy.

[0100] In addition, in one specific embodiment of this application, the following steps are also included: Figure 3 As shown.

[0101] S5. During the navigation operation, record real-time images, guidewire movement trajectory, guidewire operation parameters, and electromagnetic navigation data throughout the entire process.

[0102] Specifically, this involves recording, replaying, and archiving the entire surgical procedure. The system automatically activates full-process recording throughout navigation, angiography, biopsy, and aspiration. Recorded content includes real-time fluoroscopic images; guidewire 3D position, orientation, and trajectory; joystick operations, current commands, and magnetic field parameters; injection pressure, flow rate, dosage, and aspiration parameters.

[0103] After the surgery, the unified control interface provides operation playback, data archiving, and report generation. Operation playback: allows users to review the entire surgical process from beginning to end; data archiving: automatically saves case and surgical data; report generation: generates standardized surgical reports with one click. This enables traceability, review, teaching, and quality control of the surgery.

[0104] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0105] Although the operations are described in a specific order, this should not be construed as requiring these operations to be executed in the indicated order or sequentially. In specific application scenarios, multitasking and parallel processing can be employed to improve efficiency.

[0106] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0107] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer.

[0108] Figure 4 This is a block diagram of a human-computer interaction control device according to an embodiment of this application.

[0109] like Figure 4As shown, the human-computer interaction control device provided in this embodiment is applied to the aforementioned electromagnetic navigation vascular endoscope system. This electromagnetic navigation vascular endoscope system includes at least an intelligent analysis module, a detector, a programmable power supply, and electromagnetic navigation hardware. The human-computer interaction control method of this application is actually applied to the intelligent analysis module for preoperative planning, intraoperative real-time navigation, angiography, three-dimensional vascular reconstruction, AI guidewire tracking, dual-joystick simulated driving electromagnetic control, and postoperative data management in pulmonary vascular interventional surgery. Specifically, the device includes an interface display control module 10, a vascular display control module 20, a guidewire display control module 30, and a navigation control module 40.

[0110] The interface display control module is used to perform system initialization and unify the control interface display.

[0111] Before the surgery begins, the electromagnetic navigation angioscope system performs a power-on startup and initialization process, including hardware communication verification, electromagnetic positioning device calibration, detector exposure parameter calibration, intelligent analysis module loading, high-pressure injector self-test, and programmable power supply self-test. After system initialization is complete, a unified control interface is automatically displayed on the main control screen.

[0112] This control interface is the core interactive platform of this application. It is an integrated graphical user interface that combines patient information, image display, navigation view, parameter adjustment, workflow guidance, equipment control, and alarm prompts into a single window. Doctors can complete all surgical procedures without switching software or interfaces. The system responds to the doctor's import operations within the control interface, automatically reading and loading patient image and text data, including but not limited to basic patient information, medical history, examination and diagnostic information, DICOM format CT image data, preoperative reconstructed 3D airway model, lesion markings, and surgical plans such as the planned pathway. All data is displayed in partitioned sections of the interface, enabling one-stop viewing and management.

[0113] The vascular display control module is used to respond to vascular reconstruction requests, generate and display three-dimensional vascular models.

[0114] When surgeons need to obtain a realistic vascular structure during surgery to avoid accidental vascular puncture, they trigger a vascular reconstruction request on the unified control interface. A vascular reconstruction request is an instruction initiated by the surgeon via an interface button, used to initiate the angiography, image acquisition, digital subtraction, and 3D reconstruction processes to obtain a realistic 3D vascular model during surgery. The specific execution process has been detailed in the previous embodiment and will not be repeated here.

[0115] The guidewire display control module is used to respond to guidewire display requests and display the guidewire position and movement trajectory in real time.

[0116] During navigation, the doctor triggers a guidewire display request, and the system enters real-time guidewire tracking mode. The guidewire display request is the instruction from the doctor to activate fluoroscopy and AI recognition, enabling the system to display the guidewire's position, orientation, and trajectory in a 3D model in real time. The specific execution process has been detailed in the previous embodiment and will not be repeated here.

[0117] The navigation control module is used to respond to navigation requests and execute electromagnetic navigation control based on the joystick.

[0118] When a doctor needs to control the guidewire's advance, retreat, and deflection, a navigation request is triggered on the unified control interface. This means the doctor activates the electromagnetic navigation control mode, inputs their operational intent via a joystick, and the system automatically calculates the current and drives the guidewire movement. The specific control process has been described in detail in the previous embodiment and will not be repeated here.

[0119] As can be seen from the above technical solution, this embodiment provides a human-computer interaction control device. This device is applied to an electromagnetic navigation angiography system. Specifically, after the electromagnetic navigation angiography system completes initialization, a control interface is displayed. The control interface is used to import and display the patient's text and image data during the user's import operation; to reconstruct a three-dimensional vascular model based on a vascular reconstruction request; and to display the current position and movement trajectory of the guidewire based on a guidewire display request. Finally, based on the user's control commands input via a joystick, a current control command is output to the programmable power supply, so that the electromagnetic navigation hardware generates a spatial magnetic field under the drive of the programmable power supply to move the guidewire and realize the surgical operation. This application realizes preoperative data management → real-time intraoperative 3D vascular reconstruction → AI guidewire precise tracking → joystick-based driving navigation → image fusion display through a unified control interface. Compared with the prior art, this application has an integrated interface, reducing switching and thus avoiding the disconnect between surgical operation and data verification, improving surgical efficiency and operational accuracy.

[0120] In addition, in one specific embodiment of this application, a recording and playback module 50 is also included, such as... Figure 5 As shown.

[0121] The recording and playback module is used to record real-time images, guidewire movement trajectory, guidewire operation parameters, and electromagnetic navigation data throughout the navigation operation.

[0122] Specifically, this involves recording, replaying, and archiving the entire surgical procedure. The system automatically activates full-process recording throughout navigation, angiography, biopsy, and aspiration. Recorded content includes real-time fluoroscopic images; guidewire 3D position, orientation, and trajectory; joystick operations, current commands, and magnetic field parameters; injection pressure, flow rate, dosage, and aspiration parameters.

[0123] After the surgery, the unified control interface provides operation playback, data archiving, and report generation. Operation playback: allows users to review the entire surgical process from beginning to end; data archiving: automatically saves case and surgical data; report generation: generates standardized surgical reports with one click. This enables traceability, review, teaching, and quality control of the surgery.

[0124] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0125] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0126] Figure 6 This is a block diagram of an electronic device according to an embodiment of this application.

[0127] The following is for reference. Figure 6 This document illustrates a structural diagram suitable for implementing the electronic device in the embodiments of this disclosure. The terminal device in the embodiments of this disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. This electronic device is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this disclosure.

[0128] The electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from an input device 606 into a random access memory (RAM) 603. The RAM also stores various programs and data required for the operation of the electronic device. The processing unit, ROM, and RAM are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0129] Typically, the following devices can be connected to the I / O interface: input devices including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows the electronic device to communicate wirelessly or wiredly with other devices to exchange data. Although electronic devices with various devices are shown in the figures, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0130] This application also provides an embodiment of a computer-readable storage medium.

[0131] The aforementioned computer-readable storage medium is applied to an electronic device, specifically to the intelligent analysis module of the aforementioned electromagnetic navigation angiography system. This module carries one or more computer programs. When these programs are executed by the electronic device, it displays a control interface after the electromagnetic navigation angiography system has completed initialization. This control interface is used to import and display the patient's image and text data during user import operations; to reconstruct a three-dimensional vascular model based on vascular reconstruction requests; to display the current position and trajectory of the guidewire based on guidewire display requests; and finally, to output current control commands to the programmable power supply based on user input via a joystick. This causes the electromagnetic navigation hardware to generate a spatial magnetic field, driving the guidewire to move and thus performing the surgical operation. This application achieves preoperative data management → real-time intraoperative 3D vascular reconstruction → AI guidewire precise tracking → joystick-based navigation → image fusion display through a unified control interface. Compared to existing technologies, this application features an integrated interface, reducing switching and thus avoiding disconnect between surgical operations and data verification, improving surgical efficiency and operational accuracy.

[0132] It should be noted that the computer-readable medium described above in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0133] In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0134] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0135] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0136] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0137] The technical solution provided by the present invention has been described in detail above. Specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An electronic device applied to an electromagnetic navigation angiography system, the electromagnetic navigation angiography system comprising an intelligent analysis module, a detector, a programmable power supply, and electromagnetic navigation hardware, characterized in that, The electronic device includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs or instructions; The processor is used to execute the computer program or instructions to enable the electronic device to implement the following human-computer interaction control method: After the electromagnetic navigation angioscope system completes initialization, a control interface is displayed. The control interface is used to import and display the patient's graphic and textual data during the user's import operation. In response to a user's request for vascular reconstruction, the system receives multiple sets of two-dimensional perspective images acquired by the detector, and generates and displays the reconstructed three-dimensional vascular model on the control interface based on the multiple sets of two-dimensional perspective images. In response to the user's guidewire display request, the system receives real-time images continuously acquired by the detector, generates a set of guidewire position points based on the real-time images, and displays the current position and movement trajectory of the guidewire in the three-dimensional blood vessel model based on the set of position points. In response to a user's navigation request, a current control command is output to the programmable power supply based on the user's control command input via a joystick. This causes the programmable power supply to output a drive current to the electromagnetic navigation hardware. Under the drive current, the electromagnetic navigation hardware generates a spatial magnetic field to move the guide wire.

2. The electronic device as claimed in claim 1, characterized in that, The control interface is a unified integrated interface that supports the entire process of patient information management, DICOM file import, preoperative path planning, intraoperative real-time navigation, angiography control, aspiration / biopsy control, and postoperative data archiving. The control interface adopts a simultaneous display mode: the main view area displays the three-dimensional blood vessel model and guidewire image, while the picture-in-picture area synchronously displays the real-time fluoroscopic original image acquired by the detector; The control interface also provides a transparency adjustment control for adjusting the overlay and fusion ratio between the real-time perspective original image and the three-dimensional blood vessel model; The control interface also integrates a surgical parameter display panel, which displays at least one type of parameter in real time, including injection flow rate, tube voltage, tube current, injection pressure, aspiration negative pressure, fluoroscopy dose, and heating unit. When the parameter exceeds the preset threshold, the control interface will display an alarm message and highlight the abnormal item. The control interface provides step-by-step guidance controls, which are used to sequentially prompt the user to perform mask acquisition, contrast agent injection, contrast image acquisition, subtraction processing, and three-dimensional vascular reconstruction. After reconstruction, the control interface overlays the three-dimensional vascular model onto the airway three-dimensional model reconstructed by preoperative CT as a semi-transparent layer, and provides a switch to show and hide the layer.

3. The electronic device as claimed in claim 1, characterized in that, The process of responding to a user's vascular reconstruction request by receiving multiple sets of two-dimensional perspective images acquired by the detector, and generating and displaying a reconstructed three-dimensional vascular model on the control interface based on the multiple sets of two-dimensional perspective images includes the following steps: The detector receives the multiple sets of two-dimensional perspective images, which specifically include the mask image collected by the detector before the contrast agent is injected and the contrast image after the contrast agent is injected. Digital subtraction is performed on the mask image and the contrast image to remove the bone and soft tissue background, resulting in a blood vessel image sequence; The vascular image sequence is reconstructed using a filtered back projection algorithm to obtain the three-dimensional vascular model.

4. The electronic device as claimed in claim 1, characterized in that, The process of responding to a user's guidewire display request, receiving real-time images continuously acquired by the detector, generating a set of guidewire position points based on the real-time images, and displaying the current position and trajectory of the guidewire in the three-dimensional vascular model based on the set of position points includes the following steps: Receive the real-time images continuously acquired by the detector; The real-time images are analyzed at the pixel level using an artificial intelligence segmentation model to identify and extract the location point set of the guide wire; The coordinates of the location point set are converted into three-dimensional spatial coordinates, and the location point set is rendered in real time in the three-dimensional blood vessel model with a preset color to realize the real-time display of the current position and the movement trajectory.

5. The electronic device as claimed in claim 1, characterized in that, The joystick includes a first joystick and a second joystick, and is configured to generate a first current control command and a second current control command based on the user's operation. The first current control command is used to control the forward and backward movement of the programmable power drive wire, and the second current control command is used to control the deflection direction of the programmable power drive wire. The control interface displays the virtual control components, the desired direction of the guidewire, the actual posture, and the navigation deviation arrows in real time, and is synchronized with the operation status of the joystick.

6. The electronic device as claimed in claim 1, characterized in that, The human-computer interaction control method further includes the following steps: During the navigation operation, real-time images, guidewire movement trajectory, guidewire operation parameters, and electromagnetic navigation data are recorded throughout the process. After the operation, the control interface also provides operation playback, data archiving, and report generation controls for postoperative review and teaching.

Images