Implant virtual placement interaction method and device

Abstract

The invention discloses an implant virtual placement interaction method and device. The interaction method comprises the following steps: acquiring a three-dimensional model of a cerebral cortex; in response to a received operation instruction, updating the position of the implant, and obtaining a real-time three-dimensional coordinate of the implant; the implant is placed in the three-dimensional model of the cerebral cortex or placed on the surface of the three-dimensional model of the cerebral cortex; re-rendering the three-dimensional model of the cerebral cortex based on the real-time three-dimensional coordinates of the implant; and outputting the re-rendered three-dimensional model of the cerebral cortex. According to the scheme, the position of the implant can be adjusted on the three-dimensional model of the cerebral cortex, and the operation effect is visually displayed in real time. Moreover, in the data processing process, switching between different software is not needed, the data processing flow is simplified, and the data processing efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of brain-computer interface technology, and in particular to an implant virtual placement interaction method and device. Background Art

[0002] In neurosurgery and brain science, especially in brain-computer interface research, the conventional workflow is to analyze the patient's or experimental subject's preoperative Magnetic Resonance Imaging (MRI) data to reconstruct a three-dimensional model of the cerebral cortex; then, on the three-dimensional model of the cerebral cortex, various electrodes implanted during surgery (such as ECoG cortical electrode arrays, SEEG deep electrodes, ECoG flexible electrodes, etc.) are accurately positioned in three dimensions for subsequent clinical diagnosis or scientific analysis.

[0003] Currently, the operation process of the above workflow is rather cumbersome, requiring switching between multiple independent software (such as Freeview, MATLAB, Python command line terminal, etc.). It is also impossible to adjust the position of the electrode on the three-dimensional model, making it difficult to intuitively display the operation effect in real time. Summary of the Invention

[0004] The present invention aims to provide an interactive method and device for virtual implant placement, enabling adjustment of the implant's position on a three-dimensional model of the cerebral cortex and intuitive real-time display of the manipulation effects. Furthermore, during data processing, switching between different software programs is eliminated, simplifying the data processing process and improving data processing efficiency.

[0005] In a first aspect, the present invention provides an implant virtual placement interaction method, which is applied to a brain-computer interface, and includes: obtaining a three-dimensional model of the cerebral cortex; updating the position of the implant in response to a received operation instruction, and obtaining the real-time three-dimensional coordinates of the implant; placing the implant in the three-dimensional model of the cerebral cortex, or on the surface of the three-dimensional model of the cerebral cortex; re-rendering the three-dimensional model of the cerebral cortex based on the real-time three-dimensional coordinates of the implant; and outputting the re-rendered three-dimensional model of the cerebral cortex.

[0006] After receiving an instruction to operate the implant, the position of the implant on or within the three-dimensional model of the cerebral cortex is updated, and the real-time three-dimensional coordinates of the implant are obtained. Furthermore, based on the real-time three-dimensional coordinates of the implant, the three-dimensional model of the cerebral cortex is re-rendered and output. This allows the position of the implant to be adjusted on the three-dimensional model of the cerebral cortex, and the effects of the operation can be intuitively displayed in real time. Furthermore, during data processing, there is no need to switch between different software programs, simplifying the data processing process and improving data processing efficiency.

[0007] Optionally, the re-rendering of the three-dimensional model of the cerebral cortex based on the real-time three-dimensional coordinates of the implant includes: obtaining the real-time three-dimensional coordinates corresponding to the key points on the implant; querying the anatomical partition ROI data based on the real-time three-dimensional coordinates corresponding to the key points to determine the target brain area of ​​the key points in the cerebral cortex; obtaining the electrophysiological data corresponding to the target brain area; adjusting the visual attributes of the key points based on the electrophysiological data corresponding to the target brain area; the visual attributes of the key points include at least one of the following: the size of the key points, the color of the key points, and the brightness of the key points.

[0008] Optionally, the implant is an invasive implant that penetrates the cerebral cortex; the method further includes: in response to receiving a cross-sectional image generation instruction, obtaining a three-dimensional trajectory of the implant; performing virtual cutting along the three-dimensional trajectory to obtain the intersection points of each anatomical partition ROI passed by the three-dimensional trajectory; generating a two-dimensional cross-sectional image, the two-dimensional cross-sectional image including: the trajectory of the implant displayed in a one-dimensional linear manner, each layer of ROI passed by the implant, and the starting and ending depths of the implant passing through each layer of ROI; different layers of ROI passed by the implant are marked with different colors, and the starting and ending depths of the implant passing through each layer of ROI are marked with labels.

[0009] Optionally, the implant virtual placement interaction method further includes: outputting the two-dimensional cross-sectional image.

[0010] Optionally, the operation instruction is a movement instruction output by a human-computer interaction device to move the implant; updating the position of the implant and obtaining the real-time three-dimensional coordinates of the implant include: obtaining an actual displacement corresponding to the movement instruction; based on the actual displacement, obtaining a target displacement of the implant on the three-dimensional model of the cerebral cortex; based on the target displacement, obtaining the real-time three-dimensional coordinates of the implant.

[0011] Optionally, the operation instruction is a movement instruction output by a human-computer interaction device to move the implant; updating the position of the implant and obtaining the real-time three-dimensional coordinates of the implant include: obtaining the three-dimensional convex hull of the three-dimensional model of the cerebral cortex to generate a proxy interaction surface; obtaining the displacement of the actual displacement corresponding to the movement instruction on the proxy interaction surface; after the movement instruction ends, projecting the final displacement on the proxy interaction surface to the three-dimensional model of the cerebral cortex to obtain the target displacement of the implant on the three-dimensional model of the cerebral cortex; and obtaining the real-time three-dimensional coordinates of the implant based on the target displacement.

[0012] Optionally, obtaining a three-dimensional model of the cerebral cortex includes: performing standard preprocessing and multimodal registration on the acquired image data to generate a three-dimensional model file and an anatomical partition file corresponding to the image data; the image data includes at least one of the following: anatomical image data, implant image data; performing standard preprocessing and multimodal registration on the image data to obtain a three-dimensional model file and an anatomical partition file; and constructing a three-dimensional model of the cerebral cortex based on the three-dimensional model file and the anatomical partition file.

[0013] In a second aspect, the present invention provides an implant virtual placement interaction device for use in a brain-computer interface, comprising: a model acquisition unit for acquiring a three-dimensional model of the cerebral cortex; a three-dimensional coordinate acquisition unit for updating the position of the implant in response to a received operation instruction and acquiring the real-time three-dimensional coordinates of the implant; the implant is placed in the three-dimensional model of the cerebral cortex, or on the surface of the three-dimensional model of the cerebral cortex; a rendering unit for re-rendering the three-dimensional model of the cerebral cortex based on the real-time three-dimensional coordinates of the implant; and an output unit for outputting the re-rendered three-dimensional model of the cerebral cortex.

[0014] In a third aspect, the present invention also provides a computer-readable storage medium, which is a non-volatile storage medium or a non-transient storage medium, on which a computer program is stored. When the computer program is run by a processor, the steps of any one of the above-mentioned implant virtual placement interaction methods are executed.

[0015] In a fourth aspect, the present invention also provides another implant virtual placement interaction device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor executes the steps of any one of the above-mentioned implant virtual placement interaction methods when running the computer program. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1is a flow chart of an implant virtual placement interaction method according to an embodiment of the present invention; Figure 2 2 is a structural diagram of an implant virtual placement interaction device in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] Currently, the process of localizing implants in the cerebral cortex typically involves the following steps: 1) Data preprocessing Doctors or researchers use medical imaging viewing software (such as Freesurfer) to process the preoperative magnetic resonance imaging images of patients or experimental subjects and reconstruct a three-dimensional mesh model of the cerebral cortex (pial surface).

[0018] 2) Initial coordinate acquisition Doctors or researchers learn the approximate location of the implant's electrode array from surgical records and intraoperative 3D navigation, and then manually click and record the approximate three-dimensional coordinates of a corner of the electrode array in the medical imaging viewing software.

[0019] 3) Coordinate data entry The doctor or researcher inputs the coordinates obtained manually in step 2) into a MATLAB matrix file or a similar text file.

[0020] 4) Script execution and calculation Run img_pipe, a widely referenced command-line Python toolkit developed by the Chang Lab at UCSF. This script reads the file generated in step 3) and calculates the 3D coordinates of all electrodes in the electrode array based on the preset electrode array geometry parameters (such as the number of rows and columns and the spacing).

[0021] 5) Visualization and Verification After the above img_pipe script is executed, a static image of the brain model with electrodes is generated for doctors or researchers to check whether the positioning results are accurate.

[0022] As can be seen, the existing workflow for implant positioning is relatively cumbersome, requiring switching between multiple independent software programs (such as Freeview, MATLAB, and Python command-line terminals). Data must be manually copied and transcribed, resulting in a fragmented process and prone to errors. Furthermore, implant placement is based on offline, non-visual digital coordinate entry. Users cannot place or adjust the implant through direct manipulation (such as dragging) on ​​the 3D model, making it difficult to intuitively display user operations. Electrode position cannot be adjusted on the 3D model, making it difficult to intuitively display the effects of manipulations in real time.

[0023] Furthermore, after executing the script and generating the brain model image, if the implant position is found to be different from the expected one, it is necessary to repeat step 2 above, reselect the coordinates in the imaging software, modify the data file, and rerun the script. This entire feedback loop of trial and error and adjustment is time-consuming.

[0024] In an embodiment of the present invention, after receiving an operation instruction to operate an implant, the position of the implant on or within the three-dimensional model of the cerebral cortex is updated, and the real-time three-dimensional coordinates of the implant are obtained. Furthermore, based on the real-time three-dimensional coordinates of the implant, the three-dimensional model of the cerebral cortex is re-rendered and output. This allows the position of the implant to be adjusted on the three-dimensional model of the cerebral cortex, and the effects of the operation can be intuitively displayed in real time. Furthermore, during the data processing process, there is no need to switch between different software programs, simplifying the data processing process and improving data processing efficiency.

[0025] In order to make the above-mentioned objects, features and beneficial effects of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0026] The embodiment of the present invention provides an implant virtual placement interaction method, referring to Figure 1 , the following is a detailed description through specific steps.

[0027] In the above embodiments of the present invention, the implant virtual placement interaction method described in the following embodiments can be executed by an implant virtual placement interaction system. The implant virtual interaction system can include a front-end application module and a back-end computing service module.

[0028] In a specific implementation, the front-end application module can be built based on a modern web technology stack (such as Electron, React, and Three.js). This module can be responsible for providing a graphical user interface (GUI), which users can operate using a human-computer interface (such as a mouse and keyboard) to input operational instructions. The front-end application module includes a 3D rendering engine for rendering a 3D model of the cerebral cortex. The front-end application module can also include a real-time implant interaction module to capture and respond to user operations.

[0029] The backend computing service module, built on Python and its scientific computing ecosystem (such as nibabel and trimesh), is responsible for performing computationally intensive tasks or tasks that are inconvenient to handle on the front end. The backend computing service module is mainly used for the following: Preprocessing service, which automatically calls external command-line tools such as Freesurfer to process raw MRI data and further optimize the generated raw anatomical model, such as generating proxy surfaces for smooth interaction and converting model formats (such as to gltf / stl). Image registration services, used to assist in the registration of multimodal images (such as preoperative MRI and postoperative CT); Data storage service, responsible for saving the user's work (such as implant coordinates, project configuration) as files and reloading them.

[0030] The implant virtual placement interaction system can also include a front-end and back-end communication module, responsible for communication between the front-end application module and the back-end computing service module. This front-end and back-end communication module can be implemented in various ways, such as directly calling back-end scripts through inter-process communication (IPC), or through a more loosely coupled approach such as message queues for task distribution and result collection.

[0031] The implant virtual placement interactive system may also include a data export module, which is responsible for exporting the relevant data after the user completes the virtual placement into a file of the corresponding format according to clinical or scientific research needs.

[0032] In the following embodiments, for ease of description, the implant virtual placement interaction system is referred to as the interaction system.

[0033] Step 101: Acquire a three-dimensional model of the cerebral cortex.

[0034] In a specific implementation, a user can select a folder containing image data in a graphical user interface and select image data from the folder. The image data can include at least one of the following: anatomical image data and implant image data. The anatomical image data can be MRI image data, and the implant image data can be CT data.

[0035] In specific implementations, when the user selects the image data, the interactive system can automatically call the medical image viewing software (Freesurfer, etc.) to perform standard preprocessing on the selected image data, namely multimodal registration, to generate a three-dimensional model file and an anatomical partition file corresponding to the image data.

[0036] In some embodiments, the above-mentioned three-dimensional model files and anatomical partition files may also be pre-generated. After the user selects these pre-generated three-dimensional model files and anatomical partition files, the selected three-dimensional model files and anatomical partition files may be directly loaded.

[0037] The three-dimensional rendering engine in the interactive system can process the three-dimensional model file and the anatomical partition file, and render a three-dimensional model of the cerebral cortex in the GUI interface.

[0038] In specific implementation, the specific process of obtaining the three-dimensional model of the cerebral cortex can refer to the relevant implementation solutions in the existing technology and will not be described in detail here.

[0039] Step 102: In response to the received operation instruction, the position of the implant is updated and the real-time three-dimensional coordinates of the implant are obtained.

[0040] In an embodiment of the present invention, a user can input an operation instruction to the interactive system through a human-computer interaction interface. Upon receiving the operation instruction input by the user, the interactive system can update the position of the implant and obtain the real-time three-dimensional coordinates of the implant.

[0041] In a specific implementation, the implant can be placed in the three-dimensional model of the cerebral cortex. Alternatively, the implant can be placed on the surface of the three-dimensional model of the cerebral cortex.

[0042] In a specific implementation, the operation instruction input by the user may be to place an implant on or within the three-dimensional model of the cerebral cortex, or to move a placed implant on the three-dimensional model of the cerebral cortex to adjust the position of the implant.

[0043] When the operation command is to move the implant, the interactive system can obtain the actual displacement corresponding to the movement command. Based on the actual displacement, the interactive system maps the implant's target displacement onto the 3D model of the cerebral cortex. Based on the implant's target displacement, the interactive system obtains the implant's real-time 3D coordinates.

[0044] In some embodiments, the user can place the implant by dragging the mouse, or drag the implant to change the position of the implant in the cerebral cortex. Alternatively, the user can drag the implant to change the position of the implant by sliding a gesture.

[0045] When the user drags the implant via the human-computer interaction device, the displacement of the drag is the actual displacement. The interactive system can determine the target displacement of the implant on the three-dimensional model of the cerebral cortex based on the actual displacement of the movement instruction. The mapping relationship between the actual position input by the human-computer interaction device and the target displacement can be pre-set.

[0046] For example, the actual displacement of the mouse is 1 cm to the left, and the corresponding target displacement on the three-dimensional model of the cerebral cortex is 2 cm to the left.

[0047] After determining the target displacement of the implant on the three-dimensional model of the cerebral cortex, the interactive system can update the three-dimensional coordinates of the implant in real time according to the target displacement to obtain the real-time three-dimensional coordinates of the implant.

[0048] In a specific implementation, the implant may include an electrode array, which may include a plurality of electrodes. When acquiring the real-time three-dimensional coordinates of the implant, the real-time three-dimensional coordinates corresponding to each electrode may also be acquired simultaneously.

[0049] In a specific implementation, the user can move the entire implant. Alternatively, the user can move a single electrode or multiple electrodes within the implant. When the user moves the entire implant, the real-time 3D coordinates of the entire implant and each electrode can be obtained. When the user moves a portion of the electrodes within the implant, the real-time 3D coordinates of the moved portion can be obtained.

[0050] In a specific implementation, the user may move the implant by dragging the implant or rotating the implant. The user may move the implant by dragging the electrode or rotating the electrode.

[0051] In a specific implementation, the electrode array in the implant can be an electrocorticogram (ECoG) electrode array, a deep brain stimulation (DBS) electrode, a stereotactic electroencephalogram (SEEG) electrode, an invasive microwire electrode, a neural probe, etc., or any other virtual medical device that needs to be precisely placed on the target anatomical structure and has different physical properties (such as rigidity, flexibility) and materials.

[0052] Step 103 : re-rendering the three-dimensional model of the cerebral cortex based on the real-time three-dimensional coordinates of the implant.

[0053] Step 104: output the re-rendered three-dimensional model of the cerebral cortex.

[0054] In an embodiment of the present invention, after obtaining the real-time 3D coordinates of the implant, the interactive system can re-render the 3D model of the cerebral cortex using a 3D rendering engine. This allows the 3D model of the cerebral cortex to be updated in real time. Changes in the implant's position can be intuitively and visually presented on the 3D model of the cerebral cortex.

[0055] In an embodiment of the present invention, after obtaining the real-time three-dimensional coordinates of the implant, during the process of re-rendering the three-dimensional model of the cerebral cortex, a fusion and visualization process of multimodal information can also be performed.

[0056] During the fusion and visualization of the multimodal information, the real-time three-dimensional coordinates of the key points on the implant can be obtained. The key points on the implant can be the center of the electrode in the implant, a vertex of the electrode, or any point on the electrode.

[0057] Based on the real-time 3D coordinates of the key points, the interactive system can query anatomical partitioning data to determine the target brain region of the cerebral cortex where the key points are located. Based on the target brain region and the real-time 3D coordinates of the key points, the interactive system can query the electrophysiological data corresponding to the target brain region and obtain corresponding functional indicators (such as signal energy). The interactive system adjusts the visual attributes of the key points based on preset mapping rules.

[0058] In some embodiments, the interactive system can change the color of key points based on the anatomical region in which they are located. The interactive system can also change the size and brightness of key points based on the functional indicators corresponding to the key points, thereby re-rendering the three-dimensional model of the cerebral cortex.

[0059] That is to say, in the process of re-rendering the three-dimensional model of the cerebral cortex, the visual attributes of the key points of the electrodes can also be adjusted, thereby enriching the image information of the re-rendered three-dimensional model of the cerebral cortex.

[0060] In an embodiment of the present invention, if the implant is an invasive implant (such as a SEEG electrode), the user may select the implant in the user graphical interface and trigger a cross-sectional image generation instruction.

[0061] When the interactive system receives the cross-sectional image generation instruction, it can obtain the three-dimensional trajectory of the implant. The interactive system performs a virtual cut along the three-dimensional trajectory, obtaining the intersection points of each anatomical region passed by the three-dimensional trajectory. The interactive system can generate a new two-dimensional cross-sectional image, which can include: the implant trajectory displayed in a one-dimensional linear manner, each layer of ROI passed by the implant, and the starting and ending depths of the implant passing through each layer of ROI; the different layers of ROI passed by the implant are marked with different colors, and the starting and ending depths of the implant passing through each layer of ROI are marked with labels.

[0062] Specifically, the content in the above-mentioned tag may be specific values ​​of the start and end depths.

[0063] In an embodiment of the present invention, after the user completes all operations, the user can export various data such as the precise three-dimensional coordinates of all electrodes, tables with ROI labels, visualization results integrating electrophysiological information, and cross-sectional diagrams through a one-click export button in the user graphical interface.

[0064] In an embodiment of the present invention, the interactive system may also employ other methods to obtain the real-time 3D coordinates of the implant in step 102. Based on a 3D model of the cerebral cortex, the interactive system may calculate the 3D convex hull corresponding to the 3D model of the cerebral cortex and generate a proxy interaction surface. Compared to a 3D model of the cerebral cortex, a proxy interaction surface has lower computational overhead.

[0065] When the operation command is a move command, the interactive system obtains the actual displacement on the proxy interaction surface. When the user finishes moving the implant (i.e., the operation command ends), the final displacement on the proxy interaction surface is projected onto the 3D model of the cerebral cortex to obtain the target displacement of the implant on the 3D model of the cerebral cortex. The interactive system then obtains the real-time 3D coordinates of the implant based on the target displacement.

[0066] Compared with directly calculating the real-time three-dimensional coordinates of the implant on the three-dimensional model of the cerebral cortex, the three-dimensional convex hull method is used to separate interactive calculation and precise positioning, which improves the operation fluency while achieving precise positioning.

[0067] In summary, the implant virtual placement interaction method provided in the above embodiments of the present invention integrates the entire process, from raw data processing (obtaining a three-dimensional model of the cerebral cortex) to final data output, into a single user graphical interface, eliminating the need to switch between multiple software programs and improving work efficiency.

[0068] This system dynamically binds and visualizes the implant's anatomical position information (derived from anatomical zoning files) and functional information (derived from electrophysiological data) in real time, generating multidimensional cross-sectional views and providing rich information for clinical diagnosis and scientific research. It visualizes the user's operation process and final rendering results, minimizing learning costs. Furthermore, it provides timely and intuitive feedback on the user's operation results.

[0069] In an embodiment of the present invention, the aforementioned virtual implant placement method can also be extended and applied to a three-dimensional model of the human head. Specifically, the three-dimensional model of the human head can be acquired, and in response to received operational instructions, the position of the implant can be updated, and the real-time coordinates of the implant can be reacquired. The implant can be placed within the three-dimensional model of the human head, or on the surface of the three-dimensional model of the human head. Based on the real-time three-dimensional coordinates of the implant, the three-dimensional model of the human head can be re-rendered, and the re-rendered three-dimensional model of the human head can be output.

[0070] In this embodiment of the present invention, the real-time 3D coordinates of the implant obtained in step 102 are the real-time 3D coordinates of the implant within a 3D model of the cerebral cortex of a specific individual. After obtaining the real-time 3D coordinates of the implant, they can also be mapped to a standard brain template (such as the MNI152 template) to facilitate comparison with data from other individuals.

[0071] Reference Figure 2 , an implant virtual placement interaction device 20 according to an embodiment of the present invention is provided, comprising: a model acquisition unit 201, a three-dimensional coordinate acquisition unit 202, a rendering unit 203, and an output unit 204, wherein: A model acquisition unit 201 is used to acquire a three-dimensional model of the cerebral cortex; a three-dimensional coordinate acquisition unit 202 for updating the position of an implant in response to a received operation instruction and acquiring real-time three-dimensional coordinates of the implant; the implant is placed in the three-dimensional model of the cerebral cortex, or on the surface of the three-dimensional model of the cerebral cortex; a rendering unit 203, configured to re-render the three-dimensional model of the cerebral cortex based on the real-time three-dimensional coordinates of the implant; The output unit 204 is configured to output the re-rendered three-dimensional model of the cerebral cortex.

[0072] In a specific implementation, the specific execution process of the above-mentioned model acquisition unit 201, three-dimensional coordinate acquisition unit 202, rendering unit 203 and output unit 204 can correspond to the above-mentioned steps 101 to 104, which will not be repeated here.

[0073] In specific implementations, the modules / units included in the various devices and products described in the above embodiments may be software modules / units or hardware modules / units, or may be partially software modules / units and partially hardware modules / units.

[0074] For example, for each device or product applied to or integrated into a chip, each module / unit contained therein may be implemented in the form of hardware such as circuits, or at least some of the modules / units may be implemented in the form of software programs, which run on a processor integrated inside the chip, and the remaining (if any) modules / units may be implemented in the form of hardware such as circuits; for each device or product applied to or integrated into a chip module, each module / unit contained therein may be implemented in the form of hardware such as circuits, and different modules / units may be located in the same component (such as a chip, circuit module, etc.) or different components of the chip module, or at least some of the modules / units may be implemented in the form of software programs. The element can be implemented in the form of a software program, which runs on the processor integrated inside the chip module, and the remaining (if any) modules / units can be implemented in the form of hardware such as circuits; for various devices and products applied to or integrated in the terminal, the various modules / units contained therein can all be implemented in the form of hardware such as circuits, and different modules / units can be located in the same component (for example, chip, circuit module, etc.) or different components in the terminal, or, at least some modules / units can be implemented in the form of a software program, which runs on the processor integrated inside the terminal, and the remaining (if any) modules / units can be implemented in the form of hardware such as circuits.

[0075] An embodiment of the present invention also provides a computer-readable storage medium, which is a non-volatile storage medium or a non-transient storage medium, on which a computer program is stored. When the computer program is run by a processor, the steps of the implant virtual placement interaction method provided in any of the above embodiments are executed.

[0076] An embodiment of the present invention also provides another implant virtual placement interaction device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor runs the computer program, it executes the steps of the implant virtual placement interaction method provided in any of the above embodiments.

[0077] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, which may include: ROM, RAM, disk or CD, etc.

[0078] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the scope defined by the claims.

Claims

1. A virtual implant placement interaction method, applied to brain-computer interface, characterized in that: include: Obtain a three-dimensional model of the cerebral cortex; In response to the received operation instruction, updating the position of the implant and obtaining the real-time three-dimensional coordinates of the implant; The implant is placed in the three-dimensional model of the cerebral cortex, or on the surface of the three-dimensional model of the cerebral cortex; re-rendering the three-dimensional model of the cerebral cortex based on the real-time three-dimensional coordinates of the implant; Output the re-rendered 3D model of the cerebral cortex.

2. The implant virtual placement interaction method according to claim 1, wherein: The re-rendering of the three-dimensional model of the cerebral cortex based on the real-time three-dimensional coordinates of the implant comprises: Obtaining real-time three-dimensional coordinates corresponding to key points on the implant; Based on the real-time three-dimensional coordinates corresponding to the key points, querying anatomical partition ROI data to determine the target brain region of the key points in the cerebral cortex; Acquiring electrophysiological data corresponding to the target brain area; Based on the electrophysiological data corresponding to the target brain area, the visual attributes of the key point are adjusted; the visual attributes of the key point include at least one of the following: the size of the key point, the color of the key point, and the brightness of the key point.

3. The implant virtual placement interaction method according to claim 1, wherein: The implant is an invasive implant that penetrates the cerebral cortex; the method further comprises: In response to receiving the cross-sectional view generation instruction, acquiring a three-dimensional trajectory of the implant; Performing virtual cutting along the three-dimensional trajectory to obtain intersection points of each anatomical region ROI passed by the three-dimensional trajectory; A two-dimensional cross-sectional diagram is generated, which includes: the trajectory of the implant displayed in a one-dimensional linear manner, each layer of ROI passed by the implant, and the starting and ending depths of the implant passing through each layer of ROI; different layers of ROI passed by the implant are marked with different colors, and the starting and ending depths of the implant passing through each layer of ROI are marked with labels.

4. The implant virtual placement interaction method according to claim 3, wherein: Also includes: The two-dimensional cross-sectional view is output.

5. The implant virtual placement interaction method according to claim 1, wherein: The operation instruction is a movement instruction output by the human-computer interaction device for moving the implant; the updating of the position of the implant and obtaining the real-time three-dimensional coordinates of the implant include: Obtaining the actual displacement corresponding to the movement instruction; Based on the actual displacement, obtaining a target displacement of the implant on the three-dimensional model of the cerebral cortex; Based on the target displacement, the real-time three-dimensional coordinates of the implant are obtained.

6. The implant virtual placement interaction method according to claim 1, wherein: The operation instruction is a movement instruction output by the human-computer interaction device for moving the implant; the updating of the position of the implant and obtaining the real-time three-dimensional coordinates of the implant include: Obtaining a three-dimensional convex hull of the three-dimensional model of the cerebral cortex to generate an agent interaction surface; Acquire the actual displacement corresponding to the movement instruction on the agent interaction surface; Projecting the final displacement on the agent interaction surface onto the three-dimensional model of the cerebral cortex to obtain a target displacement of the implant on the three-dimensional model of the cerebral cortex; Based on the target displacement, the real-time three-dimensional coordinates of the implant are obtained.

7. The implant virtual placement interaction method according to claim 1, wherein: The step of obtaining a three-dimensional model of the cerebral cortex comprises: Performing standard preprocessing and multimodal registration on the acquired image data to generate a three-dimensional model file and an anatomical partition file corresponding to the image data; the image data includes at least one of the following: anatomical image data, implant image data; A three-dimensional model of the cerebral cortex is constructed based on the three-dimensional model file and the anatomical partition file.

8. An implant virtual placement interaction device, used in brain-computer interface, characterized in that: include: A model acquisition unit, used for acquiring a three-dimensional model of the cerebral cortex; a three-dimensional coordinate acquisition unit, configured to update the position of an implant in response to a received operation instruction and acquire real-time three-dimensional coordinates of the implant; the implant is placed in the three-dimensional model of the cerebral cortex, or on the surface of the three-dimensional model of the cerebral cortex; a rendering unit, configured to re-render the three-dimensional model of the cerebral cortex based on the real-time three-dimensional coordinates of the implant; The output unit is used to output the re-rendered three-dimensional model of the cerebral cortex.

9. An implant virtual placement interaction device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor runs the computer program, the processor performs the steps of the implant virtual placement interaction method according to any one of claims 1 to 7.

Images