Multi-mode intelligent prostate resection operation mechanical arm control method and system

Abstract

The invention relates to a multi-mode intelligent prostate resection operation mechanical arm control method and system, and is applied to the technical field of artificial intelligence medical treatment. The method comprises the steps that preoperative multi-mode image data and labeling rules of a target person are obtained; constructing a three-dimensional operation map based on the preoperative multi-modal image data and the labeling rule; performing operation safety boundary division on the three-dimensional operation map based on a preset magnetic field model, and generating a predicted execution path; controlling an operation mechanical arm to reach an operation position based on the preset magnetic field measurement model and the predicted execution path; acquiring a femtosecond laser scanning result of the operation position; and operation control is conducted on the operation mechanical arm based on the femtosecond laser scanning result. The surgical instrument has the effect that surgical operation can be safely and accurately carried out.

Description

Technical Field

[0001] The present application relates to the technical field of artificial intelligence medicine, and in particular to a multimodal intelligent prostatectomy operation robot arm control method and system. Background Art

[0002] In the traditional surgical treatment of benign prostatic hyperplasia, transurethral transurethral resection of the prostate is widely used as a traditional procedure. However, it relies on high-frequency electric current to generate thermal effects to remove tissue, which carries the risk of deep tissue damage caused by heat diffusion. In addition, the surgeon's experience must be relied upon to identify the boundaries of the neurovascular bundles during the operation, resulting in a low operational tolerance and difficulty in protecting function.

[0003] Existing laser surgical systems use photothermal effects instead of electrocautery, which can reduce some thermal damage. However, existing laser surgical robotic arms rely on preoperative image registration and preset paths, lacking intraoperative multimodal real-time feedback capabilities. This makes it difficult to distinguish the molecular characteristics of neurovascular bundles from proliferative tissue. Furthermore, the navigation system has insufficient spatial resolution and limited dynamic compensation capabilities, making it unable to adaptively handle complex intraoperative events and difficult to achieve closed-loop precision control. Therefore, a technology that can safely and accurately perform surgical operations is urgently needed. Summary of the Invention

[0004] In order to enable safe and accurate surgical operations, the present application provides a multi-modal intelligent prostatectomy operation robotic arm control method and system.

[0005] In a first aspect, the present application provides a multi-modal intelligent prostatectomy operation robot arm control method, which adopts the following technical solutions:

[0006] A multi-modal intelligent prostatectomy operation robot arm control method, comprising:

[0007] Obtain preoperative multimodal imaging data and annotation rules of the target person;

[0008] constructing a three-dimensional surgical map based on the preoperative multimodal imaging data and the annotation rules;

[0009] Demarcating the three-dimensional surgical map based on a preset magnetic field model to determine the safety boundaries of the operation and generating an estimated execution path;

[0010] Controlling the robot arm to reach the surgical position based on the preset magnetic field measurement model and the predicted execution path;

[0011] Obtaining a femtosecond laser scanning result of the surgical position;

[0012] The operation of the operating robot arm is controlled based on the femtosecond laser scanning result.

[0013] By adopting the above technical solution, a three-dimensional surgical map is constructed based on the collected preoperative multimodal imaging data and annotation rules. Before the operation, the physical condition of the target person and the situation in which the operation needs to be performed are determined, and the preset length measurement model is used to divide the safe boundary where the operation can be performed. The expected execution path for controlling the operating robot arm to reach the operating position is obtained, and the operating robot arm is controlled to safely reach the surgical position where the operation needs to be performed according to the expected execution path. The surgical position is scanned using a femtosecond laser to obtain a scanning result containing the actual situation of the surgical position. The operating robot arm is controlled to perform the surgical operation according to the femtosecond laser scanning result. It can directly and accurately reach the surgical position and perform the operation according to the actual result, thereby realizing the demand for safe and accurate surgical operations.

[0014] Optionally, constructing a three-dimensional surgical map based on the preoperative multimodal imaging data and the annotation rules includes:

[0015] performing denoising and registration processing on the preoperative multimodal image data to generate standard multimodal image data;

[0016] Mapping key tissue information in the standard multimodal imaging data into the same three-dimensional grid to obtain three-dimensional image data;

[0017] Key areas of the three-dimensional image data are annotated and the entire image is rendered based on the annotation rules to construct a three-dimensional surgical map.

[0018] Optionally, dividing the three-dimensional surgical map into operation safety boundaries based on a preset magnetic field model to generate an estimated execution path includes:

[0019] Obtaining the coordinates of the surgical operation location;

[0020] determining an operation risk area and a target operation area based on the three-dimensional surgical map;

[0021] Dividing the operation risk area based on the preset magnetic field model and the surgical operation position coordinates to generate an operation safety boundary;

[0022] An estimated execution path is generated based on the preset magnetic field model, the target operation area, the surgical operation position coordinates, and the operation safety boundary.

[0023] Optionally, controlling the manipulator to reach the surgical position based on the preset magnetic field measurement model and the predicted execution path includes:

[0024] Obtaining navigation impact information and execution delay information of the operating manipulator;

[0025] Calculating an interference offset value based on the navigation impact information and the preset magnetic field model;

[0026] Adjusting the expected execution path in real time based on the interference offset value and the execution delay information to generate a real-time execution path;

[0027] The preset magnetic field model guides and controls the operating robot arm to reach the surgical position based on the real-time execution path.

[0028] Optionally, the operating control of the operating robot arm based on the femtosecond laser scanning result includes:

[0029] Performing tissue analysis on the femtosecond laser scanning results to determine normal areas, dangerous areas, surgical operation areas, and regional data of the surgical operation areas;

[0030] Marking the surgical operation area and the dangerous area based on the marking rule to generate a marked operation area and a marked dangerous area;

[0031] determining an operating blue laser power of the operating robot arm based on the area data;

[0032] determining an operation execution position of the operating robot arm based on the marked operation area, the marked dangerous area, and the operating blue laser power;

[0033] The operation robot arm is controlled based on the operation execution position and the operation blue laser power.

[0034] Optionally, after the operation of the operating robot arm is controlled based on the femtosecond laser scanning result, the method further includes:

[0035] Acquiring the target person's operating position and body data in real time;

[0036] Determining whether data adjustment is required based on the body data of the operating position;

[0037] If data adjustment is required, determining target adjustment parameters based on the body data at the operating position;

[0038] performing control operation adjustment on the operating robot arm based on the target adjustment parameter;

[0039] If data adjustment is not required, the step of acquiring the operation data of the operating robot arm and the operation position body data of the target person in real time is repeated.

[0040] Optionally, the method further includes:

[0041] Acquiring full-process operation data of the operating robotic arm and postoperative physical data of the target patient;

[0042] A surgical operation report is generated based on the entire operation data and the postoperative body data.

[0043] In a second aspect, the present application provides a multi-modal intelligent prostatectomy operation robot arm control system, which adopts the following technical solutions:

[0044] A multi-modal intelligent prostatectomy operation robot arm control system, comprising:

[0045] Data rule acquisition module, used to obtain preoperative multimodal imaging data and annotation rules of the target person;

[0046] A surgical map construction module, configured to construct a three-dimensional surgical map based on the preoperative multimodal imaging data and the annotation rules;

[0047] An estimated path generation module, configured to divide the three-dimensional surgical map into operation safety boundaries based on a preset magnetic field model and generate an estimated execution path;

[0048] A mechanical execution control module, configured to control the operation of the robotic arm to reach the surgical position based on the preset magnetic field measurement model and the predicted execution path;

[0049] A scanning result acquisition module, used to acquire the femtosecond laser scanning result of the surgical position;

[0050] A mechanical operation control module is used to control the operation of the operating robot arm based on the femtosecond laser scanning result.

[0051] By adopting the above technical solution, a three-dimensional surgical map is constructed based on the collected preoperative multimodal imaging data and annotation rules. Before the operation, the physical condition of the target person and the situation in which the operation needs to be performed are determined, and the preset length measurement model is used to divide the safe boundary where the operation can be performed. The expected execution path for controlling the operating robot arm to reach the operating position is obtained, and the operating robot arm is controlled to safely reach the surgical position where the operation needs to be performed according to the expected execution path. The surgical position is scanned using a femtosecond laser to obtain a scanning result containing the actual situation of the surgical position. The operating robot arm is controlled to perform the surgical operation according to the femtosecond laser scanning result. It can directly and accurately reach the surgical position and perform the operation according to the actual result, thereby realizing the demand for safe and accurate surgical operations.

[0052] In a third aspect, the present application provides an electronic device, which adopts the following technical solution:

[0053] An electronic device comprising a processor coupled to a memory;

[0054] The processor is used to execute the computer program stored in the memory, so that the electronic device executes the computer program of the multimodal intelligent prostate resection operation robot arm control method described in any one of the first aspects.

[0055] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution:

[0056] A computer-readable storage medium stores a computer program that can be loaded by a processor and executes the multimodal intelligent prostatectomy operation robot arm control method described in any one of the first aspects.

[0057] In summary, this application includes at least one of the following beneficial technical effects:

[0058] A three-dimensional surgical map is constructed based on the collected preoperative multimodal imaging data and annotation rules. Before the operation, the physical condition of the target person and the conditions for operation are determined, and the preset length measurement model is used to divide the safe boundary for operation. The expected execution path for controlling the operating robot arm to reach the operating position is obtained. According to the expected execution path, the operating robot arm is controlled to safely reach the surgical position where the operation is required. The surgical position is scanned using a femtosecond laser to obtain a scanning result containing the actual situation of the surgical position. The operating robot arm is controlled to perform the surgical operation according to the femtosecond laser scanning result. It can directly and accurately reach the surgical position and perform the operation according to the actual result, thereby realizing the need for safe and accurate surgical operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 This is a flow chart of a multi-modal intelligent prostatectomy operation robotic arm control method provided in an embodiment of the present application.

[0060] Figure 2 This is a structural block diagram of a multimodal intelligent prostatectomy operation robotic arm control system provided in an embodiment of the present application.

[0061] Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] The present application is further described in detail below with reference to the accompanying drawings.

[0063] The present application provides a multimodal intelligent prostatectomy robot arm control method. This multimodal intelligent prostatectomy robot arm control method can be executed by an electronic device, which can be a server or a terminal device. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smartphone, a tablet computer, a desktop computer, etc., but is not limited thereto.

[0064] Figure 1 A flowchart of a multi-modal intelligent prostatectomy operation robotic arm control method provided in an embodiment of the present application.

[0065] like Figure 1 As shown, the main process of the method is described as follows (steps S101 to S106):

[0066] Step S101: Acquire preoperative multimodal imaging data and labeling rules of a target person.

[0067] In this embodiment, the preoperative multimodal imaging data of the target person includes MRI-DWI for nerve bundle visualization to display nerve distribution, CT angiography for displaying vascular distribution, and ultrasound elastography for displaying tissue hardness. The labeling rule is to use different colors to label each area to avoid safety impacts during the surgical operation. The specific labeling method can be to set red, blue, and green areas, where the red area is used to indicate high-risk blood vessels, the blue area is used to indicate nerve bundles, and the green area is used to indicate safe vaporization, that is, the green area is a safe area where surgical operations can be performed. It should be noted that the multimodal imaging data can also be increased according to actual operational needs, and the different area divisions and corresponding colors in the labeling rules also need to be adjusted according to actual needs, which is not specifically limited here.

[0068] Step S102: constructing a three-dimensional surgical map based on preoperative multimodal imaging data and annotation rules.

[0069] For step S102, the preoperative multimodal imaging data is denoised and registered to generate standard multimodal imaging data; the key tissue information in the standard multimodal imaging data is mapped to the same three-dimensional grid to obtain three-dimensional image data; the three-dimensional image data is annotated with key areas and rendered as a whole based on annotation rules to construct a three-dimensional surgical map.

[0070] In this embodiment, in order to ensure the accuracy of constructing a three-dimensional surgical map, after obtaining the preoperative multimodal image data, all the obtained multimodal image data are subjected to denoising and registration processing, that is, the format and resolution of all the multimodal image data are adjusted to unify the format of the multimodal image data, and enhance the resolution of the multimodal image data to improve the clarity of the multimodal image data. After the format and resolution are adjusted, the adjusted multimodal image data also needs to be oriented so that it is in the right front upper RAS coordinate system when displayed and used, thereby obtaining standard multimodal image data.

[0071] Because multimodal imaging data contains different types of impact data for displaying different information, when constructing three-dimensional image data, it is necessary to map the blood vessels, nerve bundles, and ultrasound elasticity values ​​in the standard multimodal imaging data to the same three-dimensional grid, that is, the blood vessels, nerve bundles, and ultrasound elasticity values ​​can be displayed simultaneously in the same three-dimensional image. In order to facilitate identification and viewing, the three-dimensional image data is annotated and rendered using annotation rules. According to the above annotation example, a three-dimensional image data map with color prompts is generated, and further detailed processing is performed. For the ultrasound elasticity value part, different colors are used to distinguish different degrees of soft and hard values, and the prostate contour is divided with a translucent gray curve. After the annotation is completed, a three-dimensional surgical map is obtained. It should be noted that the annotation color of the soft and hard values ​​needs to be set according to actual needs and is not specifically limited here.

[0072] Step S103 : dividing the three-dimensional surgical map into operation safety boundaries based on the preset magnetic field model to generate an estimated execution path.

[0073] For step S103, the coordinates of the surgical operation position are obtained; the operation risk area and the target operation area are determined based on the three-dimensional surgical map; the operation risk area is divided based on the preset magnetic field model and the surgical operation position coordinates, and the operation safety boundary is generated; and the expected execution path is generated based on the preset magnetic field model, the target operation area, the surgical operation position coordinates and the operation safety boundary.

[0074] In this embodiment, a 3D surgical map with annotated and rendered information is used to determine a dangerous operation area and a target operation area. The dangerous operation area includes dense nerve bundles and high-risk blood vessels, and the target operation area is defined as the prostate contour area delineated using a semi-transparent gray curve. After delineation, if the dangerous operation area encloses or is immediately adjacent to the target operation area, further delineation of a safe operation boundary is required to ensure that the robotic arm can safely reach the target operation area and perform safe operations. The dangerous operation area is further delineated using a preset magnetic field model and the coordinates of the surgical operation position to create a safe operation boundary. During this delineation, the area with a distance between nerve bundles and blood vessels greater than 3 mm is defined as a safe area for the robotic arm to traverse. The coordinates of the surgical operation position are used as the starting point, and the operation area coordinates of the target operation area are determined based on the surgical operation position coordinates. The operation area coordinates are then used as the end point. Based on the aforementioned safety area criteria and the start and end point coordinates, the preset magnetic field model automatically delineates the safe operation boundary. All safe operation boundaries are then connected along the shortest path, connecting the start and end points, thereby generating a predicted execution path. It should be noted that the preset magnetic field model can be a magnetic field model that uses the earth's magnetic field or an artificially generated magnetic field as a reference signal, senses magnetic field strength, direction and other information through sensors, and combines algorithms to analyze position and posture, thereby realizing navigation. It can also be other models that can achieve the same function by other means. The specific preset magnetic field model is not specifically limited here.

[0075] Step S104 : controlling the robot arm to reach the surgical position based on the preset magnetic field measurement model and the predicted execution path.

[0076] For step S104, obtain navigation impact information and execution delay information of the operating robot arm; calculate the interference offset value based on the navigation impact information and the preset magnetic field model; adjust the expected execution path in real time based on the interference offset value and the execution delay information to generate a real-time execution path; the preset magnetic field model guides the control operation robot arm to reach the surgical position based on the real-time execution path.

[0077] In this embodiment, metal products present during the operation may affect the accuracy of the magnetic field to a certain extent. After the preset execution path is established, it is necessary to adjust the preset execution path in real time based on the navigation impact information and execution delay information to ensure operational safety. During the adjustment, an interference offset value must first be calculated based on the navigation impact information and the preset magnetic field model. The navigation impact information includes the type and number of operating instruments in the operating environment, the model of the manipulator arm, etc. The preset magnetic field model calculates the interference offset value based on the navigation impact information. The interference offset value is the possible offset of the manipulator arm during movement in the presence of the navigation impact information. The execution delay information is the time difference between the manipulator arm receiving a signal and responding. To ensure that the manipulator arm can be safely and accurately controlled to the surgical location, the preset execution path is adjusted in real time using the execution delay information and the interference offset value. Specifically, the manipulator arm is guided and controlled according to the expected execution path. If the manipulator arm deviates, the preset execution path is adjusted based on the execution delay information and the interference offset value. This allows for timely correction of deviations while also preventing them in advance.

[0078] Step S105: Obtain femtosecond laser scanning results of the surgical position.

[0079] In this embodiment, after the robotic arm reaches the surgical site, a femtosecond laser is used to scan the surgical site to obtain clear tissue microstructure and biochemical characteristics to facilitate subsequent fine operations. The scanned tissue microstructure and biochemical characteristics are used as femtosecond laser scanning results.

[0080] Step S106: Control the operation of the manipulator based on the femtosecond laser scanning result.

[0081] For step S106, the femtosecond laser scanning results are subjected to tissue analysis to determine the normal area, dangerous area, surgical operation area and regional data of the surgical operation area; the surgical operation area and the dangerous area are labeled based on the labeling rules to generate labeled operation areas and labeled dangerous areas; the operating blue laser power of the operating robot arm is determined based on the regional data; the operation execution position of the operating robot arm is determined based on the labeled operation area, the labeled dangerous area and the operation blue laser power; the operating robot arm is operationally controlled based on the operation execution position and the operation blue laser power.

[0082] In this embodiment, tissue analysis is performed based on the results of femtosecond laser scanning to determine the regional situation of the prostate position in the surgical position, the normal area with no problems, the dangerous area with dense distribution of blood vessels and nerve bundles, and the surgical operation area requiring surgical treatment, and the regional data of the surgical operation area, namely the regional size, tissue hardness and tissue thickness of the surgical operation area, are determined. The surgical operation area and the dangerous area are marked again using the marking rules to facilitate the avoidance of the dangerous area during the operation and reduce the possibility of operation omissions during the operation.

[0083] The operating blue laser power of the operating robot arm during surgery is determined based on the regional data. In a marked operating area, there is not only one corresponding operating blue laser power. Since the tissue hardness and thickness at different positions in the marked operating area are different, there are also multiple different operating blue laser powers. When determining the operation execution position, it is necessary to jointly decide based on the marked operating area, the marked dangerous area and the operating blue laser power, that is, to avoid the marked dangerous area, and to correspond the operating blue laser power to the different positions of the marked operating area one by one to obtain the final operation execution position, thereby controlling the operation of the operating robot arm.

[0084] In this embodiment, the operating position body data of the target person is acquired in real time; whether data adjustment is required is determined based on the operating position body data; if data adjustment is required, target adjustment parameters are determined based on the operating position body data; the operating robot arm is controlled and adjusted based on the target adjustment parameters; if data adjustment is not required, the steps of acquiring the operating data of the operating robot arm and the operating position body data of the target person in real time are repeated.

[0085] In order to ensure the safety of the operation, it is necessary to collect the physical data of the target person's operating position in real time to determine whether there is a bleeding point or the laser intensity is not matched. If so, it will be determined whether the data needs to be adjusted, and the target adjustment parameters will be further determined according to the actual situation. If there is a bleeding point, it is necessary to switch to the green laser in time to stop the bleeding point. If there is a laser intensity mismatch, the laser power will be increased or decreased according to the current area data and the operating blue laser power. The specific adjustment needs to be set according to the actual situation and is not specifically limited here.

[0086] In this embodiment, the entire operation data of the operating robot arm and the postoperative physical data of the target patient are acquired; and a surgical operation report is generated based on the entire operation data and the postoperative physical data.

[0087] After the surgical operation is completed, the overall operation needs to be reviewed and analyzed to find imperfections and areas that need to be maintained, so as to collect the entire operation data of the robotic arm and postoperative physical data, and generate a surgical operation report to facilitate staff to conduct further analysis and processing based on the surgical operation report.

[0088] Figure 2 A structural block diagram of a multi-modal intelligent prostatectomy operation robot arm control system 200 provided in an embodiment of the application.

[0089] like Figure 2 As shown, the multimodal intelligent prostatectomy operation robot arm control system 200 mainly includes:

[0090] The data rule acquisition module 201 is used to obtain the preoperative multimodal imaging data and annotation rules of the target person;

[0091] A surgical map construction module 202 is used to construct a three-dimensional surgical map based on preoperative multimodal imaging data and annotation rules;

[0092] The estimated path generation module 203 is used to divide the three-dimensional surgical map into operation safety boundaries based on the preset magnetic field model and generate an estimated execution path;

[0093] A mechanical execution control module 204 is used to control the operation of the robotic arm to reach the surgical position based on a preset magnetic field measurement model and an expected execution path;

[0094] Scan result acquisition module 205, used to acquire femtosecond laser scanning results of the surgical location;

[0095] The mechanical operation control module 206 is used to control the operation of the manipulator based on the femtosecond laser scanning result.

[0096] As an optional implementation of this embodiment, the surgical map construction module 202 is specifically used to perform denoising and registration processing on preoperative multimodal imaging data to generate standard multimodal imaging data; map key tissue information in the standard multimodal imaging data to the same three-dimensional grid to obtain three-dimensional image data; and annotate key areas and render the entire three-dimensional image data based on annotation rules to construct a three-dimensional surgical map.

[0097] As an optional implementation of this embodiment, the expected path generation module 203 is specifically used to obtain the coordinates of the surgical operation position; determine the operation risk area and the target operation area based on the three-dimensional surgical map; divide the operation risk area based on the preset magnetic field model and the surgical operation position coordinates to generate an operation safety boundary; and generate an expected execution path based on the preset magnetic field model, the target operation area, the surgical operation position coordinates and the operation safety boundary.

[0098] As an optional implementation of this embodiment, the mechanical execution control module 204 is specifically used to obtain navigation impact information and execution delay information of the operating robot arm; calculate the interference offset value based on the navigation impact information and the preset magnetic field model; adjust the expected execution path in real time based on the interference offset value and the execution delay information to generate a real-time execution path; the preset magnetic field model guides the control operation of the robot arm to reach the surgical position based on the real-time execution path.

[0099] As an optional implementation of this embodiment, the mechanical operation control module 206 is specifically used to perform tissue analysis on the femtosecond laser scanning results, determine the normal area, dangerous area, surgical operation area and regional data of the surgical operation area; mark the surgical operation area and dangerous area based on the marking rules, and generate marked operation areas and marked dangerous areas; determine the operating blue laser power of the operating robot arm based on the regional data; determine the operation execution position of the operating robot arm based on the marked operation area, the marked dangerous area and the operation blue laser power; and perform operation control on the operating robot arm based on the operation execution position and the operation blue laser power.

[0100] As an optional implementation of this embodiment, the multimodal intelligent prostatectomy operation robot arm control system 200 further includes:

[0101] A body data acquisition module is used to obtain the target person's operating position body data in real time;

[0102] A data adjustment judgment module is used to judge whether data adjustment is required based on the body data of the operation position;

[0103] an adjustment parameter determination module for determining a target adjustment parameter based on the operating position body data;

[0104] An operation adjustment control module, used to adjust the control operation of the operating robot arm based on the target adjustment parameters;

[0105] The data repeated acquisition module is used to repeatedly acquire the operation data of the operating robot arm and the operation position body data of the target person in real time.

[0106] As an optional implementation of this embodiment, the multimodal intelligent prostatectomy operation robot arm control system 200 further includes:

[0107] Postoperative data acquisition module, used to obtain the full operation data of the operating robot arm and the postoperative physical data of the target patient;

[0108] The operation report generation module is used to generate a surgical operation report based on the entire operation data and postoperative physical data.

[0109] In one example, the module in any of the above devices can be one or more integrated circuits configured to implement the above methods, such as: one or more application specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms.

[0110] For another example, when the modules in the device can be implemented in the form of a processing element scheduling program, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor capable of calling programs. For another example, these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0111] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0112] Figure 3 This is a structural block diagram of the electronic device 300 provided in an embodiment of the present application.

[0113] like Figure 3 As shown, the electronic device 300 includes a processor 301 and a memory 302 , and may further include an information input / information output (I / O) interface 303 , one or more communication components 304 , and a communication bus 305 .

[0114] The processor 301 is used to control the overall operation of the electronic device 300 to complete all or part of the steps of the multimodal intelligent prostatectomy robot arm control method described above. The memory 302 is used to store various types of data to support the operation of the electronic device 300. For example, this data may include instructions for any application or method operating on the electronic device 300, as well as application-related data. The memory 302 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as one or more of static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0115] The I / O interface 303 provides an interface between the processor 301 and other interface modules, which may be a keyboard, a mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 304 is used for wired or wireless communication between the electronic device 300 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more thereof, therefore, the corresponding communication component 304 may include: Wi-Fi components, Bluetooth components, NFC components.

[0116] The electronic device 300 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the multi-modal intelligent prostate resection operation robot arm control method given in the above embodiment.

[0117] Communication bus 305 may include a path for transmitting information between the aforementioned components. Communication bus 305 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, for example. Communication bus 305 may be divided into an address bus, a data bus, a control bus, and the like.

[0118] The electronic device 300 may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., as well as fixed terminals such as digital TVs, desktop computers, etc., and may also be servers, etc.

[0119] The present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the above-mentioned multi-modal intelligent prostatectomy operation robot arm control method are implemented.

[0120] The computer-readable storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., which can store program codes.

[0121] The terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0122] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of application involved in this application is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the aforementioned application concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions applied for in this application.

Claims

1. A multi-modal intelligent prostatectomy operation robot arm control method, characterized in that: include: Obtain preoperative multimodal imaging data and annotation rules of the target person; constructing a three-dimensional surgical map based on the preoperative multimodal imaging data and the annotation rules; Demarcating the three-dimensional surgical map based on a preset magnetic field model to determine the safety boundaries of the operation and generating an estimated execution path; Controlling the robot arm to reach the surgical position based on the preset magnetic field measurement model and the predicted execution path; Obtaining a femtosecond laser scanning result of the surgical position; The operation of the operating robot arm is controlled based on the femtosecond laser scanning result.

2. The method according to claim 1, characterized in that The constructing of a three-dimensional surgical map based on the preoperative multimodal imaging data and the annotation rules includes: performing denoising and registration processing on the preoperative multimodal image data to generate standard multimodal image data; Mapping key tissue information in the standard multimodal imaging data into the same three-dimensional grid to obtain three-dimensional image data; Key areas of the three-dimensional image data are annotated and the entire image is rendered based on the annotation rules to construct a three-dimensional surgical map.

3. The method according to claim 1, characterized in that Demarcating the three-dimensional surgical map based on the preset magnetic field model to generate an estimated execution path includes: Obtaining the coordinates of the surgical operation location; determining an operation risk area and a target operation area based on the three-dimensional surgical map; Dividing the operation risk area based on the preset magnetic field model and the surgical operation position coordinates to generate an operation safety boundary; An estimated execution path is generated based on the preset magnetic field model, the target operation area, the surgical operation position coordinates, and the operation safety boundary.

4. The method according to claim 1, wherein The controlling the manipulator to reach the surgical position based on the preset magnetic field measurement model and the predicted execution path includes: Obtaining navigation impact information and execution delay information of the operating manipulator; Calculating an interference offset value based on the navigation impact information and the preset magnetic field model; Adjusting the expected execution path in real time based on the interference offset value and the execution delay information to generate a real-time execution path; The preset magnetic field model guides and controls the operating robot arm to reach the surgical position based on the real-time execution path.

5. The method according to claim 4, characterized in that The operating control of the operating robot arm based on the femtosecond laser scanning result includes: Performing tissue analysis on the femtosecond laser scanning results to determine normal areas, dangerous areas, surgical operation areas, and regional data of the surgical operation areas; Marking the surgical operation area and the dangerous area based on the marking rule to generate a marked operation area and a marked dangerous area; determining an operating blue laser power of the operating robot arm based on the area data; determining an operation execution position of the operating robot arm based on the marked operation area, the marked dangerous area, and the operating blue laser power; The operation robot arm is controlled based on the operation execution position and the operation blue laser power.

6. The method according to claim 5, characterized in that After the operation of the operating robot arm is controlled based on the femtosecond laser scanning result, the method further includes: Acquiring the target person's operating position and body data in real time; Determining whether data adjustment is required based on the body data of the operating position; If data adjustment is required, determining target adjustment parameters based on the body data at the operating position; performing control operation adjustment on the operating robot arm based on the target adjustment parameter; If data adjustment is not required, the step of acquiring the operation data of the operating robot arm and the operation position body data of the target person in real time is repeated.

7. The method according to claim 6, characterized in that The method further comprises: Acquiring full-process operation data of the operating robotic arm and postoperative physical data of the target patient; A surgical operation report is generated based on the entire operation data and the postoperative body data.

8. A multi-modal intelligent prostatectomy operation robot arm control system, characterized in that: include: Data rule acquisition module, used to obtain preoperative multimodal imaging data and annotation rules of the target person; A surgical map construction module, configured to construct a three-dimensional surgical map based on the preoperative multimodal imaging data and the annotation rules; An estimated path generation module, configured to divide the three-dimensional surgical map into operation safety boundaries based on a preset magnetic field model and generate an estimated execution path; A mechanical execution control module, configured to control the operation of the robotic arm to reach the surgical position based on the preset magnetic field measurement model and the predicted execution path; A scanning result acquisition module, used to acquire the femtosecond laser scanning result of the surgical position; A mechanical operation control module is used to control the operation of the operating robot arm based on the femtosecond laser scanning result.

9. An electronic device, characterized in that: comprising a processor coupled to a memory; The processor is configured to execute the computer program stored in the memory, so that the electronic device performs the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The method comprises a computer program or an instruction, which, when executed on a computer, causes the computer to execute the method according to any one of claims 1 to 7.

Images