Data remote transmission method, medical equipment and data remote transmission device
By calculating the target transformation matrix and controlling the adjustment arm movement in the processing module of the medical device, the problem of poor interactive data transmission in remote surgery is solved, comprehensive remote guidance for local surgery is achieved, and real-time and accuracy of remote surgery is improved. sex.
Patent Information
- Application Number
- CN202311054983.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-18
- Publication Date
- 2025-02-21
AI Technical Summary
The prior art has poor interactiveness in remote data transmission in remote surgery, making it difficult to achieve comprehensive, sufficient and meticulous remote guidance for local surgery.
By obtaining scan data and point cloud data in the processing module of the medical device, and based on these data and the pose parameters of the adjustment arm, the target transformation matrix is calculated. Then, according to the designated points and target transformation matrix transmitted by the remote terminal, the target point is determined, and the adjustment arm movement is controlled, and the target image data is collected and transmitted to the remote terminal.
It improves the interactivity between remote terminals and medical equipment, realizes comprehensive, sufficient and meticulous remote guidance for local surgery, and enhances the real-time and accuracy of remote surgery.
Smart Images
Figure CN119495411A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of medical devices, and in particular to a data remote transmission method, medical equipment and a data remote transmission device. Background Art
[0002] With the development of medical technology, remote surgery has emerged, which often requires experts to provide remote guidance to doctors in the operating room. At present, in order to meet the needs of remote surgery, it is often necessary to design an audio and video trolley, which can realize real-time audio and video transmission. By setting up a computer host at the remote end and connecting it to the local audio and video trolley through the network, the function of communication between the remote expert and the local doctor can be realized.
[0003] However, the current remote data transmission methods or traditional methods have the problem of poor interactivity. Summary of the invention
[0004] Based on this, it is necessary to provide a data remote transmission method, medical equipment, data remote transmission device and computer equipment that can improve interactivity in response to the above technical problems.
[0005] In a first aspect, the present application provides a method for remote data transmission, which is applied to a processing module of a medical device; the medical device further comprises an adjustment arm and an operation field acquisition module provided on the adjustment arm; the processing module is connected to the operation field acquisition module and the adjustment arm, and is also used to connect to a remote terminal; the method comprises:
[0006] Obtain the scanning data collected by the scanning device for the target part and transmit it to the remote terminal, and obtain the point cloud data collected by the surgical field acquisition module for the target part;
[0007] Based on the scanning data, the point cloud data and the posture parameters of the adjustment arm, a target transformation matrix is obtained; the target transformation matrix is a transformation matrix between the image file coordinate system corresponding to the scanning data and the base coordinate system of the medical device;
[0008] Obtain the designated point for the scan data transmitted by the remote terminal, and obtain the target point of the target part according to the designated point and the target transformation matrix;
[0009] The movement of the adjustment arm is controlled according to the target point, and the target image data collected by the surgical field acquisition module for the target point is transmitted to the remote terminal.
[0010] In one embodiment, a target transformation matrix is obtained based on the scan data, the point cloud data and the posture parameters of the adjustment arm, including:
[0011] Based on the scanning data and the point cloud data, a first transformation matrix between the surgical field acquisition module and the image file coordinate system is obtained;
[0012] According to the posture parameters of the adjustment arm and the first transformation matrix, a target transformation matrix is obtained.
[0013] In one embodiment, a target transformation matrix is obtained based on the scan data, the point cloud data and the posture parameters of the adjustment arm, including:
[0014] According to the posture parameters of the adjustment arm and the point cloud data, the target data of the target part in the base coordinate system is obtained;
[0015] According to the target data and the scan data, the target transformation matrix is obtained.
[0016] In one embodiment, based on the scanning data and the point cloud data, a first transformation matrix between the surgical field acquisition module and the image file coordinate system is obtained, including:
[0017] Based on the scan data, a first coordinate set of the target part is obtained; the first coordinate set includes the coordinates of each feature point of the target part in the coordinate system of the image file; according to the point cloud data, a second coordinate set of the target part is obtained; the second coordinate set includes the coordinates of each feature point in the coordinate system of the surgical field acquisition module;
[0018] Based on the first coordinate set and the second coordinate set, a first transformation matrix is obtained.
[0019] In a second aspect, the present application provides a medical device, the medical device comprising a processing module, an adjustment arm, and a surgical field acquisition module disposed on the adjustment arm; the processing module is connected to the surgical field acquisition module and the adjustment arm, and is also used to connect to a remote terminal;
[0020] The processing module is used to obtain the scanning data collected for the target part and transmit it to the remote terminal, and obtain the point cloud data collected for the target part by the surgical field acquisition module; based on the scanning data, the point cloud data and the posture parameters of the adjustment arm, the target transformation matrix is obtained; the target transformation matrix is the transformation matrix between the image file coordinate system corresponding to the scanning data and the base coordinate system of the medical device; the designated point for the scanning data transmitted by the remote terminal is obtained, and the target point of the target part is obtained according to the designated point and the target transformation matrix; the movement of the adjustment arm is controlled according to the target point, and the target image data collected by the surgical field acquisition module for the target point is transmitted to the remote terminal.
[0021] In one embodiment, the processing module is further used to connect to an electrophysiological device to obtain electrophysiological data collected by the electrophysiological device; the medical device further includes:
[0022] A panoramic acquisition module, which is connected to the processing module and is used to acquire panoramic image data;
[0023] A display module is connected to the processing module and is used for displaying based on at least one of the electrophysiological data, the panoramic image data, the scanning data, the point cloud data and the target image data.
[0024] In one embodiment, the medical device further comprises:
[0025] A laser indicating module, which is fixedly connected to the surgical field acquisition module; or
[0026] The laser indication module is connected to the surgical field acquisition module through at least two joints; the joints are connected to the processing module, and the joints move under the control of the processing module or the remote terminal.
[0027] In one embodiment, the medical device further comprises:
[0028] The audio acquisition module is connected to the processing module, and the processing module obtains the audio data collected by the audio acquisition module and transmits it to the remote terminal.
[0029] In a third aspect, the present application provides a data remote transmission device, wherein the control device is applied to a processing module of a medical device; the medical device further comprises an adjustment arm, and an operation field acquisition module provided on the adjustment arm; the processing module is connected to the operation field acquisition module and the adjustment arm, and is also used to connect to a remote terminal; the control device comprises:
[0030] A data acquisition module is used to obtain scanning data collected from the target part and transmit it to a remote terminal, and to obtain point cloud data collected from the target part by a surgical field acquisition module;
[0031] A matrix acquisition module is used to obtain a target transformation matrix based on the scan data, the point cloud data and the posture parameters of the adjustment arm; the target transformation matrix is a transformation matrix between the image file coordinate system corresponding to the scan data and the base coordinate system of the medical device;
[0032] A target determination module is used to obtain a designated point for the scan data transmitted by the remote terminal, and obtain a target point of the target part according to the designated point and the target transformation matrix;
[0033] The acquisition execution module is used to control the movement of the adjustment arm according to the target point, and transmit the target image data acquired by the surgical field acquisition module for the target point to the remote terminal.
[0034] In a fourth aspect, the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.
[0035] The above-mentioned data remote transmission method, medical equipment, data remote transmission device and computer equipment, the method is applied to the processing module of the medical equipment; the medical equipment also includes an adjustment arm, and a surgical field acquisition module arranged on the adjustment arm; the processing module connects the surgical field acquisition module and the adjustment arm, and is also used to connect to the remote terminal; the method includes: obtaining scanning data collected for the target part and transmitting it to the remote terminal for the doctor at the remote end to review, and obtaining point cloud data collected for the target part by the surgical field acquisition module; based on the scanning data, point cloud data and the posture parameters of the adjustment arm, obtaining the target transformation matrix, the target transformation matrix is the transformation matrix between the image file coordinate system corresponding to the scanning data and the base coordinate system of the medical equipment, and the target transformation matrix can be used for the conversion between the base coordinate system and the image file coordinate system; obtaining the specified point for the scanning data transmitted by the remote terminal, and according to the specified point and the target transformation matrix, the specified point can be converted into the target point of the target part; controlling the movement of the adjustment arm according to the target point, transmitting the target image data collected by the surgical field acquisition module for the target point to the remote terminal, which can improve the interactivity between the remote terminal and the medical equipment, so as to realize comprehensive, sufficient and detailed remote guidance of local surgery. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 is a schematic diagram of a surgical robot in an example;
[0037] Figure 2 is a schematic diagram of a medical device in one embodiment;
[0038] Figure 3 A schematic diagram of a flow chart of a method for remote data transmission in one embodiment;
[0039] Figure 4 A schematic diagram of a flow chart of a data remote transmission step in an embodiment;
[0040] Figure 5 A schematic diagram of a flow chart of a remote data transmission step in another embodiment;
[0041] Figure 6 A schematic diagram of a flow chart of a data remote transmission step in yet another embodiment;
[0042] Figure 7 is a display schematic diagram of a display module in one embodiment;
[0043] Figure 8 is a schematic diagram of a medical device in another embodiment;
[0044] Fig. 9 A schematic diagram of the connection between a medical device and a remote terminal in an example;
[0045] Fig.10It is a structural block diagram of a data remote transmission device in one embodiment;
[0046] Fig.11 FIG. 4 is a diagram showing the internal structure of a processing module in an embodiment. DETAILED DESCRIPTION
[0047] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0048] At present, common forms of surgical robots are as follows Figure 1 As shown, the main part is a surgical cart. When the patient lies on the operating table, the surgical cart can be fixed and the target part of the patient can be fixed. A display and a robotic arm can be set on the surgical cart. The display can be used to display the surgical workflow, and the robotic arm can be used to perform corresponding operations on the target part. However, the current surgical robot has poor interactivity in remote data transmission.
[0049] In one embodiment, a method for remote data transmission is provided. Figure 2 As shown, the method is applied to a processing module of a medical device 20; the medical device 20 also includes an adjustment arm 210, and an operation field acquisition module 220 disposed on the adjustment arm 210; the processing module connects the operation field acquisition module 220 and the adjustment arm 210, and is also used to connect to a remote terminal; Figure 3 As shown, the method includes:
[0050] Step 310, acquiring scanning data collected for the target part and transmitting it to the remote terminal, and acquiring point cloud data collected for the target part by the surgical field acquisition module 220;
[0051] Specifically, the medical device 20 may be used as follows Figure 2The trolley shown in the figure is implemented, the adjustment arm 210 can be set on the support rod on the trolley, and the adjustment arm 210 is provided with a surgical field acquisition module 220. For example, the surgical field acquisition module 220 can be set at the end of the adjustment arm 210, and the adjustment arm 210 and the surgical field acquisition module 220 are both connected to the processing module (not shown in the figure); the processing module can also import scan data, for example, the scan data can be imported by connecting a USB flash drive or a CD, or by connecting a scanning device to receive the scan data collected by the scanning device for the target part, and the scan data may include data corresponding to the image file; the processing module can also be connected to a remote terminal to realize communication with the remote terminal, for example, the received scan data is transmitted to the remote terminal, and the remote terminal can display the scan data for the remote doctor to consult. The surgical field acquisition module 220 can collect point cloud data for the target part and transmit the point cloud data to the processing module. The scanning data may be data obtained by scanning the target part before surgery, for example, image data obtained by CT (Computed Tomography) scanning; the point cloud data may include three-dimensional point clouds for the target part collected by the adjustment arm 210 and the surgical field acquisition module 220 after the medical device 20, the target part and the surgical cart are fixed. The scanning data and point cloud data for the target part obtained in the above manner can be used for coordinate system conversion between the base coordinate system of the medical device 20 and the coordinate system of the image file, so that the remote terminal can control the medical device 20 accordingly based on the received scanning data, and can improve the interactivity between the remote terminal and the medical device 20.
[0052] In some examples, the surgical field acquisition module 220 may include a surgical field camera, for example, implemented using a structured light camera, which can realize the acquisition of point cloud data. The chassis of the medical device 20 may be provided with casters to facilitate the movement of the medical device 20, and the casters may also be fixed to the ground to achieve the fixation of the position of the medical device 20.
[0053] Step 320, based on the scan data, the point cloud data and the posture parameters of the adjustment arm 210, a target transformation matrix is obtained; the target transformation matrix is a transformation matrix between the image file coordinate system corresponding to the scan data and the base coordinate system of the medical device 20;
[0054] Specifically, the scanning data for the target part may include the coordinates of each feature point of the target part in the image file coordinate system, and the image file coordinate system may be the coordinate system of the image file corresponding to the scanning data; the point cloud data for the target part may include the coordinates of each feature point in the coordinate system of the surgical field acquisition module 220, and each feature point in the scanning data may be matched with each feature point in the point cloud data; the adjustment arm 210 may include a plurality of electric joints, each electric joint may include a corresponding encoder, each encoder may be connected to a processing module, and then the processing module may obtain the posture parameters of the adjustment arm 210 according to the output signals of the encoders of each joint of the adjustment arm 210; further, the processing module may process the received scanning data, point cloud data and the posture parameters of the adjustment arm 210 to determine the target transformation matrix, and the target transformation matrix may include the target transformation matrix being the transformation matrix between the image file coordinate system corresponding to the scanning data and the base coordinate system of the medical device 20, for example, the target transformation matrix may include the transformation matrix of the image file coordinate system relative to the base coordinate system of the medical device 20. For example, based on the scan data for the target part and the point cloud data for the target part, the transformation matrix between the coordinate system of the surgical field acquisition module 220 and the image file coordinate system is obtained; based on the transformation matrix between the coordinate system of the surgical field acquisition module 220 and the image file coordinate system, and the posture parameters of the adjustment arm 210, the transformation matrix between the base coordinate system of the medical device 20 and the image file coordinate system can be obtained, and then the target transformation matrix can be determined. In the above manner, the target transformation matrix between the image file coordinate system corresponding to the scan data and the base coordinate system of the medical device 20 can be obtained, which is used to realize the remote terminal to control the medical device 20 accordingly based on the received scan data, thereby improving the interactivity between the remote terminal and the medical device 20.
[0055] Step 330, obtaining a designated point for the scan data transmitted by the remote terminal, and obtaining a target point of the target part according to the designated point and the target transformation matrix;
[0056] Specifically, the remote terminal can perform imaging display of the scanned data, for example, display the scanned image corresponding to the scanned data, and then the doctor at the remote end can select the point that he wants to further observe according to the scanned image and input it into the remote terminal, and the remote terminal can transmit the designated point for the scanned data to the processing module, wherein the designated point can include the coordinates of the point selected by the doctor at the remote end in the image file coordinate system, and the processing module processes the designated point and the target transformation matrix, and can convert the designated point into a target point on the target part, wherein the target point can be a point corresponding to the designated point in the base coordinate system of the medical device 20. By converting the designated point for the scanned data transmitted by the remote terminal into the target point of the target part, it is convenient for the medical device 20 to drive the adjustment arm 210 to move toward the target point under the control of the remote terminal, and the interactivity between the doctor at the remote end and the doctor at the local end can be improved.
[0057] Step 340 , controlling the movement of the adjustment arm according to the target point, and transmitting the target image data collected by the surgical field acquisition module 220 for the target point to the remote terminal.
[0058] Specifically, the processing module can control the movement of the adjustment arm according to the target point. For example, the target joint angles of each joint of the adjustment arm can be generated according to the target point, and the movement of the adjustment arm can be controlled based on the target joint angles. When the end of the adjustment arm moves to the vicinity of the target point, the surgical field acquisition module 220 can be controlled to acquire images of the area near the target point, and the acquired target image data can be transmitted to the remote terminal for viewing by the doctor at the remote end. In the above manner, the real-time image of the target point corresponding to the specified point can be transmitted to the remote terminal, so that the doctor at the remote end can provide more detailed guidance to the doctor at the local end, thereby improving interactivity.
[0059] In some examples, the surgical field acquisition module 220 may include a surgical field camera. For example, the surgical field camera may be a structured light camera to acquire target image data.
[0060] The method of the embodiment of the present application is applied to a processing module of a medical device; the medical device also includes an adjustment arm, and a surgical field acquisition module arranged on the adjustment arm; the processing module connects the surgical field acquisition module and the adjustment arm, and is also used to connect to a remote terminal; the method includes: acquiring scanning data collected for a target part and transmitting it to a remote terminal for review by a doctor at the remote end, and acquiring point cloud data collected for the target part by the surgical field acquisition module; based on the scanning data, the point cloud data and the posture parameters of the adjustment arm, obtaining a target transformation matrix between the image file coordinate system and the base coordinate system of the medical device, and the target transformation matrix can be used for coordinate system conversion between the base coordinate system and the image file coordinate system; acquiring a designated point for the scanning data transmitted by the remote terminal, and according to the designated point and the target transformation matrix, the designated point can be converted into a target point of the target part; controlling the movement of the adjustment arm according to the target point, and transmitting the target image data collected for the target point by the surgical field acquisition module to the remote terminal, which can improve the interactivity between the remote terminal and the medical device, so as to realize comprehensive, sufficient and detailed remote guidance of local surgery.
[0061] In one embodiment, if Figure 4 As shown, based on the scanning data, the point cloud data and the posture parameters of the adjustment arm 210, the target transformation matrix is obtained, including:
[0062] Step 410, obtaining a first transformation matrix between the surgical field acquisition module 220 and the image file coordinate system based on the scan data and the point cloud data;
[0063] Step 420 , obtaining a target transformation matrix according to the posture parameters of the adjustment arm 210 and the first transformation matrix.
[0064] Specifically, the scanning data for the target part may include the coordinates of each feature point of the target part in the image file coordinate system; the point cloud data for the target part may include the coordinates of each feature point in the coordinate system of the surgical field acquisition module 220; and then, according to the coordinates of each feature point in the image file coordinate system and the coordinates in the coordinate system of the surgical field acquisition module 220, the first transformation matrix between the coordinate system of the surgical field acquisition module 220 and the image file coordinate system can be obtained. For example, the first transformation matrix may include the transformation matrix of the coordinate system of the surgical field acquisition module 220 relative to the image file coordinate system. The first transformation matrix can be used to transform the coordinates in the coordinate system of the surgical field acquisition module 220 with the coordinates in the image file coordinate system. The mutual conversion between targets; the posture parameters of the adjustment arm 210 may include the size of each joint of the adjustment arm 210 and the size of the joint angle when acquiring the point cloud data. Based on the posture parameters of the adjustment arm 210, the conversion relationship between the coordinate system of the surgical field acquisition module 220 and the base coordinate system can be obtained, for example, the transformation matrix between the coordinate system of the surgical field acquisition module 220 and the base coordinate system; according to the posture parameters of the adjustment arm 210 and the first transformation matrix, for example, according to the transformation matrix between the coordinate system of the surgical field acquisition module 220 and the base coordinate system, and the transformation matrix between the coordinate system of the surgical field acquisition module 220 and the image file coordinate system, the target transformation matrix between the image file coordinate system and the base coordinate system can be obtained.
[0065] In one embodiment, if Figure 5 As shown, based on the scanning data, the point cloud data and the posture parameters of the adjustment arm 210, the target transformation matrix is obtained, including:
[0066] Step 510, obtaining target data of the target part in the base coordinate system according to the posture parameters of the adjustment arm and the point cloud data;
[0067] Step 520, obtaining a target transformation matrix according to the target data and the scan data.
[0068] Specifically, the posture parameters of the adjustment arm 210 may include the size of each joint of the adjustment arm 210 and the size of the joint angle when acquiring the point cloud data. Based on the posture parameters of the adjustment arm 210, the conversion relationship between the coordinate system of the surgical field acquisition module 220 and the base coordinate system can be obtained, for example, the transformation matrix between the coordinate system of the surgical field acquisition module 220 and the base coordinate system, and then the coordinate system of the surgical field acquisition module 220 can be transformed to the base coordinate system of the medical device 20. According to the posture parameters of the adjustment arm and the point cloud data, the coordinates of each feature point in the point cloud data in the coordinate system of the surgical field acquisition module 220 can be transformed to obtain the target data of the target part in the base coordinate system of the medical device 20, and the target data may include the coordinates of each feature point in the base coordinate system of the medical device 20; according to the target data and the scanning data, for example, the target transformation matrix can be obtained according to the coordinates of each feature point in the base coordinate system of the medical device 20 and the coordinates of each feature point in the image file coordinate system. The target transformation matrix can be used for the mutual conversion between the coordinates in the base coordinate system of the medical device 20 and the coordinates in the image file coordinate system.
[0069] In one embodiment, if Figure 6 As shown, based on the scanning data and the point cloud data, a first transformation matrix between the surgical field acquisition module 220 and the image file coordinate system is obtained, including:
[0070] Step 610, obtaining a first coordinate set of the target part based on the scan data; the first coordinate set includes the coordinates of each feature point of the target part in the image file coordinate system;
[0071] Step 620, obtaining a second coordinate set of the target part according to the point cloud data; the second coordinate set includes the coordinates of each feature point in the coordinate system of the surgical field acquisition module;
[0072] Step 630: Obtain a first transformation matrix based on the first coordinate set and the second coordinate set.
[0073] Specifically, the scanning data can be processed to determine the coordinates of each feature point of the target part in the image file coordinate system, that is, the first coordinate set; the point cloud data can be processed to determine the coordinates of each feature point of the target part in the coordinate system of the surgical field acquisition module 220, that is, the second coordinate set; the first coordinate set and the second coordinate set can be matched, for example, the first coordinate set is matched one-to-one with the feature points in the second coordinate set for each feature point of the target part, and then the first transformation matrix can be determined. The first transformation matrix can be used for mutual conversion between coordinates in the coordinate system of the surgical field acquisition module 220 and coordinates in the image file coordinate system.
[0074] In one example, the target object may be subjected to a CT scan before surgery, for example, a head CT scan, and the coordinates of each feature point of the target object in the coordinate system of the CT image file may be obtained. in, The p in the lower left corner stands for patient (i.e. the target object).
[0075] The medical device 20 can be implemented by using an audio and video trolley. A suitable position can be selected to fix the surgical trolley, the target object, and the audio and video trolley, and the position of the audio and video trolley remains unchanged.
[0076] The surgical field acquisition module 220 can be implemented using a surgical field camera. The surgical field camera can be controlled to obtain the facial point cloud of the target object. The facial point cloud may include the coordinates of each feature point of the target object in the coordinate system of the surgical field camera. It is related to the facial features of the target object and is a known quantity. Then, the coordinates of each feature point of the target object acquired by the surgical field camera in the coordinate system of the surgical field camera are is a known quantity. There is the following formula:
[0077]
[0078]
[0079] Among them, the coordinates of each feature point of the target object in the coordinate system of the CT image file are As a known quantity, each feature point in the coordinate system of the surgical field camera can be matched with each feature point in the coordinate system of the CT image file, and the homogeneous transformation matrix of the coordinate system of the surgical field camera relative to the coordinate system of the CT image file can be obtained. ( Represents the homogeneous transformation matrix of the B coordinate system relative to the A coordinate system, ).
[0080]
[0081] In the formula, Represents the homogeneous transformation matrix of the coordinate system of the surgical field camera relative to the base coordinate system of the audio and video trolley, It is only related to the joint angle of the adjustment arm and the mechanical size of the adjustment arm. When the joint angle of the adjustment arm is determined, As a known quantity, the coordinates of each feature point of the target object in the base coordinate system of the audio and video trolley can be obtained
[0082]
[0083] Based on the above formula, the coordinates of each feature point of the target object in the base coordinate system of the audio and video trolley are And the coordinates of each feature point of the target object in the coordinate system of the CT image file The homogeneous transformation matrix of the coordinate system of the CT image file relative to the base coordinate system of the audio and video trolley can be obtained It can also be achieved by using the homogeneous transformation matrix of the coordinate system of the surgical field camera relative to the coordinate system of the CT image file And the homogeneous transformation matrix of the coordinate system of the surgical field camera relative to the base coordinate system of the audio and video trolley Get the homogeneous transformation matrix of the coordinate system of the CT image file relative to the base coordinate system of the audio and video trolley
[0084] Since the positions of the target object and the audio and video trolley are fixed, the homogeneous transformation matrix The remote doctor can select any point in the CT image file displayed on the remote terminal. Based on the following formula:
[0085]
[0086] You can get a selected point Coordinates in the base coordinate system of the audio and video trolley Based on coordinates The audio and video trolley can control the adjustment arm to move the surgical field camera to the appropriate position and shoot the coordinates. The remote doctor can see the corresponding points of the target object on the CT image he selected through the surgical field camera. The whole process does not require the local doctor to do anything, and the process is highly automated.
[0087] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0088] In one embodiment, Figure 2As shown, a medical device 20 is provided, the medical device 20 includes a processing module, an adjustment arm 210, and an operation field acquisition module 220 provided on the adjustment arm 210; the processing module is connected to the operation field acquisition module 220 and the adjustment arm 210, and is also used to connect to a remote terminal;
[0089] The processing module is used to obtain the scanning data collected for the target part and transmit it to the remote terminal, and obtain the point cloud data collected for the target part by the surgical field acquisition module 220; based on the scanning data, the point cloud data and the posture parameters of the adjustment arm 210, the target transformation matrix is obtained; the target transformation matrix is the transformation matrix between the image file coordinate system corresponding to the scanning data and the base coordinate system of the medical device 20; the designated point for the scanning data transmitted by the remote terminal is obtained, and the target point of the target part is obtained according to the designated point and the target transformation matrix; the movement of the adjustment arm 210 is controlled according to the target point, and the target image data collected for the target point by the surgical field acquisition module 220 is transmitted to the remote terminal.
[0090] Specifically, the working process of the medical device 20 can refer to the above-mentioned remote data transmission method. In some examples, the medical device 20 can be provided with a button or a touch screen connected to the processing module, and the doctor at the local end can control the adjustment arm 210 through the button or the touch screen, or manually drag the adjustment arm 210. The number of the adjustment arms 210 can be one or more, and the number of the surgical field acquisition modules 220 can also be one or more.
[0091] In some examples, a doctor at the remote end can control the movement of the adjustment arm 210 through an input device connected to the remote terminal, for example, according to the direction keys of a keyboard.
[0092] In one embodiment, the processing module is further used to connect to an electrophysiological device to obtain electrophysiological data collected by the electrophysiological device; Figure 2 As shown, the medical device 20 also includes:
[0093] A panoramic acquisition module 230, which is connected to the processing module and is used to acquire panoramic image data;
[0094] The display module 240 is connected to the processing module and is used for displaying based on at least one of the electrophysiological data, the panoramic image data, the scanning data, the point cloud data and the target image data.
[0095] Specifically, the electrophysiological device can be a device that uses electrophysiological instruments, electrodes, voltage clamps, and patch clamps to record or measure the potential changes, conduction velocity, and activity of ion channels of biological or ex vivo tissues, nerves, and cell ion channels. The surgical field acquisition module 220 can include a surgical field camera for collecting the visual field required during the operation; the panoramic acquisition module 230 can include a panoramic camera, and the panoramic acquisition module 230 can be set at the top of the connecting rod, and the middle of the connecting rod can be fixedly connected to the adjustment arm 210 and the display module 240. The display module 240 is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device, and can be used for local doctors in the operating room to interact with remote doctors who guide the operation. The display screen can be a liquid crystal display screen or an electronic ink display screen. The doctor at the local end can confirm the pictures taken by the surgical field acquisition module 220 and / or the panoramic acquisition module 230 based on the display screen of the display module 240. The display module 240 can display at least one of the electrophysiological data, panoramic image data, scanning data, point cloud data and target image data. If there are multiple data to be displayed, split screen display can be performed. Figure 7 As shown in (a), the display content of the display module 240 of the medical device 20 can be mainly used to display the workflow screen containing the CT image generated based on the scan data, and can also include the screen generated by the target image data collected by the surgical field camera, the screen generated by the panoramic image data collected by the panoramic camera, and the screen generated by the electrophysiological data collected by the electrophysiological device. Figure 7 As shown in (b), the display content of the remote terminal's display screen can be mainly used to display a screen generated by target image data acquired by the surgical field camera, and can also include a workflow screen containing CT images generated based on scanning data, a screen generated by panoramic image data acquired by a panoramic camera, and a screen generated by electrophysiological data acquired by an electrophysiological device.
[0096] In some examples, when the local doctor performs relevant operations, he can drag the adjustment arm 210 to move the surgical field acquisition module 220 closer to the target part to achieve more detailed shooting, and display relevant images through the display module 240. The number of display modules 240 can be one or more. In the case of multiple display modules 240, electrophysiological data, panoramic image data, scanning data, point cloud data and target image data can be displayed separately.
[0097] In one embodiment, the medical device 20 further includes:
[0098] A laser indicating module 810, which is fixedly connected to the surgical field acquisition module 220; or
[0099] The laser indication module 810 is connected to the surgical field acquisition module 220 via at least two joints; the joints are connected to the processing module, and the joints move under the control of the processing module or the remote terminal.
[0100] Specifically, the laser indication module 810 can be fixed on the surgical field acquisition module 220, and the processing module controls the movement of each joint of the adjustment arm 210 to drive the surgical field acquisition module 220 and the laser indication module 810 of the adjustment arm 210 to move; the laser indication module 810 can also be connected to the surgical field acquisition module 220 through at least two joints, such as Figure 8 As shown, the laser indication module 810 is connected to the surgical field acquisition module 220 through the electric pan joint and the electric pitch joint in turn, wherein the electric pan joint and the electric pitch joint are both connected to the processing module, and the processing module or the remote terminal can control the electric pan joint and the electric pitch joint, thereby making the laser indication module 810 move relative to the surgical field acquisition module 220, and the electric pan joint and the electric pitch joint can also change their posture under the manual drag of the operator. The processing module can also be connected to the laser indication module 810, and the processing module or the remote terminal can control the opening or extinguishing of the laser indication module 810.
[0101] In some examples, the laser indication module 810 includes a laser indicator light. The remote terminal can control the laser indicator light to emit laser light to indicate the corresponding position, thereby guiding the local doctor to perform the operation according to the corresponding position, thereby achieving a clearer and more intuitive interaction.
[0102] In one embodiment, the medical device 20 further includes:
[0103] The audio acquisition module is connected to the processing module, and the processing module obtains the audio data collected by the audio acquisition module and transmits it to the remote terminal.
[0104] Specifically, the audio acquisition module can collect audio data from the local end, and the processing module can transmit the audio data to the remote terminal, thereby realizing comprehensive remote transmission of audio and video for communicating with the doctor at the remote end. The audio acquisition module can include a microphone.
[0105] In some examples, the medical device 20 may further include an audio output module connected to the processing module, and the audio output module may output the audio data of the remote terminal received by the processing module to guide the local doctor to perform relevant operations. The audio output module may include a speaker.
[0106] In one embodiment, the number of joints of the adjustment arm 210 is at least five.
[0107] Specifically, when the number of joints of the adjustment arm 210 is greater than or equal to six, the doctor at the remote end can specify a preset distance so that the surgical field acquisition module 220 of the adjustment arm 210 can acquire target image data of the target point at the preset distance.
[0108] In some examples, such as Fig. 9 As shown, the local audio and video trolley may include a power module, a circuit breaker, control hardware, an electric joint module, an audio hardware, a host computer, a panoramic camera, a surgical field camera, a laser indicator light, a display, communication hardware (e.g., a switch), Internet hardware (e.g., a 5G router), and an expansion connector. Among them, the host computer may be a computer host or a video terminal; the power module may be connected to the circuit breaker, the control hardware, the electric joint module, the audio hardware, the host computer, the panoramic camera, the surgical field camera, the display, and the Internet hardware for power supply; the communication hardware may be connected to the control hardware, the host computer, the panoramic camera, and the surgical field camera for mutual communication; the control hardware may be connected to the electric joint module and the laser indicator light to control the adjustment arm posture, and the opening and closing of the laser indicator light; the host computer may be connected to the audio hardware, the display, the Internet hardware, and the expansion connector to control the audio hardware, the display, the Internet hardware, and the expansion connector respectively, and the host computer may also control the communication hardware, thereby controlling the panoramic camera and the surgical field camera. The circuit breaker may be connected to an external power supply, and the expansion connector may be connected to an external electrophysiological device to obtain a corresponding video signal. The remote end may include audio hardware, a remote computer host and a display. Audio transmission can be performed between the remote computer host and the audio hardware. The remote computer host can also be connected to Internet hardware to interact with the audio and video trolley. The remote computer host is also connected to the display to transmit the received video signal to the display for corresponding display, so that the remote doctor can view the corresponding electrophysiological information.
[0109] Based on the same inventive concept, the embodiment of the present application also provides a data remote transmission device for implementing the data remote transmission method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more data remote transmission device embodiments provided below can refer to the limitations of the data remote transmission method above, and will not be repeated here.
[0110] In one embodiment, Fig.10As shown, a data remote transmission device is provided, wherein the control device is applied to a processing module of a medical device 20; the medical device 20 further comprises an adjustment arm 210, and an operation field acquisition module 220 disposed on the adjustment arm 210; the processing module is connected to the operation field acquisition module 220 and the adjustment arm 210, and is also used to connect to a remote terminal; the control device comprises:
[0111] The data acquisition module 1010 is used to acquire the scan data collected for the target part and transmit it to the remote terminal, and acquire the point cloud data collected for the target part by the surgical field acquisition module 220;
[0112] The matrix acquisition module 1020 is used to obtain a target transformation matrix based on the scan data, the point cloud data and the posture parameters of the adjustment arm 210; the target transformation matrix is a transformation matrix between the image file coordinate system corresponding to the scan data and the base coordinate system of the medical device 20;
[0113] The target determination module 1030 is used to obtain the designated point for the scan data transmitted by the remote terminal, and obtain the target point of the target part according to the designated point and the target transformation matrix;
[0114] The acquisition execution module 1040 is used to control the movement of the adjustment arm 210 according to the target point, and transmit the target image data acquired by the surgical field acquisition module 220 for the target point to the remote terminal.
[0115] In one embodiment, the matrix acquisition module 1020 is also used to obtain a first transformation matrix between the surgical field acquisition module 220 and the image file coordinate system based on the scanning data and the point cloud data; and obtain a target transformation matrix according to the posture parameters of the adjustment arm 210 and the first transformation matrix.
[0116] In one of the embodiments, the matrix acquisition module 1020 is also used to obtain target data of the target part in the base coordinate system according to the posture parameters and point cloud data of the adjustment arm 210; and obtain the target transformation matrix according to the target data and the scanning data.
[0117] In one embodiment, the matrix acquisition module 1020 is also used to obtain a first coordinate set of the target part based on the scanning data; the first coordinate set includes the coordinates of each feature point of the target part in the image file coordinate system; according to the point cloud data, a second coordinate set of the target part is obtained; the second coordinate set includes the coordinates of each feature point in the coordinate system of the surgical field acquisition module 220; based on the first coordinate set and the second coordinate set, a first transformation matrix is obtained.
[0118] Each module in the above-mentioned data remote transmission device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.
[0119] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.
[0120] In one embodiment, a processing module is provided. The processing module may be a terminal, and its internal structure diagram may be as follows: Fig.11 As shown. The processing module includes a processor, a memory, an input / output interface and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the processing module is used to provide computing and control capabilities. The memory of the processing module includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the processing module is used to exchange information between the processor and an external device. The communication interface of the processing module is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for remote data transmission is implemented.
[0121] Those skilled in the art will understand that Fig.11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0122] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.
[0123] In one embodiment, a computer program product is provided, comprising a computer program, which implements the steps of the above method when executed by a processor.
[0124] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.
[0125] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0126] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A method for remote data transmission, characterized in that: The method is applied to a processing module of a medical device; the medical device further comprises an adjustment arm and an operation field acquisition module arranged on the adjustment arm; the processing module is connected to the operation field acquisition module and the adjustment arm, and is also used to connect to a remote terminal; the method comprises: Acquire scanning data collected for the target part and transmit it to the remote terminal, and acquire point cloud data collected for the target part by the surgical field acquisition module; Based on the scan data, the point cloud data and the posture parameters of the adjustment arm, a target transformation matrix is obtained; the target transformation matrix is a transformation matrix between the image file coordinate system corresponding to the scan data and the base coordinate system of the medical device; Acquire a designated point for the scan data transmitted by the remote terminal, and obtain a target point of the target part according to the designated point and the target transformation matrix; The movement of the adjustment arm is controlled according to the target point, and the target image data collected by the surgical field acquisition module for the target point is transmitted to the remote terminal.
2. The method according to claim 1, characterized in that The step of obtaining a target transformation matrix based on the scan data, the point cloud data and the posture parameters of the adjustment arm comprises: Based on the scan data and the point cloud data, a first transformation matrix between the surgical field acquisition module and the image file coordinate system is obtained; The target transformation matrix is obtained according to the posture parameters of the adjustment arm and the first transformation matrix.
3. The method according to claim 1, characterized in that The step of obtaining a target transformation matrix based on the scan data, the point cloud data and the posture parameters of the adjustment arm comprises: Obtaining target data of the target part in the base coordinate system according to the posture parameters of the adjustment arm and the point cloud data; The target transformation matrix is obtained according to the target data and the scan data.
4. The method according to claim 2, characterized in that: The obtaining, based on the scan data and the point cloud data, a first transformation matrix between the surgical field acquisition module and the image file coordinate system comprises: Based on the scan data, a first coordinate set of the target part is obtained; the first coordinate set includes coordinates of each feature point of the target part in the image file coordinate system; According to the point cloud data, a second coordinate set of the target part is obtained; the second coordinate set includes the coordinates of each of the feature points in the coordinate system of the surgical field acquisition module; The first transformation matrix is obtained based on the first coordinate set and the second coordinate set.
5. A medical device, characterized in that: The medical device comprises a processing module, an adjustment arm, and a surgical field acquisition module provided on the adjustment arm; the processing module is connected to the surgical field acquisition module and the adjustment arm, and is also used to connect to a remote terminal; The processing module is used to obtain the scan data collected for the target part and transmit it to the remote terminal, and obtain the point cloud data collected by the surgical field acquisition module for the target part; based on the scan data, the point cloud data and the posture parameters of the adjustment arm, obtain the target transformation matrix; the target transformation matrix is the transformation matrix between the image file coordinate system corresponding to the scan data and the base coordinate system of the medical device; obtain the designated point for the scan data transmitted by the remote terminal, and obtain the target point of the target part according to the designated point and the target transformation matrix; The movement of the adjustment arm is controlled according to the target point, and the target image data collected by the surgical field acquisition module for the target point is transmitted to the remote terminal.
6. The medical device according to claim 5, characterized in that The processing module is also used to connect to an electrophysiological device to obtain electrophysiological data collected by the electrophysiological device; the medical device also includes: A panoramic acquisition module, connected to the processing module, for acquiring panoramic image data; A display module, the display module is connected to the processing module and is used for displaying based on at least one of the electrophysiological data, the panoramic image data, the scanning data, the point cloud data and the target image data.
7. The medical device according to claim 5, characterized in that The medical device also includes: A laser indicating module, wherein the laser indicating module is fixedly connected to the surgical field acquisition module; or The laser indication module is connected to the surgical field acquisition module via at least two joints; the joints are connected to the processing module, and the joints move under the control of the processing module or the remote terminal.
8. The medical device according to claim 5, characterized in that The medical device also includes: An audio acquisition module, wherein the audio acquisition module is connected to the processing module, and the processing module obtains the audio data collected by the audio acquisition module and transmits the audio data to the remote terminal.
9. A data remote transmission device, characterized in that: The control device is applied to a processing module of a medical device; the medical device further comprises an adjustment arm and an operation field acquisition module provided on the adjustment arm; the processing module is connected to the operation field acquisition module and the adjustment arm, and is also used to connect to a remote terminal; the control device comprises: A data acquisition module, used to acquire scanning data collected from a target part and transmit the data to the remote terminal, and to acquire point cloud data collected from the target part by the surgical field acquisition module; A matrix acquisition module, used to obtain a target transformation matrix based on the scan data, the point cloud data and the posture parameters of the adjustment arm; the target transformation matrix is a transformation matrix between the image file coordinate system corresponding to the scan data and the base coordinate system of the medical device; A target determination module, used to obtain a designated point for the scan data transmitted by the remote terminal, and obtain a target point of the target part according to the designated point and the target transformation matrix; The acquisition execution module is used to control the movement of the adjustment arm according to the target point, and transmit the target image data acquired by the surgical field acquisition module for the target point to the remote terminal.
10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.