Intraoperative ultrasonic registration method and system for bone operation direct positioning

Abstract

The invention discloses an intraoperative ultrasonic registration method and system for direct positioning of a bone operation, belongs to the technical field of surgical navigation, and aims to solve the problem of displacement data deviation caused by relative displacement of skin and bones due to physiological movement. Comprising the following steps: acquiring preoperative three-dimensional image data corresponding to an operative region; acquiring a plurality of pieces of three-dimensional ultrasonic image data corresponding to the surgical area; acquiring position data corresponding to the three-dimensional ultrasonic image data; according to the multiple pieces of three-dimensional ultrasonic image data, the position data corresponding to the three-dimensional ultrasonic image data and the preoperative three-dimensional image data, a homogeneous transformation matrix is obtained; acquiring real-time ultrasonic image data acquired by an ultrasonic probe fixedly attached to the surgical area; acquiring real-time position data of an ultrasonic probe for acquiring the real-time ultrasonic image data; according to the homogeneous transformation matrix and the real-time position data, the real-time ultrasonic image data and the preoperative three-dimensional image data are registered, and in-vivo skeleton structure displacement data are estimated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of surgical navigation, in particular to an intraoperative ultrasound registration method and system for direct positioning of bone surgery. BACKGROUND

[0002] In bone surgery, accurately obtaining real-time displacement data of the bone is a core requirement to ensure the safety and effectiveness of the surgery. Traditional surgical navigation monitoring techniques rely on single modal displacement monitoring methods, such as bone displacement tracking technology based on ultrasound images. However, in actual surgical procedures, physiological activities such as respiratory motion, slight changes in body position, or muscle contraction of the patient can cause relative displacement between the skin and the bone. When a single ultrasound monitoring technology is used, this relative displacement between the skin and the bone will directly cause the ultrasound image data to be unable to accurately reflect the true motion state of the bone, thereby causing the obtained bone displacement data to have deviations. Such deviations can cause positioning errors in the surgical navigation system, increase the risk of nerve damage, deviation of internal fixation position, and other surgical risks, and severely restrict the improvement of the precision level of bone surgery.

[0003] The disclosure of the above background art content is only used to assist in understanding the concept and technical solutions of the present application, and it does not necessarily belong to the prior art of the present patent application. In the absence of explicit evidence that the above content has been disclosed on the filing date of the present patent application, the above background art should not be used to evaluate the novelty and inventiveness of the present application. SUMMARY

[0004] The present application provides an intraoperative ultrasound registration method and system for direct positioning of bone surgery, which is used to solve the problem of displacement data deviation caused by relative displacement between the skin and the bone due to physiological motion.

[0005] To achieve the above-mentioned purpose, the embodiments of the present application disclose the following technical solutions:

[0006] In a first aspect, the embodiments of the present application provide an intraoperative ultrasound registration method for direct positioning of bone surgery, comprising the following steps:

[0007] Obtaining preoperative three-dimensional image data corresponding to the surgical area;

[0008] Obtaining a plurality of three-dimensional ultrasound image data corresponding to the surgical area;

[0009] Obtaining position data corresponding to each three-dimensional ultrasound image data, the position data being the position data of the ultrasound probe used to collect the three-dimensional ultrasound image data in the global coordinate system established by the binocular camera when capturing the corresponding three-dimensional ultrasound image data;

[0010] According to the plurality of three-dimensional ultrasound image data and the position data corresponding to each three-dimensional ultrasound image data and the preoperative three-dimensional image data, a homogeneous transformation matrix is obtained;

[0011] Real-time ultrasound image data collected by an ultrasound probe fixedly attached to the surgical area is acquired;

[0012] Real-time position data corresponding to the ultrasound probe used to collect the real-time ultrasound image data in the global coordinate system established by the binocular camera is acquired;

[0013] According to the homogeneous transformation matrix and the real-time position data, the real-time two-dimensional ultrasound image is embedded in a three-dimensional ultrasound space and registered with the preoperative three-dimensional image data, and then real-time displacement data of the in-vivo bone structure is estimated.

[0014] In some possible implementation manners of the first aspect, the preoperative three-dimensional image data is obtained by CT scanning.

[0015] In some possible implementation manners of the first aspect, the ultrasound probe used to collect the three-dimensional ultrasound image data comprises an array, N is the number of one-dimensional arrays, M is the number of array elements of a single one-dimensional array, and an infrared reflective bead is fixedly arranged thereon; the position data of each three-dimensional ultrasound image data is determined by tracking the infrared reflective bead through binocular vision.

[0016] In some possible implementation manners of the first aspect, the step of obtaining the homogeneous transformation matrix according to the plurality of three-dimensional ultrasound image data and the position data corresponding to each three-dimensional ultrasound image data and the preoperative three-dimensional image data comprises:

[0017] A real-time displacement rotation matrix of the probe is constructed by tracking the infrared reflective bead through binocular vision ;

[0018] A local coordinate system of an imaging section of the i th three-dimensional ultrasound image data is constructed according to the position data corresponding to the i th three-dimensional ultrasound image data, to obtain a relative displacement rotation matrix of the local coordinate system of the imaging section of the i th three-dimensional ultrasound image data relative to the infrared reflective bead structure ; ;

[0019] According to the relative displacement rotation matrix ; and the real-time displacement rotation matrix , a transformation corresponding relationship of a pixel (t) on the real-time ultrasound image data to a voxel coordinate in the binocular global coordinate system is obtained, that is,

[0020] ​

[0021] N ultrasound three-dimensional data are constructed based on the transformation correspondence. The three-dimensional ultrasonic volume is obtained;

[0022] The first volume region is obtained based on the three-dimensional ultrasound volume;

[0023] The second volume region was obtained based on the preoperative three-dimensional image data;

[0024] Based on point cloud registration or contour registration, the first volume data and the second volume data are aligned to obtain the homogeneous transformation matrix. .

[0025] In some possible implementations of the first aspect, the step of obtaining a first volume region based on the three-dimensional ultrasound volume includes segmenting the three-dimensional ultrasound volume using threshold segmentation or the U-net deep learning method to segment the first volume or boundary region of the bone. ;in, Represents three-dimensional ultrasound data. ; , and These represent the number of pixels in the three dimensions of the ultrasound volume; subsequently, based on the region... Construct point cloud or contour data for registration.

[0026] In some possible implementations of the first aspect, the step of obtaining the second volume region based on the preoperative three-dimensional image data includes: processing the preoperative three-dimensional image data using threshold segmentation or the U-net deep learning method. Perform segmentation operations to segment out the second volume or boundary region of the skeleton. ;in, and These represent the number of pixels in the three dimensions respectively; then, based on the region... Construct point cloud or contour data for registration.

[0027] Secondly, embodiments of this application provide an intraoperative ultrasound registration system for direct localization in bone surgery, comprising:

[0028] The preoperative 3D image data acquisition module is used to acquire preoperative 3D image data corresponding to the surgical area;

[0029] The three-dimensional ultrasound image data acquisition module is used to acquire multiple three-dimensional ultrasound image data corresponding to the surgical area;

[0030] The three-dimensional ultrasound image position data acquisition module is used to acquire the position data corresponding to each three-dimensional ultrasound image data. The position data is the position data corresponding to the ultrasound probe used to acquire three-dimensional ultrasound image data by the binocular camera in the global coordinate system constructed by the binocular camera when capturing the corresponding three-dimensional ultrasound image data.

[0031] The homogeneous transformation matrix calculation module is used to obtain the homogeneous transformation matrix based on multiple three-dimensional ultrasound image data and the position data corresponding to each three-dimensional ultrasound image data and the preoperative three-dimensional image data.

[0032] The real-time ultrasound image data acquisition module is used to acquire real-time ultrasound image data collected by an ultrasound probe that is fixedly attached to the surgical area.

[0033] The real-time position data acquisition module is used to acquire real-time position data of the ultrasound probe in the global coordinate system constructed by the binocular camera, which is used to collect real-time ultrasound image data.

[0034] The image registration module is used to embed the real-time two-dimensional ultrasound image into a three-dimensional ultrasound space according to the homogeneous transformation matrix and real-time position data, and register it with the preoperative three-dimensional image data to estimate the real-time displacement data of the skeletal structure in the body.

[0035] In some possible implementations of the second aspect, preoperative three-dimensional image data is obtained via CT scans.

[0036] In some possible embodiments of the second aspect, the ultrasound probe for acquiring three-dimensional ultrasound image data includes The array, where N is the number of one-dimensional arrays and M is the number of array elements in a single one-dimensional array, is fixed with infrared reflective spheres; the position data of each three-dimensional ultrasound image is determined by tracking the infrared reflective spheres using binocular vision.

[0037] In some possible implementations of the second aspect, the homogeneous transformation matrix calculation module is specifically used for:

[0038] The real-time displacement and rotation matrix of the probe is constructed by tracking infrared reflective spheres using binocular vision. ;

[0039] Based on the location data corresponding to the i-th 3D ultrasound image data, construct the imaging section of the i-th 3D ultrasound image data. Using the local coordinate system, we obtain the imaging cross-section of the i-th three-dimensional ultrasound image data. The local coordinate system compared to the relative displacement and rotation matrix of the infrared reflective sphere structure ;

[0040] Based on the relative displacement rotation matrix ; and real-time displacement and rotation matrix obtaining real-time ultrasound image data (t) upper pixels to voxel coordinates in the binocular global coordinate system correspondence relationship of transformation,:

[0041]

[0042] constructing N ultrasound three-dimensional data according to the correspondence relationship of transformation , obtaining a three-dimensional ultrasound volume;

[0043] obtaining a first volume region according to the three-dimensional ultrasound volume;

[0044] obtaining a second volume region according to the preoperative three-dimensional image data;

[0045] aligning the first volume data and the second volume data based on point cloud registration or contour registration to obtain a homogeneous transformation matrix .

[0046] In some possible implementation modes of the second aspect, the homogeneous transformation matrix calculation module is specifically configured to be further configured to perform a segmentation operation on the three-dimensional ultrasound volume by a threshold segmentation method or a U-net deep learning method to segment out a first volume or a boundary region of the bone ; wherein, represents the ultrasound three-dimensional data, ; , and respectively represent the number of pixels in three dimensions in the ultrasound volume; and then point cloud or contour data for registration is constructed according to the region .

[0047] In some possible implementation modes of the second aspect, the homogeneous transformation matrix calculation module is specifically configured to be further configured to perform a segmentation operation on the preoperative three-dimensional image data by a threshold segmentation method or a U-net deep learning method to segment out a second volume or a boundary region of the bone ; wherein, and respectively represent the number of pixels in three dimensions; and then point cloud or contour data for registration is constructed according to the region .

[0048] In a third aspect, the embodiments of the present application provide an electronic device, including one or more processors; a storage device having one or more programs stored thereon; and when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of the technical solutions of the first aspect.

[0049] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the method according to any one of the first aspect.

[0050] In a fifth aspect, an embodiment of the present application provides a computer program product, and the computer program product comprises a computer program. The computer program is executed by a processor to implement the method according to any one of the first aspect.

[0051] Compared with the prior art, the one or more technical solutions provided in the first aspect of the embodiments of the present application have at least the following technical effects or advantages:

[0052] The pre-acquired preoperative three-dimensional image data reflects the detailed structure of the bone, and establishes an accurate reference for the comparison of real-time bone displacement data. By acquiring a plurality of three-dimensional ultrasound image data and using a binocular camera to build a global coordinate system to obtain the spatial position information corresponding to the ultrasound probe during acquisition, accurate spatial positioning between the ultrasound image data and the real structure of the bone is achieved. The use of the pre-computed homogeneous transformation matrix enables the real-time acquired ultrasound image data to be matched in the same coordinate space as the preoperative three-dimensional image data, thereby eliminating the deviation caused by the relative movement between the skin and the bone due to the patient's respiratory movement, slight change in body position or muscle contraction. The establishment of a mathematical mapping relationship converts the local coordinates corresponding to the ultrasound image to the preoperative three-dimensional image coordinates, and realizes the accurate reconstruction of the bone displacement information in the real-time monitoring process. This process effectively avoids the errors introduced by soft tissue movement in single ultrasound monitoring technology, and ensures the accuracy of target positioning and the reliability of data in the surgical navigation process.

[0053] In addition, the preoperative three-dimensional image data obtained by CT scanning can accurately present the bone morphology in terms of anatomical structure information, providing a solid basis for establishing an accurate spatial mapping relationship between the ultrasound image and the pre-set anatomical basis. At the same time, the use of infrared reflective beads and binocular vision technology to obtain the accurate position of the ultrasound probe in the global coordinate system ensures that the spatial conversion accuracy is improved in the multi-modal data registration; the threshold segmentation or U-net deep learning method is used to effectively segment the bone region of the ultrasound three-dimensional data and the preoperative three-dimensional image data, respectively, to ensure that the image segmentation result has a clear bone contour and structure boundary, and to provide accurate image data for the point cloud or contour registration process, thereby improving the data consistency and registration accuracy in the bone surgery navigation positioning process.

[0054] The technical effects brought by any one of the second aspect to the fifth aspect can be referred to the technical effects brought by different design manners in the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0055] In order to illustrate the embodiments of the present application or the technical solutions in the prior art more clearly, a brief introduction to the drawings needed in the description of the embodiments or the prior art will be given. Obviously, the drawings in the following description are only exemplary, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of the provided drawings.

[0056] Figure 1 A flowchart of an intraoperative ultrasound registration method for direct positioning of bone surgery provided for some embodiments of the present application Figure 1 ;

[0057] Figure 2 A flowchart of an intraoperative ultrasound registration method for direct positioning of bone surgery provided for some embodiments of the present application Figure 2 ;

[0058] Figure 3 A data processing flowchart of a second volume region provided for some embodiments of the present application;

[0059] Figure 4 A structural diagram of an intraoperative ultrasound registration method for direct positioning of bone surgery provided for some embodiments of the present application;

[0060] Figure 5 A structural diagram of an electronic device suitable for implementing some embodiments of the present application. DETAILED DESCRIPTION

[0061] Reference will now be made in detail to the present application. While the application will be described in conjunction with these specific embodiments, it will be understood that they are not intended to limit the application to these embodiments. On the contrary, the application is intended to cover alternatives, modifications, and equivalents, which can be included within the spirit and scope of the application as defined by the appended claims. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. The present application can be practiced without some or all of these specific details.

[0062] The singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0063] In bone surgery, accurate acquisition of real-time displacement data of bones is the core requirement to ensure the safety and effectiveness of surgery. Traditional surgical navigation monitoring techniques rely on single modal displacement monitoring methods, such as bone displacement tracking technology based on ultrasound images. However, in actual surgical procedures, physiological activities such as respiratory movement, slight change in body position, or muscle contraction of the patient can cause relative displacement between the skin and the bone. When a single ultrasound monitoring technology is used, this relative displacement between the skin and the bone will directly cause the ultrasound image data to be unable to accurately reflect the real movement state of the bone, thereby causing the acquired bone displacement data to have deviations. Such deviations can cause positioning errors of the surgical navigation system, increase the risk of nerve damage, deviation of internal fixation position, and other surgical risks, and seriously restrict the improvement of the precision level of bone surgery.

[0064] To solve the above technical problems, the technical scheme provided by the present application has the following general idea: an intraoperative ultrasound registration method for direct positioning of bone surgery is provided, comprising the following steps: acquiring preoperative three-dimensional image data corresponding to the surgical area; acquiring a plurality of three-dimensional ultrasound image data corresponding to the surgical area; acquiring position data corresponding to each three-dimensional ultrasound image data, the position data being the position data of the ultrasound probe used to collect the three-dimensional ultrasound image data in the global coordinate system established by the binocular camera when capturing the corresponding three-dimensional ultrasound image data; obtaining a homogeneous transformation matrix according to the plurality of three-dimensional ultrasound image data and the position data corresponding to each three-dimensional ultrasound image data and the preoperative three-dimensional image data; acquiring real-time ultrasound image data collected by an ultrasound probe fixedly attached to the surgical area; acquiring real-time position data of the ultrasound probe used to collect the real-time ultrasound image data in the global coordinate system established by the binocular camera; and registering the real-time ultrasound image data with the preoperative three-dimensional image data according to the homogeneous transformation matrix and the real-time position data to obtain in-vivo bone structure displacement data.

[0065] After introducing the basic principles of the present application, the various non-limiting embodiments of the present application will be specifically introduced in conjunction with the drawings of the specification. Please refer to Figures 1-3 The embodiment of the present application provides an intraoperative ultrasound registration method for direct positioning of bone surgery, comprising the following steps:

[0066] S101: acquiring preoperative three-dimensional image data corresponding to the surgical area;

[0067] Specifically, in some embodiments, the preoperative three-dimensional image data is obtained by CT scanning. Of course, the present application is not limited thereto. In other embodiments, the preoperative three-dimensional image data can also be obtained by nuclear magnetic resonance.

[0068] S102: acquiring a plurality of three-dimensional ultrasound image data corresponding to the surgical area;

[0069] Specifically, in some embodiments, the ultrasound probe for collecting the three-dimensional ultrasound image data comprises an array, N is the number of one-dimensional arrays, and M is the number of array elements of a single one-dimensional array. The ultrasound probe is manually scanned near the surgical area against the skin to obtain a plurality of three-dimensional ultrasound image data.

[0070] S103: Obtain position data corresponding to each three-dimensional ultrasound image data, the position data being position data corresponding to the ultrasound probe for collecting the three-dimensional ultrasound image data in the global coordinate system established by the binocular camera when capturing the corresponding three-dimensional ultrasound image data at the time of capture;

[0071] Specifically, in some embodiments, an infrared reflective bead is fixedly arranged on the ultrasound probe for collecting the three-dimensional ultrasound image data; the position data of each three-dimensional ultrasound image data is determined by tracking the infrared reflective bead through binocular vision.

[0072] S104: Obtain a homogeneous transformation matrix according to the plurality of three-dimensional ultrasound image data, the position data corresponding to each three-dimensional ultrasound image data, and the preoperative three-dimensional image data;

[0073] Specifically, in some embodiments, the homogeneous transformation matrix can be obtained according to the plurality of three-dimensional ultrasound image data, the position data corresponding to each three-dimensional ultrasound image data, and the preoperative three-dimensional image data through the following steps:

[0074] First, a real-time displacement and rotation matrix of the probe is constructed by tracking the infrared reflective bead through binocular vision ;

[0075] Second, a local coordinate system of an imaging section of the i-th three-dimensional ultrasound image data is constructed according to the position data corresponding to the i-th three-dimensional ultrasound image data, to obtain a relative displacement and rotation matrix of the local coordinate system of the imaging section of the i-th three-dimensional ultrasound image data relative to the infrared reflective bead structure ; ;

[0076] Third, a transformation corresponding relationship of a pixel on the real-time ultrasound image data to a voxel coordinate in the binocular global coordinate system is obtained according to the relative displacement and rotation matrix and the real-time displacement and rotation matrix :

[0077]

[0078] Fourth, N three-dimensional ultrasound data ​The three-dimensional ultrasonic volume is obtained;

[0079] Fifth step: Based on the three-dimensional ultrasound volume, obtain the first volume region;

[0080] Specifically, the step of obtaining the first volume region based on the three-dimensional ultrasound volume includes segmenting the three-dimensional ultrasound volume using threshold segmentation or the U-net deep learning method to segment out the first volume or boundary region of the bone. ;in, Represents three-dimensional ultrasound data. ; , and These represent the number of pixels in the three dimensions of the ultrasound volume; subsequently, based on the region... Construct point cloud or contour data for registration.

[0081] Step 6: Based on the preoperative 3D image data, obtain the second volume region;

[0082] Specifically, the step of obtaining the second volume region based on preoperative three-dimensional image data includes processing the preoperative three-dimensional image data using threshold segmentation or the U-net deep learning method. Perform segmentation operations to segment out the second volume or boundary region of the skeleton. ;in, and These represent the number of pixels in the three dimensions respectively; then, based on the region... Construct point cloud or contour data for registration.

[0083] Step 7: Based on point cloud registration or contour registration, align the first volume data and the second volume data to obtain the homogeneous transformation matrix. .

[0084] S105: Acquire real-time ultrasound image data from an ultrasound probe fixedly attached to the surgical area;

[0085] Specifically, the ultrasound probe used to acquire real-time ultrasound image data is the same ultrasound probe used to acquire three-dimensional ultrasound image data;

[0086] S106: Acquire the real-time position data of the ultrasound probe used for acquiring real-time ultrasound image data in the global coordinate system constructed by the binocular camera;

[0087] S107: Based on the homogeneous transformation matrix and real-time position data, the real-time two-dimensional ultrasound image is embedded into the three-dimensional ultrasound space and registered with the preoperative three-dimensional image data to estimate the real-time displacement data of the skeletal structure in the body.

[0088] Please seeFigure 4 Based on the same inventive concept as the intraoperative ultrasound registration method for direct positioning of bone surgery in the foregoing embodiments, the embodiments of the present application provide an intraoperative ultrasound registration system for direct positioning of bone surgery, comprising:

[0089] a preoperative three-dimensional image data acquisition module 201 configured to acquire preoperative three-dimensional image data corresponding to a surgical area;

[0090] a three-dimensional ultrasound image data acquisition module 202 configured to acquire a plurality of three-dimensional ultrasound image data corresponding to the surgical area;

[0091] a three-dimensional ultrasound image position data acquisition module 203 configured to acquire position data corresponding to each three-dimensional ultrasound image data, the position data being position data of an ultrasound probe used to acquire the three-dimensional ultrasound image data in a global coordinate system established by a binocular camera when capturing the corresponding three-dimensional ultrasound image data;

[0092] a homogeneous transformation matrix calculation module 204 configured to obtain a homogeneous transformation matrix according to the plurality of three-dimensional ultrasound image data, the position data corresponding to each three-dimensional ultrasound image data, and the preoperative three-dimensional image data;

[0093] a real-time ultrasound image data acquisition module 205 configured to acquire real-time ultrasound image data acquired by an ultrasound probe fixedly attached to the surgical area;

[0094] a real-time position data acquisition module 206 configured to acquire real-time position data of the ultrasound probe used to acquire the real-time ultrasound image data in the global coordinate system established by the binocular camera;

[0095] an image registration module 207 configured to embed the real-time two-dimensional ultrasound image in a three-dimensional ultrasound space according to the homogeneous transformation matrix and the real-time position data, and to register the real-time two-dimensional ultrasound image with the preoperative three-dimensional image data, thereby estimating real-time displacement data of the bone structure in the body.

[0096] In some embodiments, the preoperative three-dimensional image data is obtained by CT scanning.

[0097] In some embodiments, the ultrasound probe used to acquire the three-dimensional ultrasound image data comprises an array, N is the number of one-dimensional arrays, M is the number of array elements of a single one-dimensional array, and an infrared reflective bead is fixedly arranged thereon; the position data of each three-dimensional ultrasound image data is determined by tracking the infrared reflective bead through binocular vision.

[0098] In some embodiments, the homogeneous transformation matrix calculation module 206 is specifically configured to:

[0099] The real-time displacement rotation matrix of the probe is constructed by tracking the infrared reflective bead through binocular vision ;

[0100] constructing an imaging cut of the i-th three-dimensional ultrasound image data according to the local coordinate system of the i-th three-dimensional ultrasound image data ; a relative displacement rotation matrix of the local coordinate system of the imaging cut of the i-th three-dimensional ultrasound image data compared to the infrared retro-reflective sphere structure ;

[0101] obtaining a real-time ultrasound image data ; and a real-time displacement rotation matrix ; a transformation correspondence of the pixel on the real-time ultrasound image data to the voxel coordinate ;

[0102]

[0103] constructing N ultrasound three-dimensional data according to the transformation correspondence, to obtain a three-dimensional ultrasound volume;

[0104] obtaining a first volume region according to the three-dimensional ultrasound volume;

[0105] obtaining a second volume region according to the preoperative three-dimensional image data;

[0106] aligning the first volume data and the second volume data based on point cloud registration or contour registration to obtain a homogeneous transformation matrix .

[0107] In some embodiments, the homogeneous transformation matrix calculation module 206 is specifically configured to be further configured to perform a segmentation operation on the three-dimensional ultrasound volume by a threshold segmentation or a U-net deep learning method, to segment out a first volume or a boundary region of the bone ; wherein, represents the ultrasound three-dimensional data, ; , and respectively represent the number of pixels in three dimensions in the ultrasound volume; and then constructing point cloud or contour data for registration according to the region .

[0108] In some embodiments, the homogeneous transformation matrix calculation module 206 is specifically configured to be further configured to perform a segmentation operation on the preoperative three-dimensional image data by a threshold segmentation or a U-net deep learning method, to segment out a second volume or a boundary region of the bone ; wherein, and These represent the number of pixels in the three dimensions respectively; then, based on the region... Construct point cloud or contour data for registration.

[0109] Understandably, the modules recorded in this intraoperative ultrasound registration system for direct localization in bone surgery are similar to those in the reference system. Figure 1 The steps described correspond to those in the intraoperative ultrasound registration method for direct localization in bone surgery. Therefore, the operations, features, and beneficial effects described above also apply to the intraoperative ultrasound registration system for direct localization in bone surgery and its constituent modules, and will not be repeated here.

[0110] Please see Figure 5 Based on the inventive concept of an intraoperative ultrasound registration method for direct positioning in bone surgery as described in the foregoing embodiments, this application provides an electronic device. This electronic device may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), etc., and fixed terminals such as digital TVs, desktop computers, etc. The electronic device includes a processing unit 301 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM 302 (read-only memory) or a program loaded from storage device 308 into RAM 303 (random access memory). RAM 303 also stores various programs and data required for the operation of the electronic device. The processing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output interface (i.e., I / O interface 305) is also connected to the bus 304.

[0111] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touch screens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data.

[0112] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program according to some embodiments of the present application. For example, some embodiments of the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising program code for performing the methods illustrated by the flowcharts. In some such embodiments, the computer program can be downloaded and installed from a network via the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the methods of some embodiments of the present application are performed.

[0113] It should be noted that the computer readable medium in some embodiments of the present application can be a computer readable signal medium or a computer readable storage medium or any combination thereof. The computer readable storage medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In some embodiments of the present application, the computer readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus or device. In some embodiments of the present application, the computer readable signal medium can include a data signal carried in a baseband or as part of a carrier wave, in which the computer readable program code is carried. Such a propagated data signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. Program code contained on a computer readable medium can be transmitted by any suitable medium, including but not limited to wire, cable, radio frequency (RF), and any suitable combination thereof.

[0114] In some embodiments, the client, server, or other computing devices can communicate information using any known or future developed end-to-end communications protocol, such as the HyperText Transfer Protocol (HTTP), and can be interconnected via any form or medium of digital data communication (for example, a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet, and peer-to-peer networks (for example, ad hoc peer-to-peer networks), as well as any current or future developed network.

[0115] The computer readable medium described above can be included within the electronic device described above; or can exist separately from the electronic device, and not be assembled into the electronic device. The computer readable medium described above carries one or more programs, when the one or more programs are executed by the electronic device, cause the electronic device to implement the method steps of any of the technical solutions described above.

[0116] Computer program code for carrying out operations of some embodiments of the application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network ("LAN") or a wide area network ("WAN"), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0117] The computer program product of the present application can be a computer program embodied on a computer readable medium. The computer program product can be stored on a computer readable medium. The computer readable medium can be a magnetic or optical disk, a magnetic tape, or any other suitable medium from which a computer processor can read instructions. The computer program product can also be transmitted to an on-site computer processor via a computer network. The computer program product can also be embodied in at least one of an integrated circuit, a digital signal processor, a computer, a personal digital assistant, and a cellular telephone.

[0118] The modules described in some embodiments of the present application can be implemented by software, or by hardware. The described modules can also be located in a processor. The name of the modules does not constitute a limitation of the modules themselves in some cases.

[0119] The functions described above in the specification of the present application can be performed at least in part by one or more hardware logic components. For example, and without limitation, non-limiting examples of such components include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-chips (SOCs), complex programmable logic devices (CPLDs), etc.

[0120] Some embodiments of the present application also provide a computer program product comprising a computer program which, when executed by a processor, implements any of the above-described intraoperative ultrasound registration methods for direct positioning in bone surgery.

[0121] Although the present application has been described in detail with reference to the foregoing embodiments, the above description is merely illustrative of the application. Various modifications and changes can be made to the application by those skilled in the art which fall within the scope of the present application as defined by the appended claims. Accordingly, the detailed description is to be considered in an illustrative sense and not a restrictive sense.

Claims

1. An intraoperative ultrasound registration method for direct localization of bone surgery, characterized in that, The method comprises the following steps: obtaining preoperative three-dimensional image data corresponding to a surgical area; obtaining a plurality of three-dimensional ultrasound image data corresponding to the surgical area; obtaining position data corresponding to each three-dimensional ultrasound image data, the position data being position data corresponding to the three-dimensional ultrasound image data captured by the ultrasound probe for collecting the three-dimensional ultrasound image data in the global coordinate system established by the binocular camera at the time of capturing the corresponding three-dimensional ultrasound image data; obtaining a homogeneous transformation matrix according to the plurality of three-dimensional ultrasound image data, the position data corresponding to each three-dimensional ultrasound image data, and the preoperative three-dimensional image data; obtaining real-time ultrasound image data collected by an ultrasound probe fixedly attached to the surgical area; obtaining real-time position data corresponding to the ultrasound probe for collecting the real-time ultrasound image data in the global coordinate system established by the binocular camera; embedding the real-time two-dimensional ultrasound image into a three-dimensional ultrasound space according to the homogeneous transformation matrix and the real-time position data, and registering the real-time two-dimensional ultrasound image with the preoperative three-dimensional image data, thereby estimating real-time displacement data of the skeletal structure in the body.

2. The intraoperative ultrasound registration method for direct localization of bone surgery of claim 1, wherein, The preoperative three-dimensional image data is obtained by CT scanning.

3. The intraoperative ultrasound registration method for direct localization of bone surgery of claim 2, wherein, An ultrasound probe for employing the three-dimensional ultrasound image data comprises An array, N is the number of one-dimensional arrays, M is the number of elements of a single one-dimensional array, and infrared reflective beads are fixedly arranged thereon; the position data of each three-dimensional ultrasound image data is determined by tracking the infrared reflective beads through binocular vision.

4. The intraoperative ultrasound registration method for direct localization of bone surgery of claim 3, wherein, The step of obtaining a homogeneous transformation matrix according to the plurality of three-dimensional ultrasound image data, the position data corresponding to each three-dimensional ultrasound image data, and the preoperative three-dimensional image data comprises: Real-time displacement rotation matrix of the probe is constructed by binocular vision tracking infrared reflective beads ; constructing an imaging section of the i-th three-dimensional ultrasound image data according to the position data corresponding to the i-th three-dimensional ultrasound image data , obtaining a local coordinate system of the imaging section of the i-th three-dimensional ultrasound image data , obtaining a relative displacement rotation matrix of the local coordinate system of the imaging section of the i-th three-dimensional ultrasound image data compared to the infrared reflective bead structure ; According to the relative displacement rotation matrix ; and the real-time displacement rotation matrix , real-time ultrasound image data (t) is obtained The transformation relationship of the pixel to the voxel coordinate in the binocular global coordinate system constructing N three-dimensional ultrasound data according to the transformation correspondence relationship , obtaining a three-dimensional ultrasound volume; obtaining a first volume region according to the three-dimensional ultrasound volume; obtaining a second volume region according to the preoperative three-dimensional image data; aligning the first volume data and the second volume data based on point cloud registration or contour registration, resulting in a homogeneous transformation matrix .

5. The intraoperative ultrasound registration method for direct localization of bone surgery of claim 4, wherein, According to the three-dimensional ultrasound volume, the step of obtaining the first volume region comprises a segmentation operation on the three-dimensional ultrasound volume by threshold segmentation or U-net deep learning method to segment out the first volume or boundary region of the bone ; wherein, represents the three-dimensional ultrasound data, ; , and respectively represent the number of pixels in three dimensions in the ultrasound volume; then, according to the region , point cloud or contour data for registration is constructed.

6. The intraoperative ultrasound registration method for direct localization of bone surgery of claim 4, wherein, According to the preoperative three-dimensional image data, the step of obtaining the second volume region comprises: threshold segmentation or U-net deep learning method is used to the preoperative three-dimensional image data Segmentation operation is performed to segment out the second volume or boundary region of the bone ; wherein, and respectively represent the number of pixels in three dimensions; then, according to the region Point cloud or contour data for registration is constructed.

7. An intraoperative ultrasound registration system for direct localization of bone surgery, characterized in that, The method comprises: a preoperative three-dimensional image data acquisition module for obtaining preoperative three-dimensional image data corresponding to a surgical area; a three-dimensional ultrasound image data collection module for obtaining a plurality of three-dimensional ultrasound image data corresponding to the surgical area; a three-dimensional ultrasound image position data acquisition module for obtaining position data corresponding to each three-dimensional ultrasound image data, the position data being position data corresponding to the three-dimensional ultrasound image data captured by the ultrasound probe for collecting the three-dimensional ultrasound image data in the global coordinate system established by the binocular camera at the time of capturing the corresponding three-dimensional ultrasound image data; a homogeneous transformation matrix calculation module for obtaining a homogeneous transformation matrix according to the plurality of three-dimensional ultrasound image data, the position data corresponding to each three-dimensional ultrasound image data, and the preoperative three-dimensional image data; a real-time ultrasound image data collection module for obtaining real-time ultrasound image data collected by an ultrasound probe fixedly attached to the surgical area; a real-time position data acquisition module for obtaining real-time position data corresponding to the ultrasound probe for collecting the real-time ultrasound image data in the global coordinate system established by the binocular camera; an image registration module for registering the real-time ultrasound image data with the preoperative three-dimensional image data according to the homogeneous transformation matrix and the real-time position data, thereby obtaining skeletal structure displacement data in the body.

8. An electronic device, comprising: The method comprises: one or more processors; a storage device having one or more programs stored thereon; when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer program stored on the computer readable storage medium is executed by the processing device to implement the method of any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program, which is executed by the processing device, implements the method of any one of claims 1 to 6.

Images