Registration method, device, equipment, system and storage medium

Abstract

The embodiment of the invention provides a registration method, apparatus, device and system, and a storage medium. The method comprises the steps of obtaining target image information and image data of a target part of a target object; determining spatial position information of the coded marker based on a global marker in the target image information and the coded marker; performing three-dimensional reconstruction based on the image data to obtain point cloud data corresponding to the target part; registration is performed based on the spatial position information corresponding to the coded marker and the point cloud data corresponding to the target part, so that the scanned image and the shot image including part of the coded marker on the fitting part can be registered, and registration can be realized only by acquiring the scanned image through one-time scanning. Compared with the prior art, one-time image scanning can be reduced, so that the operation process of the registration scheme based on the marker can be simplified.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of medical treatment, and particularly relates to a registration method, device, equipment, system and storage medium. BACKGROUND

[0002] Space registration is an important basis for implementing visual medical technology. In the related art, the surgical procedure of a registration scheme based on markers usually includes the following steps. First, a preoperative image is obtained by performing computed tomography (CT) image detection upon admission. A doctor diagnoses and confirms surgery through the preoperative image. Before surgery, a preoperative marker is pasted, and then CT scanning is performed again to ensure registration and registration of the preoperative image and real-time position. After registration and registration, the lesion can be located, and then surgery can be performed. It can be seen that the entire procedure requires two CT scans. However, the operating room and the scanning room are usually far apart, which does not meet the clinical requirement of reducing the number of CT scans. In addition, the marker in the related art is usually metal, which can easily cause skin deformation when pasted on the surface of the patient's skin, thereby reducing the registration accuracy. SUMMARY

[0003] To solve the above problems, the embodiments of the present application provide a registration method, device, equipment, system and storage medium. A user can only obtain image data of a target part by preoperative scanning, and register the image data with image information including coded markers on a fitting component. In this way, compared with the related art, the image scanning in surgery can be reduced, and there is no need to switch between the operating room and the scanning room, thereby simplifying the surgical procedure of the registration scheme based on markers.

[0004] The embodiments of the present application provide a registration method, which includes the following steps.

[0005] Obtaining target image information and image data of a target part of a target object, wherein the target image information includes a global marker and at least part of coded markers on a fitting component, the fitting component is used to fit on the target part, and the global marker is fixed relative to the target part.

[0006] Determining spatial position information of the coded markers based on the global marker and the coded markers in the target image information.

[0007] Performing three-dimensional reconstruction based on the image data to obtain point cloud data corresponding to the target part.

[0008] Performing registration based on the spatial position information corresponding to the coded markers and the point cloud data corresponding to the target part.

[0009] In some embodiments, the performing three-dimensional reconstruction based on the image data to obtain point cloud data corresponding to the target part includes the following steps.

[0010] perform three-dimensional reconstruction based on the image data to determine initial point cloud data corresponding to the target part;

[0011] obtain a thickness of the fitting component and a thickness of the coded marker;

[0012] determine a target thickness based on the thickness of the fitting component and the thickness of the coded marker;

[0013] adjust the initial point cloud data based on the target thickness to obtain point cloud data corresponding to the target part.

[0014] In some embodiments, the coded marker includes coded and marked points, and the method further includes:

[0015] In a case where a user takes a picture through a navigation device, recognizing the coded and marked points in the image based on the collected image;

[0016] determining first marked points in a fitting component model based on the coded and marked points in the image;

[0017] displaying a fitting component model corresponding to the fitting component, and respectively displaying the first marked points and second marked points in a first display mode and a second display mode on the fitting component model to guide the user to take the target image information, wherein the second marked points are marked points remaining in the fitting component model after excluding the first marked points.

[0018] In some embodiments, the determining of the first marked points in the fitting component model based on the coded and marked points in the image includes:

[0019] determining a correspondence between the marked points in the image and the marked points on the fitting component based on the code in the image;

[0020] determining the first marked points in the fitting component model based on the correspondence.

[0021] In some embodiments, the method further includes:

[0022] determining a number of the first marked points;

[0023] determining a collection progress percentage based on the number and a preset minimum collection number;

[0024] outputting the collection progress percentage to enable the user to take the target image information based on the collection progress percentage, wherein a number of the first marked points in the target image information is greater than or equal to the minimum collection number.

[0025] The embodiment of the present application provides a registration device, comprising:

[0026] An acquisition module is configured to acquire target image information and image data of a target part of a target object, wherein the target image information comprises a global marker and at least part of coded markers on a fitting component, the fitting component is used for fitting on the target part, and the global marker is fixed relative to the target part.

[0027] A determination module is configured to determine spatial position information of the coded markers based on the global marker and the coded markers in the target image information.

[0028] A reconstruction module is configured to perform three-dimensional reconstruction based on the image data to obtain point cloud data corresponding to the target part.

[0029] A registration module is configured to perform registration based on the spatial position information corresponding to the coded markers and the point cloud data corresponding to the target part.

[0030] The embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the registration method in any of the above.

[0031] The embodiment of the present application provides a registration system, comprising:

[0032] A fitting component, wherein the fitting component is provided with coded markers, and the fitting component is used for fitting on a target part of a target object.

[0033] A global marker, wherein the global marker is fixed relative to the target part.

[0034] A navigation device, wherein the navigation device is used for acquiring target image information, and the target image information comprises the global marker and at least part of coded markers on the fitting component.

[0035] The electronic device is in communication connection with the navigation device.

[0036] In some embodiments, the coded markers comprise coded and marker points, and the coded is used for distinguishing the marker points.

[0037] The embodiment of the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method in any of the above.

[0038] The embodiment of the present application provides a computer program product, when the computer program product runs on a terminal device, makes an electronic device execute the registration method.

[0039] The registration method provided by the embodiment of the present application comprises the following steps: obtaining target image information and image data of a target part of a target object, wherein the target image information comprises a global marker and at least part of an encoding marker on a fitting component, the fitting component is used for being fitted on the target part, and the global marker is fixed relative to the target part; determining spatial position information of the encoding marker based on the global marker and the encoding marker in the target image information; obtaining point cloud data corresponding to the target part through three-dimensional reconstruction based on the image data; and performing registration based on the spatial position information corresponding to the encoding marker and the point cloud data corresponding to the target part, so that scanning image and the image including part of the encoding marker on the fitting component can be registered, the registration can be realized only by obtaining the scanning image through one-time scanning, and one-time image scanning can be reduced, so that a surgical procedure of a registration scheme based on a marker can be simplified. BRIEF DESCRIPTION OF DRAWINGS

[0040] The present application will be described in more detail below based on the embodiments and with reference to the drawings.

[0041] Figure 1 An implementation flowchart of the registration method provided by the embodiment of the present application is shown in the figure.

[0042] Figure 2 An implementation flowchart of another registration method provided by the embodiment of the present application is shown in the figure.

[0043] Figure 3 An overall structure diagram of the fitting component provided by the embodiment of the present application is shown in the figure.

[0044] Figure 4 A diagram of the target image information provided by the embodiment of the present application is shown in the figure.

[0045] Figure 5 A flowchart of registration calculation provided by the embodiment of the present application is shown in the figure.

[0046] Figure 6 A structure diagram of a registration device provided by the embodiment of the present application is shown in the figure.

[0047] Figure 7 A composition structure diagram of an electronic device provided by the embodiment of the present application is shown in the figure.

[0048] In the drawings, the same components are denoted by the same reference numerals, and the drawings are not drawn according to the actual scale. DETAILED DESCRIPTION

[0049] In order to make the purposes, technical solutions and advantages of the present application clearer, the following will further describe the present application in detail with reference to the drawings, and the described embodiments should not be regarded as limitations to the present application. All other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0050] In the following description, “some embodiments” are referred to, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0051] If similar descriptions of “first\second\third” appear in the application file, the following description is added. In the following description, the terms “first\second\third” referred to are only to distinguish similar objects, and do not represent a specific order of the objects. It can be understood that “first\second\third” can be interchanged in a specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0053] In surgical navigation surgery, especially in neurosurgical navigation surgery, accurate preoperative image and intraoperative real-time position registration is the key to ensure the success of the surgery. The surgical procedure based on the registration scheme of the marker is generally as follows: hospital admission for computed tomography (CT) image detection to obtain preoperative images, and the doctor confirms the surgery through preoperative image diagnosis. Before the surgery, preoperative markers need to be pasted, and then CT scanning is performed again to ensure the registration of the preoperative image and the intraoperative real-time position. After registration, the lesion can be located, and then the surgery can be performed. It can be seen that the entire process needs to be performed twice CT scanning, and usually the operating room and the scanning room are far away, which does not meet the clinical demand of reducing the number of CT scanning. In addition, the traditional registration method usually relies on high-cost devices such as binocular cameras and depth cameras, and needs to paste metal markers and perform additional image scanning, which increases the complexity of the surgery and the waiting time of the patient. In addition, the traditional markers may cause a decrease in registration accuracy due to deformation caused by contact with the surface of the patient's skin.

[0054] Based on the problems in the related art, the embodiment of the present application provides a registration method, which can be applied to electronic devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, notebook computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), scanning devices, and the like. The embodiment of the present application does not make any limitation on the specific type of the electronic device.

[0055] The function realized by the registration method provided in the embodiment of the present application can be realized by calling program code by the processor of the electronic device, where the program code can be saved in a computer storage medium.

[0056] The embodiment of the present application provides a registration method, Figure 1 The implementation flowchart of the registration method provided in the embodiment of the present application is shown in Figure 1 The registration method includes:

[0057] In step S101, target image information and image data of a target part of a target object are acquired, where the target image information includes a global marker and at least part of a coded marker on a fitting component, the fitting component is used to fit on the target part, and the global marker is fixed relative to the target part.

[0058] In the embodiment of the present application, the target image information should include the global marker and at least part of the coded marker on the fitting component of the target part, both of which are used for spatial positioning and registration of the target part and the preoperative image.

[0059] In the embodiment of the present application, the global marker is a reference frame fixed in space relative to the target part. It is usually placed within the surgical field and maintains a constant position to provide a stable coordinate system for the navigation system. The presence of the global marker enables the navigation system to accurately determine the position of the surgical instrument and the patient relative to this fixed reference.

[0060] In the embodiment of the present application, the global marker is fixed relative to the target part, and the global marker usually has distinctive features, such as specific shape, color or texture, which are easy to identify in the image. The number and position of the global marker should be determined according to the specific circumstances of the surgical requirements and the target part to ensure that sufficient spatial reference information can be provided.

[0061] In the embodiments of the present application, the fitting component is a flexible structure used for fitting on a target site (such as the scalp or skull of a patient), and has code markers attached thereon, which can be two-dimensional or three-dimensional, and contain code information that can be resolved by image processing technology. Unlike global markers, the code markers are in direct contact with the target site, so their position information can more directly reflect the spatial position of the target site. In addition, the flexible nature of the fitting component allows it to easily adapt to the shape and curves of the target site, ensuring that the code markers can be accurately fixed in the desired position. Compared with traditional metal markers, the fitting component is less likely to cause skin deformation, thereby avoiding registration errors caused by deformation during the fixing process. Therefore, the design of the code markers enables the target site to be accurately identified and positioned in a complex image environment. The fitting component can include a fitting cap. For example, for the head, the fitting cap can be a marker point head cap.

[0062] For example, in the production and fitting of the marker point head cap, during production, special ink containing infrared reflective particles can be used to print code markers on a transparent film, thereby obtaining a marker point head cap, and then the film is attached to the skin of the head by water transfer printing or sticker, so that the marker point head cap is attached to the scalp of the user's head.

[0063] In the embodiments of the present application, before the operation begins, the fitting component can be accurately attached to the target site, and it is ensured that the global marker is also in the appropriate position. Then, using appropriate imaging equipment (such as a camera, an X-ray machine, an infrared camera, etc.), the target site is photographed or scanned to obtain target image information containing the global marker and the code marker.

[0064] In the embodiments of the present application, the target object can be a patient, and the target site can be any one of the head, chest, abdomen, etc. The image data of the target site is image data obtained by medical imaging technology (such as CT, MRI, etc.) before the operation, and in the registration process, the image data is the key information for constructing three-dimensional point cloud data to provide detailed position information of the internal structure of the target site.

[0065] In step S102, the spatial position information of the code marker is determined based on the global marker and the code marker in the target image information.

[0066] In the embodiments of the present application, the target image information is image data containing the global marker and the code marker, and through image processing technology, the features of the global marker and the code marker can be identified and extracted, and then necessary spatial positioning information can be provided for surgical navigation operations, which will be used for registration with the preoperative image data to achieve accurate navigation and surgical positioning.

[0067] Step S103, three-dimensional reconstruction is performed based on the image data to obtain point cloud data corresponding to the target part.

[0068] In the embodiments of the present application, the image data of the target part obtained by medical imaging technology can be used to generate three-dimensional point cloud data of the target part through a three-dimensional reconstruction algorithm.

[0069] Step S104, registration is performed based on the spatial position information corresponding to the coded marker and the point cloud data corresponding to the target part.

[0070] In the embodiments of the present application, the iterative closest point algorithm can be used to align the spatial position information corresponding to the coded marker and the point cloud data corresponding to the target part, thereby achieving accurate registration.

[0071] According to the registration method provided in the embodiments of the present application, target image information and image data of a target part of a target object are obtained, wherein the target image information includes a global marker and at least part of the coded markers on the fitting component, the fitting component is used to fit on the target part, and the global marker is fixed relative to the target part; the spatial position information of the coded markers is determined based on the global marker and the coded markers in the target image information; three-dimensional reconstruction is performed based on the image data to obtain point cloud data corresponding to the target part; and registration is performed based on the spatial position information corresponding to the coded markers and the point cloud data corresponding to the target part. The scanning image can be registered with the image including part of the coded markers on the fitting component. In this way, the registration can be achieved only by obtaining the scanning image through one scanning, and the image scanning process can be reduced, thereby simplifying the surgical procedure of the registration scheme based on the marker.

[0072] According to the present application, the spatial position is determined by using the global marker and the coded marker, which avoids the need for additional scanning or complex equipment in the traditional method, and simplifies the registration process. The use of the coded marker can provide more spatial position information, which helps to more accurately determine the corresponding relationship between the target part and the preoperative image, and improves the accuracy of registration. In addition, since the registration process is simplified, unnecessary scanning and operation steps are reduced, thereby shortening the waiting time of the patient, and the method is particularly suitable for emergency surgery and other scenarios that require rapid registration.

[0073] When the method provided in the embodiments of the present application is applied in a clinical end workflow, the workflow can be: image detection is taken upon admission; then the doctor diagnoses and confirms the surgery; and registration is registered. Compared with the method in the related art, the image scanning process can be reduced once.

[0074] In some embodiments, step S104, registration is performed based on the spatial position information corresponding to the coded marker and the point cloud data corresponding to the target part, including:

[0075] In step S1041, the spatial position information corresponding to the coding marker and the point cloud data corresponding to the target site are registered by using the iterative closest point algorithm.

[0076] In the embodiments of the present application, after the spatial position information of the coding marker is determined, the point cloud data corresponding to the coding marker can be determined based on the spatial position information corresponding to the coding marker, and then the point cloud data corresponding to the coding marker and the point cloud data corresponding to the target site are registered by using the iterative closest point algorithm. The point cloud data is a set of three-dimensional coordinate points, which can accurately describe the shape and structure of an object or a scene. The iterative closest point (ICP) algorithm can solve the correspondence between two point cloud data, thereby realizing the geometric alignment of them.

[0077] In the embodiments of the present application, when using the iterative closest point algorithm, the iterative parameters such as the maximum number of iterations, the convergence threshold, etc. can be initialized first. Then, for each point in the point cloud data corresponding to the coding marker, the nearest point in the point cloud data corresponding to the target site is searched as the corresponding point. By continuously iterating the transformation matrix between the corresponding points, the distance between the corresponding points of the two point clouds is minimized. In each iteration, the selection of the corresponding points and the calculation of the transformation matrix are updated until the convergence condition is met or the maximum number of iterations is reached.

[0078] In the embodiments of the present application, by using the iterative closest point algorithm for registration, the spatial position information corresponding to the coding marker and the point cloud data corresponding to the target site can be accurately aligned in geometry, thereby providing accurate spatial position information for subsequent surgical navigation.

[0079] In some embodiments, in step S103, the point cloud data corresponding to the target site is obtained by three-dimensional reconstruction based on the image data, which includes:

[0080] In step S1031, the initial point cloud data corresponding to the target site is determined by three-dimensional reconstruction based on the image data.

[0081] In this step, the image data is obtained by medical imaging technology (such as CT, MRI, etc.) before surgery, which can show the two-dimensional image information of the target site from multiple angles and multiple layers. The three-dimensional model of the target site is reconstructed by using the obtained image data, and the three-dimensional reconstruction process may involve image segmentation, feature extraction, voxelization, etc. Finally, an initial point cloud data composed of a large number of three-dimensional coordinate points is generated, which can preliminarily describe the general position information of the target site.

[0082] In step S1032, the thickness of the fitting component and the thickness of the coding marker are obtained.

[0083] In this step, after the fitting component is made, the thickness of the fitting component and the thickness of the coded marker are known, and the thickness of the fitting component and the coded marker can be stored in the electronic device for acquisition when needed.

[0084] Step S1033, determining the target thickness based on the thickness of the fitting component and the thickness of the coded marker.

[0085] In this step, when the coded marker is arranged on the surface of the fitting component, the thickness sum of the fitting component and the coded marker can be determined to obtain the target thickness. After the target thickness is determined, certain adjustment or correction can be made according to the actual situation to avoid the influence of the target thickness on the accuracy in the registration process.

[0086] Step S1034, adjusting the initial point cloud data based on the target thickness to obtain the point cloud data corresponding to the target part.

[0087] In this step, the initial point cloud data is adjusted according to the determined target thickness. The adjustment methods for different target parts are different. Taking the head as an example, the initial point cloud data can be increased by the target thickness outwardly during the adjustment. After the point cloud data is adjusted, some optimization steps such as smoothing processing or removing outliers may be needed to improve the accuracy and consistency of the point cloud data. In addition, the adjusted point cloud data needs to be verified to ensure that it meets the preset requirements and standards.

[0088] The adjusted point cloud data is output as the point cloud data corresponding to the target part. The output format can be a common point cloud file format such as PLY, PCD, etc. to facilitate subsequent processing and application.

[0089] In these embodiments, the initial point cloud data corresponding to the target part can be corrected based on the thickness sum of the fitting component and the coded marker to obtain the point cloud data corresponding to the target part in the image data, further improving the accuracy and reliability of the point cloud data, and providing accurate spatial position information for subsequent surgical navigation and positioning.

[0090] In some embodiments, step S102, determining the spatial position information of the coded marker based on the global marker and the coded marker in the target image information, comprises:

[0091] Step S1021, determining the spatial position information of the coded marker in the coordinate system of the global marker based on the global marker and the coded marker in the target image information by using a triangulation method.

[0092] In this step, the electronic device captures the target image information and identifies the global marker and the coded marker in the target image information. The global marker, due to its fixed position and known spatial position information, becomes the reference for determining the position of the coded marker. Using the principle of triangulation, the electronic device can analyze the relative positional relationship between the global marker and the coded marker, including the angle and distance between them. Combined with the known spatial position of the global marker, the spatial position information of the coded marker in the global coordinate system can be accurately obtained through calculation.

[0093] In these embodiments, by accurately determining the spatial position of the coded marker, more accurate and reliable positioning information can be provided for subsequent surgical operations, treatment planning or robot control, shortening the operation time and improving the quality and efficiency of medical services.

[0094] According to the present application, it is crucial to ensure that the coded markers are correctly captured and used for registration. To this end, in some embodiments, the coded markers include coded and marker points, and the registration method further includes:

[0095] Step S105, in the case where the user takes a picture through the navigation device, identifying the coded and marker points in the image based on the captured image.

[0096] In this step, the user can be a doctor, and the image is real-time image information obtained before the operation. The coded marker contains two parts, coded and marker points. The code provides unique identity information, so each coded marker has specific coded information, which is usually unique and can be extracted and identified through image processing technology. As an example, the code can be a barcode or a two-dimensional code. The marker point part provides explicit position indication. The marker point is usually a pattern or structure with distinctive features that can be easily detected and located by image processing algorithms. As an example, the marker point can be a dot, a cross line or other easily identifiable shape. In actual application, the marker point can be a round spot, or a chessboard corner point can be extracted. The coding method is not limited to using two-dimensional code, and other coding methods such as Gray code can also be selected.

[0097] In this step, the identification based on the captured image includes the steps of image preprocessing, such as denoising, contrast enhancement, etc., to improve the accuracy of subsequent identification. Image processing techniques are used to extract the coded and marker points in the image.

[0098] In the embodiments of the present application, the navigation device can be an infrared camera, and the infrared camera can be used for shooting or scanning.

[0099] Step S106, determining the first marker point in the fitting component model based on the coded and marker points in the image.

[0100] In the embodiments of the present application, the correspondence between the marker points in the image and the marker points on the fitting component can be determined based on the codes in the image; and the first marker point in the fitting component model can be determined based on the correspondence.

[0101] In the embodiments of the present application, the marker points in the image can be corresponded to the marker points on the fitting component by comparing the codes in the image with the preset codes based on computer vision and image processing technology, so as to identify the first marker point. In the identification process, the electronic device will perform a series of processing on the image, such as denoising, enhancement and scale normalization, etc., to improve the accuracy and stability of identification. Finally, through feature extraction and matching algorithm, the electronic device can determine the position of the first marker point collected on the fitting component, and the first marker point corresponds to the marker point on the fitting component model, so as to determine the first marker point in the fitting component model.

[0102] In step S107, the fitting component model corresponding to the fitting component is displayed, and the first marker point and the second marker point are displayed in the first display mode and the second display mode respectively on the fitting component model, wherein the second marker point is the marker point remaining in the fitting component model after excluding the first marker point.

[0103] In this step, the fitting component model is a three-dimensional model loaded or generated based on the fitting component, which should be consistent with the actual fitting component in shape, size and structure, and can be displayed on the display screen.

[0104] In this step, the first marker point is identified in the image taken by the navigation device through image processing and recognition technology such as feature extraction, matching and positioning. Next, the electronic device needs to map the position information of the identified first marker point to the fitting component model. As an example, registration technology can be used to convert the position coordinates in the two-dimensional image to the corresponding points in the three-dimensional model to ensure accurate alignment between the image and the model.

[0105] In the embodiments of the present application, the first display mode and the second display mode can include a combination of one or more of the display color, shape, and display state of the marker points. For example, in terms of display color, after identifying the first marker points that have been collected, the electronic device can display these first marker points on the model of the fitting component in a first color (e.g., green) by modifying the rendering properties (e.g., color properties) of the model. Meanwhile, those second marker points that have not been collected will be marked on the model in a second color (e.g., red). Such color coding is eye-catching enough to allow the user to easily distinguish between collected and uncollected coded markers, so that the user can adjust the angle and position of the shot according to the color indication on the model of the fitting component, improve the shooting efficiency and accuracy, and reduce the registration problem caused by the lack of markers. Finally, the electronic device displays the updated model of the fitting component on its screen.

[0106] For example, in terms of display shape, the first marker points can be displayed in a circular shape, and the second marker points can be displayed in a square shape. For another example, in terms of display state, the first marker points can flash, and the second marker points do not flash. For example, in terms of a combination of display state and display color, the first marker points are displayed in a flashing and red color, and the second marker points are displayed in a non-flashing and blue color.

[0107] The method provided in the embodiments of the present application can use an infrared camera to capture the coded markers. The infrared camera is in communication connection with the display module. In the capturing process, the first marker points and the second marker points are determined and sent to the display module, so that the display module displays the model of the fitting component corresponding to the fitting component, and distinguishes and displays the first marker points and the second marker points on the model of the fitting component in the first display mode and the second display mode, respectively, to guide the user to shoot to obtain the target image information. Thus, the user can be provided with intuitive visual guidance to help them more effectively complete the image acquisition task. In this way, the accuracy of the surgery is improved, the difficulty of the user operation is reduced, and the overall user experience is improved.

[0108] In these embodiments, the surgical environment can have light changes, occlusions, or partial damage to the markers, etc. By combining the information of the codes and the marker points, the system can more flexibly cope with these situations, so as to improve the robustness and stability of the identification.

[0109] According to the present application, in order to ensure that a sufficient number of coded markers can be obtained in a surgical operation to perform accurate point cloud registration and registration, the registration method can further include some additional steps to guide the user to shoot and provide real-time feedback on the collection progress.

[0110] In some embodiments, after step S107, the registration method further includes:

[0111] Step S108, determine the number of the first marker points.

[0112] In this step, since the first marker points are identified, the number of the first marker points can be counted, which is crucial to ensure the accuracy and safety of the surgery, as it provides real-time feedback on the progress of the marker collection.

[0113] Step S109, determine the collection progress percentage based on the number and the preset minimum collection number.

[0114] In this step, the electronic device compares the number of collected coded markers with the preset minimum collection number. Here, the preset minimum collection number is the number of coded markers that the user needs to collect at least to ensure the accuracy of the surgical navigation. It can be determined according to factors such as surgical requirements, the size of the fitted part, and the distribution of coded markers. As an example, the preset minimum collection number can be 5, 10, 20, or 30.

[0115] In this step, by comparing the calculated number of collected markers with the minimum collection number, the electronic device can calculate the collection progress percentage. The collection progress percentage is calculated by dividing the number of collected markers by the minimum collection number and then multiplying by 100%. As an example, the collection progress percentage can be 20%, 50%, 60%, or 100%.

[0116] Step S109, output the collection progress percentage, wherein the number of the first marker points in the target image information is greater than or equal to the minimum collection number.

[0117] In this step, the electronic device outputs the collection progress percentage to the user, for example, through the display screen of the navigation device or voice prompts, etc. to output the percentage to the user. In this way, by displaying the collection progress percentage, the user can also know how many collection numbers are left to reach the minimum collection number to determine whether to continue shooting. In addition, by displaying the collection progress percentage, the user can adjust the shooting strategy to ensure that the final target image information contains a sufficient number of coded markers, thereby improving the accuracy and reliability of registration to meet the precision requirements of surgical operations.

[0118] In some embodiments, the shooting needs to include global markers and marker points. Due to the possibility of line-of-sight obstruction, there may be cases where some locations cannot be shot at the same time. In this case, multiple global markers can be used, and their relative pose relationship can be calibrated to solve the line-of-sight obstruction problem.

[0119] Based on the foregoing various embodiments, the application provides a specific example application, and the application provides a registration method for spatial point cloud registration in neurosurgery.

[0120] According to the scheme of the application, compared with the related art, the point cloud spatial registration and registration between the image data and the calibration target can be realized only by using a monocular camera and a calibration target without relying on high-cost devices such as binocular cameras and depth cameras, and the device dependency is low. In addition, according to the scheme of the application, compared with the traditional marker registration and registration scheme, the CT scan needs to be performed again after the marker is pasted, especially for emergency surgery such as cerebral hemorrhage, which simplifies the workflow, greatly shortens the waiting time of the patient, and improves the operation efficiency, thereby saving valuable treatment time for the patient. In addition, compared with the traditional marker registration, the metal marker contacts the patient's scalp, which avoids the problem of precision reduction caused by the deformation of the patient's skin surface, and improves the registration and registration precision.

[0121] According to the application, the components applied by the registration method include an image device, a navigation device, a coded marker, a global marker, a registration trigger component, and a head fixing support. Each plays an important role in the registration and registration process. The image device is used to obtain image data of the patient, and provides basic information for subsequent registration and registration. The navigation device is used to provide accurate navigation and guidance during the operation. The coded marker and the global marker are used as the key to spatial positioning, and help the system to accurately identify the physiological structure characteristics of the patient. The registration trigger component can be used to trigger the start of the registration and registration process. The head fixing support is used to ensure the stability of the patient's head position during the entire operation process, and prevent the registration and registration from being invalid due to movement.

[0122] Figure 2 Another registration method provided by the application is shown in the implementation flowchart of the registration method, specifically as shown in Figure 2 The implementation flowchart of the registration method includes:

[0123] Step S201, the patient wears the marker point headgear.

[0124] The marker point headgear is provided with marker points, and each marker point can provide clear and reliable image data, so that the navigation device can image. The navigation device is not limited to a general ordinary camera, an infrared camera, a binocular camera, etc.

[0125] Between different markers, the coding auxiliary distinguishes by coding forms, including but not limited to coding point size, shape, coding ring, and specific coding identifier. The marker can be closely attached to the patient's head and can be pasted on the patient's skin surface, and the whole registration process will not fall off and displace.

[0126] Figure 3 An overall structure diagram of a fitting component is provided for the embodiments of the present application. Referring to Figure 3 , a fitting component includes a headgear, m markers, and n codes. The bottom of the headgear is attached with adhesive, and the material of the adhesive can be referred to similar. The feature points are made of directional reflection material, which can obtain high-contrast ideal images. The form of the coding identifier is not limited, which is made of directional reflection material, and the figure is only an example, and the form can meet the second item.

[0127] Step S202, fix the patient's head and the global reference.

[0128] In clinical practice, doctors generally fix the patient's head by a three-pin headrest. After fixing the patient's head, a global marker with a fixed pose relative to the patient's head is also needed, which can be shot by the navigation device. In the following registration process, the positions of the marker headgear, the patient's head lesion, and the global marker are also fixed.

[0129] Step S203, use the navigation device to take multiple target image information of the markers on the marker headgear and the global marker, until the navigation needs are met.

[0130] In the embodiments of the present application, Figure 4 A schematic diagram of target image information is provided for the embodiments of the present application, as shown in Figure 4 The following two points need to be met:

[0131] The target image information needs to include the complete global marker;

[0132] The target image information needs to include part of the markers on the marker headgear.

[0133] When the user takes pictures using the navigation device, the camera pose needs to be changed so that the sample meets the above two requirements. After the user takes pictures, the navigation device will guide the user to complete the shooting of the remaining markers according to the recognized marker coding information, until all sufficient marker information is obtained.

[0134] Step S204, perform point cloud registration calculation.

[0135] In the embodiments of the present application, Figure 5 A flowchart of registration calculation is provided for the embodiments of the present application, as shown in Figure 5As shown, the image of the mark point and the image of the global marker are acquired by an infrared monocular camera, the point cloud data corresponding to the mark point on the headgear is determined by using the epipolar geometry algorithm, the head skin surface point cloud is determined by using the preoperative medical image, the point cloud data corresponding to the mark point on the headgear is matched with the head skin surface point cloud by using the ICP algorithm for fine registration, so as to realize the registration and registration.

[0136] In the embodiment of the application, when the image data is three-dimensionally reconstructed, the thickness of the headgear (equivalent to the thickness of the fitting component mentioned in the above embodiment) and the thickness of the retroreflective material (equivalent to the thickness of the coded marker mentioned in the above embodiment) need to be compensated by outward expansion. Considering the influence of the thickness of the headgear and the retroreflective material on the actual mark point position, the influence is eliminated by outward expansion compensation, so that more accurate point cloud data is obtained. This compensation mechanism helps to improve the accuracy of registration and registration and ensures the accuracy of surgical navigation.

[0137] According to the application, by following the shooting requirements and using the intelligent guidance function of the navigation device, the user can ensure that sufficient accurate and complete photo information is obtained, so as to realize accurate surgical navigation and positioning.

[0138] Based on the above principle, the present scheme realizes a non-contact registration and registration method without the need for secondary image scanning.

[0139] In the embodiment of the application, the form of the mark point is not limited, and the foregoing is an example of taking a plane circle as the mark point form. In other embodiments, the mark point can be realized by a reflective small ball. Changing the form of the mark point from a plane circle to a reflective small ball helps to improve the visibility and recognition accuracy of the mark point in the image. The reflective small ball can maintain good reflective effect under different angles and light conditions, so that the mark point is clearer and easier to identify during shooting. Therefore, the accuracy and reliability of registration and registration can be further improved. The number of global markers in the embodiment is not limited, and the foregoing is an example of using only one global marker. In other embodiments, if the global marker and a sufficient number of mark points cannot be shot at the same time due to occlusion, the problem can be solved by increasing the number of global markers. By increasing the number of global markers, the problem that the global marker and a sufficient number of mark points cannot be shot at the same time due to occlusion can be solved. Therefore, the scheme is more flexible and has strong adaptability, and can cope with different surgical scenes and different conditions of patient's head. By reasonably arranging multiple global markers, it can be ensured that a sufficient number of mark points and global markers can be captured at the same time during shooting, so as to realize more accurate registration and registration.

[0140] The method provided by the embodiments of the present application can provide an efficient, accurate and reliable solution for surgical navigation based on the registration and registration scheme of the marker headgear and the global marker. Not only can the contact with the patient and the need for secondary image scanning be reduced, but also different surgical scenes and different conditions of the patient's head can be coped with, which has important clinical application value.

[0141] Based on the foregoing embodiments, the embodiments of the present application provide a registration device, each module included in the device and each unit included in each module can be implemented by a processor in a computer device; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc.

[0142] The embodiments of the present application provide a registration device, Figure 6 The structure diagram of the registration device provided by the embodiments of the present application is shown in Figure 6 The registration device 600 includes:

[0143] The acquisition module 601 is configured to acquire target image information and image data of a target part of a target object, wherein the target image information includes a global marker and at least part of an encoding marker on a fitting component, the fitting component is used to fit on the target part, and the global marker is fixed relative to the target part.

[0144] The determination module 602 is configured to determine spatial position information of the encoding marker based on the global marker and the encoding marker in the target image information.

[0145] The reconstruction module 603 is configured to perform three-dimensional reconstruction based on the image data to obtain point cloud data corresponding to the target part.

[0146] The registration module 604 is configured to perform registration based on the spatial position information corresponding to the encoding marker and the point cloud data corresponding to the target part.

[0147] In some embodiments, the registration module includes:

[0148] The registration unit is configured to perform registration on the spatial position information corresponding to the encoding marker and the point cloud data corresponding to the target part by using an iterative closest point algorithm.

[0149] In some embodiments, the reconstruction module comprises:

[0150] A determination unit configured to determine initial point cloud data corresponding to the target part based on the image data;

[0151] An acquisition unit configured to acquire a thickness of the fitting component and a thickness of the coded marker;

[0152] A calculation unit configured to determine a target thickness based on the thickness of the fitting component and the thickness of the coded marker;

[0153] An adjustment unit configured to adjust the initial point cloud data based on the target thickness to obtain point cloud data corresponding to the target part.

[0154] In some embodiments, the determination module comprises:

[0155] A triangulation unit configured to determine spatial position information of the coded marker in a coordinate system of the global marker based on the global marker and the coded marker in the target image information by using a triangulation method.

[0156] In some embodiments, the coded marker comprises a code and a marker point, and the registration device further comprises:

[0157] A first acquisition module configured to, in a case where a user takes a picture through a navigation device, identify the code and the marker point in the image based on the acquired image;

[0158] A marker point determination module configured to determine a first marker point in a fitting component model based on the code and the marker point in the image.

[0159] A display module configured to display the fitting component model and distinguishively display the first marker point and a second marker point on the fitting component model in a first display mode and a second display mode, respectively, wherein the second marker point is a marker point remaining in the fitting component model after excluding the first marker point.

[0160] In some embodiments, the marker point determination module comprises:

[0161] A corresponding relationship determination unit configured to determine a corresponding relationship between a marker point in the image and a marker point on the fitting component based on the code in the image;

[0162] A marker point determination unit configured to determine a first marker point in a fitting component model based on the corresponding relationship.

[0163] In some embodiments, the registration device further comprises:

[0164] a quantity determination module configured to determine a quantity of the first marker points;

[0165] a percentage determination module configured to determine a collection progress percentage based on the quantity and a preset minimum collection quantity;

[0166] an output module configured to output the collection progress percentage, so that a user performs photographing based on the percentage to obtain the target image information, wherein a quantity of the coded markers in the target image information is greater than or equal to the minimum collection quantity.

[0167] An electronic device is provided in an embodiment of the present application. Figure 7 A schematic diagram of a constituent structure of the electronic device provided in an embodiment of the present application is shown in FIG. 3. Figure 7 As shown in FIG. 3, the electronic device 300 includes a processor 301, at least one communication bus 302, a user interface 303, at least one external communication interface 304, and a memory 305. The communication bus 302 is configured to realize connection and communication among the components. The user interface 303 can include a display screen, and the external communication interface 304 can include a standard wired interface and a wireless interface. The processor 301 is configured to execute a program of a registration method stored in the memory, to realize the steps in the registration method provided in the above embodiments.

[0168] In an embodiment of the present application, if the above-mentioned image establishment method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solutions of the embodiments of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk, and various media that can store program codes. Thus, the embodiments of the present application are not limited to any particular hardware and software combination.

[0169] Correspondingly, the embodiments of the present application provide a storage medium having a computer program stored thereon, and the computer program is executed by a processor to realize the steps in the registration method provided in the above embodiments.

[0170] The embodiments of the present application further provide a computer program product, which, when executed on a terminal device, causes an electronic device to perform the registration method described in any one of the above embodiments.

[0171] The descriptions of the above electronic device and storage medium embodiments are similar to the descriptions of the above method embodiments, and have similar beneficial effects as the method embodiments. For technical details not disclosed in the computer device and storage medium embodiments of the present application, please refer to the description of the method embodiments of the present application for understanding.

[0172] Based on the foregoing various embodiments, the embodiments of the present application further provide a registration system, comprising:

[0173] a fitting component, the fitting component being provided with coded markers, and the fitting component being used for fitting on a target part of a target object;

[0174] a global marker, the global marker being fixed relative to the target part;

[0175] a navigation device, the navigation device being used for collecting target image information, and the target image information including at least part of the coded markers on the fitting component and the global marker;

[0176] the above electronic device, the electronic device being in communication connection with the navigation device.

[0177] In the embodiments of the present application, the coded markers include codes and marker points, and the codes are used for distinguishing the marker points.

[0178] In some embodiments, the marker points are in a circular shape, a rectangular shape or a spherical shape.

[0179] In some embodiments, the marker points are made of a reflective material.

[0180] In some embodiments, the navigation device includes a camera.

[0181] In some embodiments, the registration system further includes:

[0182] a fixing support, the fixing support being used for fixing the target part of the target object.

[0183] In the embodiments of the present application, the fixing support can include a triangular support.

[0184] In some embodiments, the registration system further includes:

[0185] an imaging device, the imaging device being in communication connection with the electronic device, and the imaging device being used for collecting imaging data of the target part of the target object.

[0186] In the embodiments of the present application, the imaging device can be a CT device.

[0187] It should be understood that every feature, structure, or characteristic described herein is within a preferred embodiment of the present application. Thus, it is meant that the features, structures, or characteristics can be combined with each other in any manner within a preferred embodiment of the present application. In addition, it is contemplated that each feature, structure, or characteristic can be implemented in hardware, software, or a combination thereof.

[0188] It should be noted that, as used herein, the articles "a", "an", "the", and "at least one" are intended to mean that there is one or more of the elements in the preceding descriptions. The articles "a" (or "an"), as well as the first article "the" and "at least one" are intended to be interpreted as including both the singular and the plural, unless otherwise indicated. Thus, for example, "a" and "the" can include both the instance in which there is only one of the elements and the instance in which there is more than one of the elements.

[0189] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed components can be through some interfaces, indirect coupling or communication connection between devices or units, which can be electrical, mechanical or other forms.

[0190] The units described above as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units; they can be located in one place or distributed on multiple network units; some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0191] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be realized in the form of hardware or hardware plus software functional unit.

[0192] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROMs), magnetic disks, optical disks, and other media that can store program codes.

[0193] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a controller to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks or optical disks.

[0194] The above is merely an embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method of registration, characterized by, The method comprises: acquiring target image information and image data of a target part of a target object, wherein the target image information comprises a global marker and at least part of an encoded marker on a fitting component, the fitting component is used for fitting on the target part, and the global marker is fixed relative to the target part; determining spatial position information corresponding to the encoded marker based on the global marker and the encoded marker in the target image information; performing three-dimensional reconstruction based on the image data to obtain point cloud data corresponding to the target part; registering based on the spatial position information corresponding to the encoded marker and the point cloud data corresponding to the target part.

2. The method of claim 1, wherein, The three-dimensional reconstruction based on the image data to obtain the point cloud data corresponding to the target part comprises: determining initial point cloud data corresponding to the target part based on three-dimensional reconstruction of the image data; acquiring a thickness of the fitting component and a thickness of the encoded marker; determining a target thickness based on the thickness of the fitting component and the thickness of the encoded marker; adjusting the initial point cloud data based on the target thickness to obtain the point cloud data corresponding to the target part.

3. The method according to claim 1 or 2, characterized in that, The encoded marker comprises an encoded marker and a marker point, and the method further comprises: in a case where a user takes a picture through a navigation device, recognizing the encoded marker and the marker point based on the collected image; determining a first marker point in a fitting component model based on the encoded marker and the marker point in the image; displaying the fitting component model and distinguishing and displaying the first marker point and a second marker point in the fitting component model in a first display mode and a second display mode respectively, wherein the second marker point is a remaining marker point in the fitting component model after excluding the first marker point.

4. The method of claim 3, wherein, The determination of the first marker point in the fitting component model based on the encoded marker and the marker point in the image comprises: determining a corresponding relationship between the marker point in the image and a marker point on the fitting component based on the encoded marker in the image; determining the first marker point in the fitting component model based on the corresponding relationship.

5. The method of claim 3, wherein, The method further comprises: determining a number of the first marker points; determining a collection progress percentage based on the number and a preset minimum collection number; outputting the collection progress percentage, wherein the number of the first marker points in the target image information is greater than or equal to the minimum collection number.

6. A registration device characterized by The method comprises: an acquisition module, configured to acquire target image information and image data of a target part of a target object, wherein the target image information comprises a global marker and at least part of an encoded marker on a fitting component, the fitting component is used for fitting on the target part, and the global marker is fixed relative to the target part; a determination module, configured to determine spatial position information of the encoded marker based on the global marker and the encoded marker in the target image information; a reconstruction module, configured to perform three-dimensional reconstruction based on the image data to obtain point cloud data corresponding to the target part; a registration module, configured to register based on the spatial position information corresponding to the encoded marker and the point cloud data corresponding to the target part.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor implements the registration method as claimed in any one of claims 1 to 5 when executing the computer program.

8. A registration system characterized by Comprise: a fitting component, the fitting component being provided with coded markers, the fitting component being used for fitting on a target part of a target object; a global marker, the global marker being fixed relative to the target part; a navigation device, the navigation device being used for collecting target image information, the target image information including the global marker and at least part of the coded markers on the fitting component; The electronic device as claimed in claim 7, the electronic device being communicatively connected with the navigation device.

9. The registration system of claim 8, wherein, The coded markers include: codes and marker points, the codes being used for distinguishing the marker points.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the registration method as claimed in any one of claims 1 to 5.

Images