Image registration method, device, terminal device and computer-readable storage medium

Abstract

An embodiment of the present application provides an image registration method, apparatus, terminal device, and computer-readable storage medium, which relate to the field of image fusion. Obtain a first image sequence and a second image sequence; wherein, the first image sequence and the second image sequence are medical images of different modalities; register the first image sequence and the second image sequence to obtain first transformation parameters of the positions of each pixel point in the first image sequence with respect to the actual positions and second transformation parameters of the positions of each pixel point in the second image sequence with respect to the actual positions; determine the registration relationship between the first image sequence and the second image sequence according to the first transformation parameters and the second transformation parameters. Thereby, doctors can more conveniently, intuitively, and accurately observe the lesion structure from various angles during diagnosis, realize the accurate positioning and analysis of the lesion position, and are beneficial to the formulation of medical diagnosis, surgery, and radiotherapy plans.

Description

Technical Field

[0001] The present application relates to the field of image fusion, and in particular, to an image registration method, apparatus, terminal device, and computer-readable storage medium. Background Art

[0002] Nowadays, with the rapid development of computer and imaging technologies, radiotherapy technologies have also been continuously matured. When treating patients using radiotherapy technologies, it is necessary to accurately observe, analyze, and locate the lesion site.

[0003] Currently, generally, images containing accurate anatomical information are collected, and the position of the lesion is determined through the observation of doctors. However, this method requires doctors to perform spatial imagination based on different images and analyze and locate the lesion according to their subjective experience. Therefore, there is a problem that the lesion localization is not accurate enough, which will affect medical diagnosis and the formulation of surgical and radiotherapy plans. Summary of the Invention

[0004] In view of this, the purpose of the present application is to provide an image registration method, apparatus, terminal device, and computer-readable storage medium, so as to achieve accurate registration of medical images of different modalities, thereby enabling accurate localization of the lesion position during diagnosis, which is beneficial to medical diagnosis and the formulation of surgical and radiotherapy plans.

[0005] To achieve the above purpose, the technical solutions adopted in the embodiments of the present application are as follows:

[0006] In a first aspect, the present application provides an image registration method, the method including: obtaining a first image sequence and a second image sequence; wherein, the first image sequence and the second image sequence are medical images of different modalities; registering the first image sequence and the second image sequence to obtain a first transformation parameter of the position of each pixel point in the first image sequence with respect to the actual position, and a second transformation parameter of the position of each pixel point in the second image sequence with respect to the actual position; determining the registration relationship between the first image sequence and the second image sequence according to the first transformation parameter and the second transformation parameter.

[0007] In a second aspect, the present application provides an image registration apparatus, the apparatus including: an obtaining module, configured to obtain a first image sequence and a second image sequence; wherein, the first image sequence and the second image sequence are medical images of different modalities; a registration module, configured to register the first image sequence and the second image sequence to obtain a first transformation parameter of the position of each pixel point in the first image sequence with respect to the actual position, and a second transformation parameter of the position of each pixel point in the second image sequence with respect to the actual position; a registration module, configured to determine the registration relationship between the first image sequence and the second image sequence according to the first transformation parameter and the second transformation parameter.

[0008] In a third aspect, the present application provides a terminal device, including a processor and a memory. The memory stores a computer program that can be executed by the processor, and the processor can execute the computer program to implement the image registration method according to any one of the foregoing embodiments.

[0009] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the image registration method according to any one of the foregoing embodiments.

[0010] The image registration method, device, terminal device, and computer-readable storage medium provided by the embodiments of the present application register different modalities of medical images to obtain the first transformation parameters between each pixel point in the first image sequence and the actual position, and the second transformation parameters between each pixel point position in the second image sequence and the actual position. Then, according to the first transformation parameters and the second transformation parameters, the registration relationship between the first image sequence and the second image sequence is determined. Taking the actual position as a medium, accurate image registration between the first image sequence and the second graphic sequence is achieved, enabling doctors to more conveniently, intuitively, and accurately observe the lesion structure from various angles during diagnosis, realizing the accurate positioning and targeted analysis of the lesion position, which is beneficial to medical diagnosis and the formulation of surgical and radiotherapy plans.

[0011] To make the above objects, features, and advantages of the present application more obvious and understandable, the following specifically enumerates preferred embodiments and, in conjunction with the accompanying drawings, makes the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] To more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required in the embodiments. It should be understood that the following drawings only show some embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0013] Figure 1 Shows a schematic flowchart of an image registration method provided by an embodiment of the present application;

[0014] Figure 2 Shows another schematic flowchart of an image registration method provided by an embodiment of the present application;

[0015] Figure 3 Shows still another schematic flowchart of an image registration method provided by an embodiment of the present application;

[0016] Figure 4 Shows a schematic diagram of a straight line in the second image sequence;

[0017] Figure 5 Shows another schematic flowchart of the image registration method provided by the embodiment of the present application;

[0018] Figure 6 Shows another schematic flowchart of the image registration method provided by the embodiment of the present application;

[0019] Figures 7A - 7B Shows a schematic diagram of the center point of the first image sequence;

[0020] Figure 8 Shows another schematic flowchart of the image registration method provided by the embodiment of the present application;

[0021] Figure 9 Shows another schematic flowchart of the image registration method provided by the embodiment of the present application;

[0022] Figure 10 Shows a functional module diagram of the image registration device provided by the embodiment of the present application;

[0023] Figure 11 Shows a schematic block diagram of the terminal device provided by the embodiment of the present application.

[0024] Icons: 100 - Acquisition module; 110 - Registration module; 120 - Registration module; 20 - Second image; 21 - Second straight line; 200 - Terminal device; 210 - Memory; 220 - Processor; 230 - Communication module. Detailed implementation manners

[0025] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Usually, the components of the embodiments of the present application described and shown in the accompanying drawings here can be arranged and designed in various different configurations.

[0026] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application to be protected, but only represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0027] It should be noted that relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0028] At present, radiotherapy technology is often used for tumor treatment. Since a certain dose needs to be set based on accurate localization and analysis of the lesion when formulating a radiotherapy plan nowadays, accurate localization of the lesion is crucial for radiotherapy technology.

[0029] In traditional technology, doctors often need to perform targeted analysis on the lesion through multiple image sequences containing anatomical information to determine the location of the lesion, and then conduct medical diagnosis and formulate corresponding surgical and radiotherapy plans. However, the multiple image sequences containing anatomical information obtained currently are all independent image sequences. When doctors perform analysis, they can only judge the location of the lesion through spatial imagination and subjective experience. This localization method obviously makes the analysis process of doctors difficult and there are localization errors, which will further affect the medical diagnosis of doctors and the formulation of surgical plans.

[0030] In addition, the image sequences obtained by shooting may have errors caused by medical imaging equipment. When analyzing the lesion, multiple image sequences of the patient can be collected using different medical imaging equipment. There may be errors caused by different medical imaging equipment in the multiple image sequences, thus affecting the analysis conclusion of doctors on the same lesion site, resulting in inaccurate localization of the lesion, and further affecting the formulation of surgical and radiotherapy plans.

[0031] To solve the above problems, the present application proposes an image registration method to achieve accurate registration of medical images of different modalities, so as to accurately locate the lesion during diagnosis, which is beneficial to medical diagnosis and the formulation of surgical and radiotherapy plans.

[0032] Based on this, taking the terminal device as the execution subject, a possible implementation manner is given below to exemplarily illustrate how the present application implements image registration. Specifically, Figure 1 A schematic flowchart of an image registration method provided by an embodiment of the present application is shown in Figure 1 and the method includes:

[0033] Step S20: Obtain a first image sequence and a second image sequence.

[0034] Among them, the first image sequence and the second image sequence are medical images of different modalities. Exemplarily, the first image sequence and the second image sequence can be image sequences containing accurate anatomical information, such as medical images like CT images, MRI images, or PET images.

[0035] In this embodiment, the first image sequence and the second image sequence can be medical images acquired by the same medical imaging device, or can be medical images acquired by different medical imaging devices. For example, the first image sequence and the second image sequence can both be Computed Tomography (CT) images or both be Magnetic Resonance Imaging (MRI) images, or can be a CT image and an MRI image respectively. This application does not make any limitations in this regard and can be set according to specific needs.

[0036] In this embodiment, the medical images of different modalities can be medical images of the same patient part under different imaging fields of view. For example, the first image sequence and the second image sequence can be an image sequence including the complete structure of the patient's photographed part and an image sequence including the local structure of the patient's photographed part respectively. In one example, the first image sequence can reflect the complete abdominal structure of the patient, while the second image sequence can reflect the gastric structure of the patient.

[0037] In addition, the medical images of different modalities can also be medical images taken at different times that reflect the same part. For example, the first image sequence can be an image sequence of the patient's local part taken at the current time; the second image sequence can be an image sequence of the patient's same part taken some time ago. In one example, the first image sequence can reflect the gastric structure of the patient obtained by a CT device on the day of diagnosis, and the second image sequence can reflect the gastric structure of the patient obtained by an MRI device two days before the diagnosis.

[0038] It can be understood that if the medical images of different modalities reflect the medical images of the same photographed part under different imaging fields of view, then multi-angle analysis can be performed based on these medical images of different modalities to achieve precise positioning of the lesion location during diagnosis; if the medical images of different modalities are taken at different times and reflect the medical images of the same part, then an image registration relationship of the same part at different times can be established, and further, the changes of the lesion and the organ can be quantitatively analyzed, which is beneficial to the formulation of the subsequent radiotherapy plan.

[0039] Exemplarily, both the first image sequence and the second image sequence can feedback the structure of the patient's imaged part in a three-dimensional space. The first image sequence can include multiple first images. It can be understood that each first image is a slice image of the target imaged part with the same slice angle but different slice positions. The multiple first images can be placed in sequence according to the slice positions to form the first image sequence, that is, the three-dimensional image of the target imaged part, so as to completely represent the structure of the target imaged part. The second image sequence can include multiple second images. It can be understood that each second image is a slice image of the target imaged part with the same slice angle but different slice positions. The multiple second images can be placed in sequence according to the slice positions to form the first image sequence, that is, the three-dimensional image of the target imaged part, so as to completely represent the structure of the target imaged part.

[0040] In this embodiment, the medical imaging device can be communicatively connected to the terminal device. After the medical imaging device captures the first image sequence and the second image sequence, it can send the first image sequence and the second image sequence to the terminal device. When image registration is required, the terminal device can directly obtain the first image sequence and the second image sequence.

[0041] Step S21: Register the first image sequence and the second image sequence to obtain the first transformation parameters of the positions of each pixel point in the first image sequence with respect to the actual positions, and the second transformation parameters of the positions of each pixel point in the second image sequence with respect to the actual positions.

[0042] In this embodiment, corresponding registration software or registration plug-ins can be installed on the terminal device. After obtaining the first image sequence and the second image sequence, the terminal device can respectively calculate the first transformation parameters of the positions of each pixel point in the first image sequence with respect to the actual positions, and the second transformation parameters of the positions of each pixel point in the second image sequence with respect to the actual positions, and save them to the terminal device, thereby completing the registration of the first image sequence and the second image sequence.

[0043] It can be understood that the registration process of the first image sequence and the second image sequence is the process of calculating the first transformation parameters of the first image sequence and the second transformation parameters of the second image sequence and saving them.

[0044] Step S22: Determine the registration relationship between the first image sequence and the second image sequence according to the first transformation parameters and the second transformation parameters.

[0045] In this embodiment, since the first transformation parameters are the transformation parameters of the positions of each pixel point in the first image sequence with respect to the actual positions, and the second transformation parameters are the transformation parameters of the positions of each pixel point in the second image sequence with respect to the actual positions, therefore, the actual positions can be used as a medium to establish the registration relationship between the first image sequence and the second image sequence according to the first transformation parameters and the second transformation parameters.

[0046] For example, after obtaining the registration relationship between the first image sequence and the second image sequence, an association relationship between the first image sequence and the second image sequence may be established according to the registration relationship, so that an operation on either image sequence may be synchronized to the other image sequence according to the registration relationship.

[0047] In one example, taking the case where the first image sequence feeds back the complete structure of the patient's head and the second image sequence feeds back the lesion structure of the patient's head, and synchronizing the operations in the second image sequence to the first image sequence, the doctor operates the second image sequence through the terminal device, such as outlining the lesion site. Since the first image sequence and the second image sequence have established an association relationship, the terminal device can synchronize the outlining information to the first image sequence according to the registration relationship, and then can synchronously feedback the outlining information in the first image sequence, so that the doctor can not only observe the outlining site of the lesion based on the perspective of the local structure, but also observe the outlining site of the lesion based on the perspective of the overall structure of the patient's head, thereby achieving multi-angle observation.

[0048] The image registration method provided in the embodiment of the present application first acquires a first image sequence and a second image sequence, wherein the first image sequence and the second image sequence are medical images of different modalities, then registers the first image sequence, obtains a first transformation parameter between the position of each pixel point in the first image sequence and the actual position, and a registration relationship between the position of each pixel point in the second image sequence and the actual position, and finally determines the registration relationship between the first image sequence and the second image sequence based on the first transformation parameter and the second transformation parameter, and realizes accurate image registration of the first image sequence and the second image sequence with the actual position as the medium, so that the doctor can observe the lesion structure from various angles more conveniently, intuitively and accurately when making a diagnosis, realizes accurate positioning and targeted analysis of the lesion position, and is beneficial to medical diagnosis and the formulation of surgery and radiotherapy plans.

[0049] In one of the embodiments, taking into account that medical imaging equipment may produce errors due to machine or environmental influences during shooting, in order to compensate for possible errors and facilitate shooting of the patient's body parts, a reference device may be set to fix the patient's shooting part, and the first transformation parameters of the position of each pixel point in the first image sequence and the actual position, as well as the second transformation parameters of the position of each pixel point in the second image sequence and the actual position are calculated based on the feature points on the reference device.

[0050] In one embodiment, in Figure 1 On the basis of Figure 2 For another flow chart of the image registration method provided in the embodiment of the present application, please refer to Figure 2 , step S21 can be implemented by the following steps:

[0051] Step S21-1: Obtain the characteristic point coordinates of the reference device.

[0052] Exemplarily, the reference device can be a device that can fix the photographed part of the patient therein and serve as a reference, such as a fixing frame. In one example, if the part to be photographed is the patient's head, the reference device can be a headrest that can fix the head therein.

[0053] Exemplarily, the characteristic point coordinates can be the coordinates in the original coordinate system. The original coordinate system can be set in advance, and the characteristic point coordinates of the reference device in the original coordinate system are obtained. Therefore, the terminal device can directly obtain the characteristic point coordinates of the reference device in the original coordinate system. In a possible implementation manner, the characteristic point can be a point at a special position of the reference device, such as the center point of the reference device, or a point set in advance at a special position of the reference device, such as each vertex of the reference device.

[0054] Exemplarily, the original coordinate system is a pre-stored standard coordinate system. In a possible implementation manner, the original coordinate system can be a Digital Imaging and Communications in Medicine (DICOM) coordinate system, and the DICOM coordinate system defines a medical image format that can be used for data exchange with quality meeting clinical requirements.

[0055] Step S21-2: Identify the marker points in the first image sequence and the marker points in the second image sequence, and determine the first coordinates of the marker points in the first image sequence and the second coordinates of the marker points in the second image sequence.

[0056] Exemplarily, when photographing the photographed part of the patient, the photographed part can be fixed in the above-mentioned reference device. Therefore, when photographing the photographed part of the patient, the reference device will also be photographed. The reference device is shown in each first image in the first image sequence and each second image in the second image sequence in the form of marker points. Specifically, the reference device can be a headrest, and a developing component that can be developed under medical imaging devices such as CT and MRI is provided on the headrest. For example, N-shaped developing components are provided on each surface of the headrest. The following embodiments are all described by taking the reference device as a headrest and the headrest being provided with N-shaped developing components as an example.

[0057] It can be understood that the marked points are the points reflected by the reference device on each first image in the first image sequence during shooting, that is, the developed positions of the head frame in the image. Therefore, the feature points in the first image sequence can be calculated based on the marked points on each first image. Obviously, the calculated feature points in the first image sequence and the feature points in the above original coordinate system are in a one-to-one correspondence. In one example, if the calculated feature point in the first image sequence is the center point of the first image sequence, then the center point of the first image sequence corresponds to the center point of the reference device in the original coordinate system.

[0058] Exemplarily, each first image and each second image may include multiple marked points. In one possible implementation, the number of marked points may be six.

[0059] Step S21-3: Determine the first transformation parameter between the positions of each pixel point in the first image sequence and the actual positions, and the second transformation parameter between the positions of each pixel point in the second image sequence and the actual positions according to the first coordinates, the second coordinates, and the feature point coordinates.

[0060] In the image registration method provided by the embodiments of the present application, since the points on the reference device, that is, the marked points, can be reflected in the first image sequence and the second image sequence during shooting, the terminal device can identify the marked points in the first image sequence and the marked points in the second image sequence, and determine the first coordinates of the marked points in the first image sequence and the second coordinates of the marked points in the second image sequence. Then, by combining the feature point coordinates of the reference device, the first transformation parameter between the positions of each pixel point in the first image sequence and the actual positions and the second transformation parameter between the positions of each pixel point in the second image sequence and the actual positions are calculated, so that the errors existing during the shooting of the medical imaging device can be compensated, thereby reducing the errors and ensuring the registration accuracy of the first image sequence and the second image sequence.

[0061] In one of the embodiments, since each first image in the first image sequence and each second image in the second image sequence both include multiple marked points, multiple straight lines can be generated in the first image sequence according to the marked points at the same positions on each first image, and multiple straight lines can be generated in the second image sequence according to the marked points at the same positions on each second image.

[0062] Exemplarily, each N-shaped developed component on the head frame can be displayed in both the first image sequence and the second image sequence. Taking one N-shaped developed component displayed in the first image sequence as an example, since the first image is a slice image, each first image shows a cross-section of the N-shaped line, that is, three points. Multiple first images form the first image sequence, and the three points in each first image are connected to form an N-shaped line.

[0063] Based on this, the first transformation parameter and the second transformation parameter can be calculated through the straight lines in the first image sequence and the straight lines in the second image sequence. Specifically, on the basis of Figure 2 the Figure 3 FIG. Figure 3 is another schematic flowchart of the image registration method provided by the embodiments of the present application. Please refer to Figure 3 , and the above step S21-3 can be implemented through the following steps:

[0064] Step S21-3-1: Obtain the unit vectors in each direction in the original coordinate system.

[0065] Step S21-3-2: Fit a first straight line equation according to multiple first coordinates.

[0066] Exemplarily, the marked points at the same position on different first images can be fitted into a straight line, so as to obtain the first straight line equation. Specifically, it is first possible to judge the positions of the respective marked points on the first image where they are located, take the currently to-be-judged marked point as the target marked point, and take the first image where the target marked point is located as the target first image, then the position of the target marked point in the target first image can be judged according to the first coordinate of the target marked point.

[0067] Since each first image includes multiple marked points, the positions of all the marked points on the target first image can thus be obtained, and further the positions of all the marked points on each first image can be obtained. Therefore, the marked points at the same position on each first image can be regarded as constituting a straight line, and thus the first coordinates of the marked points constituting the same straight line can be fitted to obtain the first straight line equation.

[0068] It can be understood that the first image sequence may include multiple first straight lines, so that multiple first straight line equations can be fitted and obtained.

[0069] Exemplarily, in order to determine the position of the target marked point in the target first image, the first image can be divided into multiple regions in advance, and it can be judged which region of the target first image the target marked point is located in according to the first coordinate of the target marked point, so as to determine the position of the marked point in the first image where it is located; in addition, the position of the target marked point can also be determined by judging the position of the target marked point relative to other marked points. For example, the target marked point is located in the upper left of the positions of all other marked points.

[0070] Step S21-3-3: Fit a second straight line equation according to multiple second coordinates;

[0071] Exemplarily, the fiducial points at the same position on different second images can be fitted into a straight line to obtain the second straight line equation. Specifically, first, the position of each fiducial point on its corresponding second image can be determined. Taking the fiducial point to be determined currently as the target fiducial point and the second image where the target fiducial point is located as the target second image, the position of the target fiducial point in the target second image can be determined according to the second coordinate of the target fiducial point.

[0072] Since each second image includes multiple fiducial points, the positions of all fiducial points on the target second image can be obtained, and further the positions of all fiducial points on each second image can be obtained. Therefore, the fiducial points at the same position on each second image can be regarded as forming a straight line, and thus the second coordinates of the fiducial points forming the same straight line can be fitted to obtain the second straight line equation.

[0073] It can be understood that the second image sequence may include multiple second straight lines, and thus multiple second straight line equations can be fitted and obtained.

[0074] Exemplarily, to determine the position of the target fiducial point in the target second image, the second image can be divided into multiple regions in advance, and according to the second coordinate of the target fiducial point, it can be determined which region the target fiducial point is located in the target second image, so as to determine the position of the fiducial point in its corresponding second image; in addition, the position of the target fiducial point can also be determined by judging the position of the target fiducial point relative to other fiducial points. For example, the target fiducial point is located in the upper left of the positions of all other fiducial points.

[0075] In one example, taking the fitting of the second straight line equation as an example, if each second image includes 6 fiducial points, 6 straight lines can be determined in the second image sequence. On a target second image, the 6 fiducial points are respectively a, b, c, d, e, f. Then, according to the second coordinates of the above 6 points, the positions of each fiducial point in the target second image can be determined respectively. Taking the point a as the target fiducial point as an example, if the point a is located in the upper right of the positions of all other fiducial points, it is determined that the target fiducial point a is located in the upper right of the target second image.

[0076] In this way, the positions of each fiducial point in all second images can be determined, and further the fiducial points at the same position in each second image and their second coordinates can be determined. Therefore, a straight line can be determined according to the fiducial points at the same position in each second image, and further the second straight line equation can be obtained by fitting the second coordinates of the fiducial points on this straight line.

[0077] Taking the determination of the fiducial points at the upper left of each second image and their second coordinates as an example, please refer to Figure 4, a second straight line 21 formed by the marked points located in the upper left corner of each second image 20 can be obtained, and a second straight line equation obtained by fitting the second coordinates of the marked points located in the upper right corner of each second image. By analogy, the second straight line equations of the 6 straight lines in the second image sequence can be obtained respectively.

[0078] Step S21-3-4, determine the first transformation parameter according to the first straight line equation, the characteristic point coordinates and the unit vector;

[0079] Step S21-3-5, determine the second transformation parameter according to the second straight line equation, the characteristic point coordinates and the unit vector.

[0080] In this embodiment, the first transformation parameter can be calculated respectively according to the first straight line equation, the characteristic point coordinates and the unit vector, and the second transformation parameter can be calculated according to the second straight line equation, the characteristic point coordinates and the unit vector.

[0081] In one embodiment, the first transformation parameter includes a first translation parameter and a first rotation parameter, and the second transformation parameter includes a second translation parameter and a second rotation parameter.

[0082] In one embodiment, the first translation parameter is the translation distance of each pixel point position in the first image sequence relative to the actual position, and the first rotation angle is the rotation angle of the image coordinate system in the first image sequence relative to the standard original coordinate system; the second translation parameter is the translation distance of each pixel point position in the second image sequence relative to the actual position, and the second rotation angle is the rotation angle of the image coordinate system in the second image sequence relative to the standard original coordinate system.

[0083] Based on this, in a possible implementation manner, the first transformation parameter can be calculated through the following steps. Specifically, in Figure 3 on the basis of Figure 5 is another process schematic diagram provided by the embodiments of the present application. Please refer to Figure 5 , the above step S21-3-4 can be implemented through the following steps:

[0084] Step S21-3-4-1, determine the center point coordinates of the first image sequence and the normal vectors in each direction of the first image sequence according to the first straight line equation;

[0085] Optionally, the center point coordinates of the first image sequence are located in the image coordinate system.

[0086] Step S21-3-4-2, calculate the first translation parameter of the position of each pixel point in the first image sequence relative to the actual position according to the center point coordinates and the characteristic point coordinates of the first image sequence;

[0087] Exemplarily, since the center point coordinates of the first image sequence are in the image coordinate system, while the feature point coordinates are in the original coordinate system, it is necessary to convert the two coordinate systems, so as to convert the coordinate values of each pixel point in the image coordinate system into the coordinate values of each pixel point in the original coordinate system, which is convenient for the calculation of the first translation parameter.

[0088] Exemplarily, the feature point coordinates may be the center point coordinates of a reference device pre-stored in the original coordinate system. Since the marker points on each first image in the first image sequence are the points of the reference device reflected on each first image, therefore, the center point of the first image sequence determined according to the first straight line equation is the point of the center point of the reference device reflected in the first image sequence. Based on this, the center point translation distance can be calculated according to the center point coordinates of the first image sequence and the center point coordinates of the reference device, and this center point translation distance is the first translation parameter of the positions of each pixel point and the actual positions in the first image sequence.

[0089] Step S21-3-4-3: Calculate the first rotation parameter of the positions of each pixel point and the actual positions in the first image sequence according to the normal vectors and unit vectors of the first image sequence in each direction.

[0090] Optionally, the unit vector is the unit vector in each direction of the aforementioned original coordinate system.

[0091] Optionally, the normal vectors of the first image sequence in each direction include the normal vectors of the first image sequence in the X, Y, and Z directions respectively. The center point coordinates of the first image sequence and the normal vectors of the first image sequence in the X, Y, and Z directions respectively can be calculated in the following manner. Specifically, on the Figure 5 basis, Figure 6 For another flowchart of the image registration method provided by the embodiment of the present application, please refer to Figure 6 , the above step S21-3-4-1 can be implemented through the following steps:

[0092] Step S21-3-4-1-1: Determine the center point coordinates of the first image sequence and the normal vector of the first image sequence in the Z direction according to multiple first straight line equations.

[0093] Exemplarily, the first image sequence may include multiple first straight lines, and the center point coordinates of the first image sequence and the normal vector of the first image sequence in the Z direction can be determined according to multiple first straight line equations.

[0094] Exemplarily, since the first image sequence consists of multiple first images, the first image sequence can feedback the imaged part of the patient in a three-dimensional space. In a possible implementation manner, the multiple first straight lines at least include four vertical lines approximately perpendicular to the XY plane, and oblique lines that can intersect with different two vertical lines respectively. Since two straight lines can determine a plane, at least two planes can be determined in the first image sequence.

[0095] In one example, if the first image sequence includes six first straight lines, and the six first straight lines include four vertical lines approximately perpendicular to the XY plane, and two oblique lines that intersect with different two vertical lines respectively. Since the two oblique lines intersect with different two vertical lines respectively, therefore, a plane can be determined respectively according to two vertical lines and one oblique line, that is, two non-intersecting planes can be determined in the first image sequence.

[0096] Specifically, please refer to Figure 7A , two vertical lines and one oblique line can determine two non-intersecting planes A and B in the first image sequence. Taking plane A as an example, please refer to Figure 7B , vertical line a and vertical line b intersect with oblique line c respectively, thus forming intersection points M and N. Then, the coordinates of intersection points M and N can be calculated according to the first straight line equations of vertical line a, vertical line b, and oblique line c. Taking M and N as two endpoints, line segment MN can be formed. Therefore, according to the coordinates of point M and the coordinates of point N, the center point H1 of line segment MN can be determined, and then the center point H1 is used as the center point of plane A. On this basis, the center point H2 of plane B can also be calculated in the same way.

[0097] After calculating the coordinates of the center point H1 of plane A and the coordinates of the center point H2 of plane B, please continue to refer to Figure 7A , the coordinates of the midpoint H of the connection line between point H1 and point H2 can be calculated according to the coordinates of point H1 and the coordinates of point H2, and the coordinates of point H are the center point coordinates of the first image sequence.

[0098] Exemplarily, in order to obtain the normal vector of the first image sequence in the Z direction, the slopes of the four vertical lines approximately perpendicular to the XY plane can be calculated respectively, and the average slope of all the vertical lines can be calculated according to the calculated slopes of each vertical line. The average slope of all the vertical lines is the normal vector of the first image sequence in the Z direction.

[0099] Step S21-3-4-1-2, determine the oblique line equation in the multiple first straight line equations, and calculate the average oblique line slope according to the oblique line equation;

[0100] Exemplarily, since multiple straight lines in the first image sequence may include vertical lines and oblique lines, the oblique lines among them can be determined, and the slope of the oblique lines can be calculated according to the oblique line equations of the oblique lines. Taking the example that there are 6 first straight lines in the first sequence, including 4 vertical lines and 2 oblique lines, the slope of each oblique line can be calculated respectively according to the first straight line equations of the 2 oblique lines, and then the average slope of the 2 oblique lines can be calculated according to the slope of each oblique line. The average slope of the oblique lines is the average oblique line vector.

[0101] Step S21-3-4-1-3: Calculate the normal vector of the first image sequence in the X direction according to the normal vector of the first image sequence in the Z direction and the average slope of the oblique lines.

[0102] Exemplarily, the normal vector in the Z direction and the average oblique line vector can be cross-multiplied, and the cross-multiplication result is used as the normal vector of the first image sequence in the X direction.

[0103] Step S21-3-4-1-4: Calculate the normal vector of the first image sequence in the Y direction according to the normal vector of the first image sequence in the Z direction and the normal vector of the first image sequence in the X direction.

[0104] Optionally, after obtaining the normal vector of the first image sequence in the X direction, the normal vector in the Z direction and the normal vector of the first image sequence in the X direction can be cross-multiplied, and the cross-multiplication result is used as the normal vector of the first image sequence in the Y direction.

[0105] In one embodiment, the first rotation parameter may be the Euler angles of the image coordinate system in the first image sequence relative to the original coordinate system in each direction. Since the unit vectors in the original coordinate system are pre-obtained, including the unit vectors in the X, Y, and Z directions, after obtaining the normal vectors of the first image sequence in the X, Y, and Z directions respectively, the first rotation parameter of the position of each pixel point in the first image sequence relative to the actual position can be calculated in the following manner:

[0106] Calculate the direction cosine matrix of the position of each pixel point in the first image sequence relative to the actual position through the following formula:

[0107]

[0108] Wherein, I, J, and K are the unit vectors in the X, Y, and Z directions in the original coordinate system respectively, and i, j, and k are the normal vectors in the X, Y, and Z directions obtained in the first image sequence respectively.

[0109] i G Represents the vector i in the original coordinate system, and i G= {I.i, I.j, I.k}, where I.i represents the scalar dot product of the normal vector i and the unit vector I, I.j represents the scalar dot product of the normal vector j and the unit vector I, and I.k represents the scalar dot product of the normal vector k and the unit vector I; j G represents the vector j in the original coordinate system, and j G = {J.i, J.j, J.k}, where J.i represents the scalar dot product of the normal vector i and the unit vector J, J.j represents the scalar dot product of the normal vector j and the unit vector J, and J.k represents the scalar dot product of the normal vector k and the unit vector J; k G represents the vector k in the original coordinate system, and k G = {K.i, K.j, K.k}, where K.i represents the scalar dot product of the normal vector i and the unit vector K, K.j represents the scalar dot product of the normal vector j and the unit vector K, and K.k represents the scalar dot product of the normal vector k and the unit vector K.

[0110] represents the expansion terms of G , j G , k G .

[0111] represents the cosine of the angles of all possible combinations of the normal vector of the first image sequence and the unit vectors of the original coordinate system. That is:

[0112] I.i = |I||i| * cos(I, i) = cos(I, i)

[0113] DCM G represents the direction cosine matrix.

[0114] After obtaining the direction cosine matrix, the Euler angles can be calculated through the rotation matrix, and the rotation matrix is:

[0115] where R represents the rotation matrix. On this basis, the Euler angles in each direction can be calculated through the following formula:

[0116] θ X = atan 2(cos(K, j), cos(K, k))

[0117] cos(K, j) 2

[0118]

[0119] θ Z = atan 2(cos(J, i), cos(I, i))

[0120] where θX is the Euler angle of the image coordinate system in the first image sequence relative to the original coordinate system in the X direction, θ Y is the Euler angle of the image coordinate system in the first image sequence relative to the original coordinate system in the Y direction, θ Z is the Euler angle of the image coordinate system in the first image sequence relative to the original coordinate system in the Z direction.

[0121] Exemplarily, the Euler angles θ X , θ Y and θ Z of the image coordinate system in the first image sequence relative to the original coordinate system in the X, Y, and Z directions together constitute the first rotation parameter. Optionally, in order to calculate the second translation parameter and the second rotation parameter, based on Figure 3 , Figure 8 FIG. Figure 8 is another schematic flowchart provided by the embodiment of the present application. Please refer to

[0122] Step S21-3-5-1: Determine the center point coordinates of the second image sequence and the normal vectors in each direction of the second image sequence according to the second straight line equation;

[0123] Step S21-3-5-2: Calculate the second translation parameter between the positions of each pixel point and the actual position in the second image sequence according to the center point coordinates and the feature point coordinates of the second image sequence;

[0124] Step S21-3-5-3: Calculate the second rotation parameter between the positions of each pixel point and the actual position in the second image sequence according to the normal vectors and the unit vectors in each direction of the second image sequence.

[0125] It can be understood that the second translation parameter and the second rotation parameter of the second image sequence can be calculated in the same way as the first image sequence, which will not be elaborated here.

[0126] For the image registration method provided by the embodiment of the present application, after calculating the first translation parameter, the first rotation parameter, the second translation parameter, and the second rotation parameter, the first transformation parameter of each pixel point in the first image sequence relative to the actual position, and the second transformation parameter of each pixel point in the second image sequence relative to the actual position can be obtained. Therefore, the first image sequence can be corrected according to the first transformation parameter, and the second image sequence can be corrected according to the second transformation parameter, so as to reduce the error caused by device hardware or environment, facilitate the doctor to perform subsequent diagnosis and analysis based on the images, and formulate a radiotherapy plan.

[0127] Optionally, to further reduce the error, after obtaining the first transformation parameters between the positions of each pixel point in the first image sequence and the actual positions respectively, and the second transformation parameters between the positions of each pixel point in the second image sequence and the actual positions, the first image in the first image sequence and the second image in the second image sequence can be screened before registration. Specifically, based on Figure 2 and Figure 9 which is another schematic flowchart provided by the embodiments of the present application, please refer to Figure 9 , the method further includes:

[0128] Step S23, obtaining the first reference coordinate of the first reference point and the second reference coordinate of the second reference point;

[0129] The first reference point is the point generated on each first image after the reference device is mapped to the first image sequence, and the second reference point is the point generated on each second image after the reference device is mapped to the second image sequence.

[0130] Exemplarily, since the size of the reference device is fixed, and the centers of the first image sequence and the second image sequence calculated both correspond to the center of the reference device in the original coordinate system, therefore, the centers of the first image sequence and the second image sequence can be used as the center of the reference device respectively, and according to the known size of the reference device, the reference device is mapped into the first image sequence and the second image sequence, so as to generate multiple reference points in each first image of the first image sequence and each second image of the second image sequence.

[0131] It can be understood that the marked points are the points presented in the image when the reference device is collected into the image by the medical imaging device during image acquisition, and the reference points are the points directly mapped in the image sequence with the center of the image sequence as the center of the reference device according to the size of the reference device. Since there may be certain errors during the image acquisition by the medical imaging device, therefore, the marked points are the points with certain errors reflected by the reference device in the image, while the reference points are the points without errors mapped by the reference device in the image, and the marked points and the reference points are in one-to-one correspondence.

[0132] In one example, taking the first image as an example, 6 marked points at different positions in the same first image respectively correspond to 6 reference points at the same positions as them, that is, a marked point at the upper left corresponds to a reference point at the upper left, and a marked point at the upper right corresponds to a reference point at the upper right.

[0133] Step S24, calculating the first deviation value of the marked points in each first image in the first image sequence relative to the first reference point, and the second deviation value of the marked points in each second image in the second image sequence relative to the second reference point;

[0134] Among them, the first deviation value is obtained from the first coordinate and the first reference coordinate, and the second deviation value is obtained from the second coordinate and the second reference coordinate;

[0135] Exemplarily, since the reference points and the marked points in each first image and each second image correspond one by one, and the marked points are the points with certain errors reflected by the reference device in the image, and the reference points are the points without errors mapped by the reference device in the image, the first deviation value can be calculated according to the coordinates of the marked points and the reference points in each first image in the first image sequence, and the second deviation value can be calculated according to the coordinates of the marked points and the reference points in each second image in the second image sequence.

[0136] It can be understood that since each first image contains multiple marked points, multiple reference points can also be correspondingly generated in the same first image, so that multiple first deviation values can be calculated in the same first image; since each second image contains multiple marked points, multiple reference points can also be correspondingly generated in the same second image, so that multiple second deviation values can be calculated in the same second image.

[0137] In one example, taking the first image as an example, if there are 6 marked points in the first image, 6 reference points can be correspondingly generated, so 6 first deviation values can be calculated in the first image.

[0138] Step S25, when the first deviation value meets the first preset condition, it is determined that the first image sequence meets the registration condition;

[0139] Step S26, when the second deviation value meets the second preset condition, it is determined that the second image sequence meets the registration condition.

[0140] Exemplarily, the first preset condition and the second preset condition can be set according to the photographed part and the first image sequence and the second image sequence, so as to judge whether the first image sequence and the second image sequence meet the registration condition according to the preset condition. If they do not meet the registration condition, the image sequence needs to be re-obtained for registration.

[0141] Exemplarily, the first preset condition can be that the number of first images meeting the screening condition reaches a preset first quantity threshold, and the second preset condition can be that the number of second images meeting the screening condition reaches a preset second quantity threshold.

[0142] Exemplarily, when setting the first quantity threshold and the second quantity threshold, it should be ensured that there are enough first images and second images to reflect the structure of the photographed part.

[0143] In one example, if the imaging part is the head, the first image sequence reflects the complete structure of the patient's head, and the second image sequence reflects the lesion structure of the patient's head, then the first quantity threshold can be set to be greater than the second quantity threshold. For example, the first quantity threshold can be set to 25 images, and the second quantity threshold can be set to 15 images.

[0144] Optionally, the screening condition can be that the first deviation value is not greater than a preset deviation threshold, and the second deviation value is not greater than the preset deviation threshold.

[0145] In one example, the deviation threshold can be preset, such as 2 mm. If the first deviation value or the second deviation value is not greater than the deviation threshold, it is considered that the marker point meets the deviation range. When the deviation values of the marker points in the first image or the second image all meet the deviation range, it is considered that the first image or the second image meets the screening condition. Obviously, if any marker point in the first image or the second image does not meet the deviation range, it is considered that the first image or the second image does not meet the screening condition. To avoid affecting the feedback on the imaging structure, the first image or the second image that does not meet the screening condition can be deleted.

[0146] The image registration method provided by the embodiments of the present application calculates the first deviation value of the marker points of each first image in the first image sequence relative to the first reference point, and the second deviation value of the marker points of each second image in the second image sequence relative to the second reference point by obtaining the first reference coordinate of the first reference point and the second reference coordinate of the second reference point, screens the first image and the second image according to the deviation value, so as to determine whether the first image sequence meets the first preset condition and whether the second image sequence meets the second preset condition, and determines that the first image sequence or the second image sequence meets the registration condition under the condition of meeting the preset condition, thereby further reducing the error and improving the registration accuracy.

[0147] To execute the corresponding steps in the above embodiments and various possible ways, an implementation manner of an image registration device is given below. Further, please refer to Figure 10 , Figure 10 which is a functional module diagram of an image registration device provided by the embodiments of the present application. It should be noted that the basic principle and the technical effects generated by the image registration device provided in this embodiment are the same as those in the above embodiments. For the sake of brief description, for the parts not mentioned in this embodiment, reference can be made to the corresponding content in the above embodiments. The image registration device includes: an acquisition module 100, a registration module 110, and a registration module 120.

[0148] The acquisition module 100 is configured to acquire a first image sequence and a second image sequence; wherein, the first image sequence and the second image sequence are medical images of different modalities;

[0149] It can be understood that the acquisition module 100 can be used to execute the above-mentioned step S20.

[0150] The registration module 110 is used to register the first image sequence and the second image sequence to obtain a first transformation parameter between the positions of each pixel point in the first image sequence and the actual positions, and a second transformation parameter between the positions of each pixel point in the second image sequence and the actual positions.

[0151] It can be understood that the registration module 110 can be used to execute the above-mentioned step S21.

[0152] The registration matching module 120 is used to determine the registration relationship between the first image sequence and the second image sequence according to the first transformation parameter and the second transformation parameter.

[0153] It can be understood that the registration matching module 120 can be used to execute the above-mentioned step S22.

[0154] The image registration device provided by the embodiments of the present application acquires a first image sequence and a second image sequence through an acquisition module, where the first image sequence and the second image sequence are medical images of different modalities; a registration module registers the first image sequence and the second image sequence to obtain a first transformation parameter between the positions of each pixel point in the first image sequence and the actual positions and a second transformation parameter between the positions of each pixel point in the second image sequence and the actual positions; a registration matching module determines the registration relationship between the first image sequence and the second image sequence according to the first transformation parameter and the second transformation parameter, so that doctors can more conveniently, intuitively, and accurately observe the lesion structure from various angles during diagnosis, realize the accurate positioning and analysis of the lesion position, and is beneficial to the formulation of medical diagnosis and surgical and radiotherapy plans.

[0155] Optionally, the registration module 110 is further configured to acquire the coordinate of the feature point of the reference device; identify the marker points in the first image sequence and the marker points in the second image sequence, and determine the first coordinate of the marker points in the first image sequence and the second coordinate of the marker points in the second image sequence; determine the first transformation parameter between the positions of each pixel point in the first image sequence and the actual positions and the second transformation parameter between the positions of each pixel point in the second image sequence and the actual positions according to the first coordinate, the second coordinate, and the coordinate of the feature point.

[0156] It can be understood that the registration module can also be used to execute the above-mentioned steps S21-1 to S21-3.

[0157] Optionally, the registration module 110 is further configured to obtain unit vectors in each direction in the original coordinate system; fit a first straight-line equation based on multiple first coordinates; fit a second straight-line equation based on multiple second coordinates; determine a first transformation parameter according to the first straight-line equation, the characteristic point coordinates, and the unit vectors; and determine a second transformation parameter according to the second straight-line equation, the characteristic point coordinates, and the unit vectors.

[0158] Understandably, the registration module can also be configured to perform the above steps S21-3-1 to S21-3-5.

[0159] Optionally, the registration module 110 is further configured to determine the center point coordinates of the first image sequence and the normal vectors of the first image sequence in each direction according to the first straight-line equation; calculate a first translation parameter between the positions of each pixel point in the first image sequence and the actual positions according to the center point coordinates of the first image sequence and the characteristic point coordinates; and calculate a first rotation parameter between the positions of each pixel point in the first image sequence and the actual positions according to the normal vectors of the first image sequence in each direction and the unit vectors.

[0160] Understandably, the registration module can also be configured to perform the above steps S21-3-4-1 to S21-3-4-3.

[0161] Optionally, the registration module 110 is further configured to determine the center point coordinates of the first image sequence and the normal vector of the first image sequence in the Z direction according to multiple first straight-line equations; determine the oblique line equations in the multiple first straight-line equations, and calculate the average oblique line slope according to the oblique line equations; calculate the normal vector of the first image sequence in the X direction according to the normal vector of the first image sequence in the Z direction and the average oblique line slope; and calculate the normal vector of the first image sequence in the Y direction according to the normal vector of the first image sequence in the Z direction and the normal vector of the first image sequence in the X direction.

[0162] Understandably, the registration module can also be configured to perform the above steps S21-3-4-1-1 to S21-3-4-1-4. Optionally, the registration module 110 is further configured to determine the center point coordinates of the second image sequence and the normal vectors of the second image sequence in each direction according to the second straight-line equation; calculate a second translation parameter between the positions of each pixel point in the second image sequence and the actual positions according to the center point coordinates of the second image sequence and the characteristic point coordinates; and calculate a second rotation parameter between the positions of each pixel point in the second image sequence and the actual positions according to the normal vectors of the second image sequence in each direction and the unit vectors.

[0163] Understandably, the registration module can also be configured to perform the above steps S21-3-5-1 to S21-3-5-3.

[0164] Optionally, the registration module 120 is further configured to obtain a first reference coordinate of a first reference point and a second reference coordinate of a second reference point; the first reference point is a point generated on each first image after the reference device is mapped to the first image sequence, and the second reference point is a point generated on each second image after the reference device is mapped to the second image sequence; calculate a first deviation value of a marking point of each first image in the first image sequence relative to the first reference point, and a second deviation value of a marking point of each second image in the second image sequence relative to the second reference point; wherein, the first deviation value is obtained from the first coordinate and the first reference coordinate, and the second deviation value is obtained from the second coordinate and the second reference coordinate; determine that the first image sequence meets the registration condition when the first deviation value meets a first preset condition; determine that the second image sequence meets the registration condition when the second deviation value meets a second preset condition.

[0165] Understandably, the registration module 120 can also be configured to execute the above steps S23 to S26.

[0166] Please refer to Figure 11 , which is a block diagram of the terminal device 200. Understandably, the terminal device is the terminal device mentioned in the above method. Optionally, the terminal device can be a PC, a tablet, or other devices.

[0167] The terminal device 200 includes a memory 210, a processor 220, and a communication module 230. The elements of the memory 210, the processor 220, and the communication module 230 are directly or indirectly electrically connected to each other to achieve data transmission or interaction. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines.

[0168] Among them, the memory 210 is used to store programs or data. The memory can be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc.

[0169] The processor 220 is used to read / write data or programs stored in the memory and execute corresponding functions.

[0170] The communication module 230 is used to establish a communication connection between the server and other communication terminals through the network, and is used to send and receive data through the network.

[0171] It should be understood that Figure 11 The structure shown is only a schematic diagram of the terminal device, and the terminal device may also include more or fewer components than those shown Figure 11 shown, or have a different configuration from that Figure 11 shown. Figure 11 Each component shown can be implemented by hardware, software, or a combination thereof.

[0172] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0173] In addition, in each embodiment of the present application, the functional modules can be integrated together to form an independent part, or each module can exist alone, or two or more modules can be integrated to form an independent part.

[0174] When the above-mentioned functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs.

[0175] The foregoing are only the preferred embodiments of this application and are not intended to limit this application. For those skilled in the art, this application may have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of this application shall be included within the protection scope of this application.

Claims

1. An image registration method, characterized in that, The method includes: Obtaining a first image sequence and a second image sequence; wherein, the first image sequence and the second image sequence are medical images of different modalities; Registering the first image sequence and the second image sequence to obtain a first transformation parameter of the position of each pixel point in the first image sequence with respect to the actual position, and a second transformation parameter of the position of each pixel point in the second image sequence with respect to the actual position; The first transformation parameter includes a first translation parameter and a first rotation parameter. The first translation parameter is calculated based on the center point coordinates of the first image sequence and the feature point coordinates of the reference device. The first rotation parameter is calculated based on the normal vectors and unit vectors of the first image sequence in each direction, and the unit vectors are the unit vectors in each direction in the original coordinate system; The second transformation parameter includes a second translation parameter and a second rotation parameter. The second translation parameter is calculated based on the center point coordinates of the second image sequence and the feature point coordinates of the reference device. The second rotation parameter is calculated based on the normal vectors and unit vectors of the second image sequence in each direction, and the unit vectors are the unit vectors in each direction in the original coordinate system; Determining the registration relationship between the first image sequence and the second image sequence according to the first transformation parameter and the second transformation parameter.

2. The method according to claim 1, characterized in that The registering the first image sequence and the second image sequence to obtain a first transformation parameter of the position of each pixel point in the first image sequence with respect to the actual position, and a second transformation parameter of the position of each pixel point in the second image sequence with respect to the actual position includes: Obtaining the feature point coordinates of the reference device; Identifying the marker points in the first image sequence and the marker points in the second image sequence, and determining the first coordinates of the marker points in the first image sequence and the second coordinates of the marker points in the second image sequence; Determining a first transformation parameter of the position of each pixel point in the first image sequence with respect to the actual position, and a second transformation parameter of the position of each pixel point in the second image sequence with respect to the actual position according to the first coordinates, the second coordinates, and the feature point coordinates.

3. The method according to claim 2, wherein The determining a first transformation parameter of the position of each pixel point in the first image sequence with respect to the actual position, and a second transformation parameter of the position of each pixel point in the second image sequence with respect to the actual position according to the first coordinates, the second coordinates, and the feature point coordinates includes: Obtaining the unit vectors in each direction in the original coordinate system; Fitting a first straight line equation according to multiple first coordinates; Fitting a second straight line equation according to multiple second coordinates; Determining the first transformation parameter according to the first straight line equation, the feature point coordinates, and the unit vector; Determining the second transformation parameter according to the second straight line equation, the feature point coordinates, and the unit vector.

4. The method according to claim 3, characterized in that, The determining the first transformation parameter according to the first straight line equation, the feature point coordinates, and the unit vector includes: Determining the center point coordinates of the first image sequence and the normal vectors of the first image sequence in each direction according to the first straight line equation; Calculate a first translation parameter between the positions of each pixel point in the first image sequence and the actual positions according to the center point coordinates of the first image sequence and the feature point coordinates; Calculate a first rotation parameter between the positions of each pixel point in the first image sequence and the actual positions according to the normal vectors of the first image sequence in each direction and the unit vector.

5. The method according to claim 4, wherein The normal vectors of the first image sequence in each direction include the normal vectors in the X, Y, and Z directions. Determining the center point coordinates of the first image sequence and the normal vectors of the first image sequence in each direction according to the first straight line equation includes: Determine the center point coordinates of the first image sequence and the normal vector of the first image sequence in the Z direction according to a plurality of the first straight line equations; Determine the oblique line equations among a plurality of the first straight line equations, and calculate an average oblique line slope according to the oblique line equations; Calculate the normal vector of the first image sequence in the X direction according to the normal vector of the first image sequence in the Z direction and the average oblique line slope; Calculate the normal vector of the first image sequence in the Y direction according to the normal vector of the first image sequence in the Z direction and the normal vector of the first image sequence in the X direction.

6. The method according to claim 3, characterized in that Determining a second transformation parameter according to the second straight line equation, the feature point coordinates, and the unit vector includes: Determine the center point coordinates of the second image sequence and the normal vectors of the second image sequence in each direction according to the second straight line equation; Calculate a second translation parameter between the positions of each pixel point in the second image sequence and the actual positions according to the center point coordinates of the second image sequence and the feature point coordinates; Calculate a second rotation parameter between the positions of each pixel point in the second image sequence and the actual positions according to the normal vectors of the second image sequence in each direction and the unit vector.

7. The method according to claim 2, characterized in that Before determining the registration relationship between the first image sequence and the second image sequence according to the first transformation parameter and the second transformation parameter, the method further includes: Obtain a first reference coordinate of a first reference point and a second reference coordinate of a second reference point; the first reference point is a point generated on each of the first images after the reference device is mapped to the first image sequence, and the second reference point is a point generated on each of the second images after the reference device is mapped to the second image sequence; Calculate a first deviation value of the marker points of each first image in the first image sequence relative to the first reference point, and a second deviation value of the marker points of each second image in the second image sequence relative to the second reference point; wherein, the first deviation value is obtained from the first coordinate and the first reference coordinate, and the second deviation value is obtained from the second coordinate and the second reference coordinate; Determine that the first image sequence meets the registration condition when the first deviation value meets a first preset condition; Determine that the second image sequence meets the registration condition when the second deviation value meets a second preset condition.

8. An image registration device, characterized in that, The device includes: An acquisition module, configured to acquire a first image sequence and a second image sequence; wherein, the first image sequence and the second image sequence are medical images of different modalities; A registration module, configured to register the first image sequence and the second image sequence to obtain a first transformation parameter between the positions of each pixel point in the first image sequence and the actual positions, and a second transformation parameter between the positions of each pixel point in the second image sequence and the actual positions; The first transformation parameter includes a first translation parameter and a first rotation parameter. The first translation parameter is calculated according to the center point coordinates of the first image sequence and the feature point coordinates of a reference device. The first rotation parameter is calculated according to the normal vectors and unit vectors of the first image sequence in each direction. The unit vectors are the unit vectors of each direction in the original coordinate system; The second transformation parameter includes a second translation parameter and a second rotation parameter. The second translation parameter is calculated according to the center point coordinates of the second image sequence and the feature point coordinates of a reference device. The second rotation parameter is calculated according to the normal vectors and unit vectors of the second image sequence in each direction. The unit vectors are the unit vectors of each direction in the original coordinate system; A registration module, configured to determine the registration relationship between the first image sequence and the second image sequence according to the first transformation parameter and the second transformation parameter.

9. A terminal device, characterized in that, It includes a processor and a memory. The memory stores a computer program that can be executed by the processor. The processor can execute the computer program to implement the image registration method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the image registration method according to any one of claims 1-7.

Images