Image registration method, electronic device, and computer readable storage medium

Abstract

Disclosed in the present disclosure are an image registration method, an electronic device, and a computer readable storage medium. The method comprises: acquiring at least two images, wherein any image comprises photographing results of an object to be analyzed under different conditions; respectively performing feature extraction on the at least two images to obtain image features of any image; determining a target displacement vector field of said object on the basis of the extracted image features; and on the basis of the target displacement vector field of said object, performing image registration on the at least two images to obtain an image registration result. The present disclosure solves the technical problems in the related art of low accuracy and limited application scenarios of the image registration method.

Description

Image registration method, electronic device, and computer-readable storage medium

[0001] Cross-reference

[0002] This disclosure claims priority to a Chinese patent application filed with the Patent Office of China on June 26, 2023, with application number 2023107674466 and invention name “Image registration method, electronic device and computer-readable storage medium”, the entire contents of which are incorporated by reference into this disclosure. Technical Field

[0003] The present disclosure relates to the field of image registration, and in particular to an image registration method, an electronic device, and a computer-readable storage medium. Background Art

[0004] Current image registration methods generally use feature descriptors that provide modality and contrast invariance information. However, due to the limited information represented by the above feature descriptors, they face challenges in environments with large deformations or complex anatomical differences (for example, the abdomen between patients). Therefore, existing image registration methods have low accuracy in processing registration tasks and are difficult to apply to scenarios that handle complex registration tasks.

[0005] To address the above-mentioned problems, no effective solutions have been proposed so far.

[0006] Summary of the Invention

[0007] The embodiments of the present disclosure provide an image registration method, an electronic device, and a computer-readable storage medium to at least solve the technical problems of low accuracy and limited application scenarios of image registration methods in related technologies.

[0008] According to one aspect of an embodiment of the present disclosure, there is provided an image registration method, comprising: acquiring at least two images, wherein any one of the images contains shooting results of an object to be analyzed under different conditions; performing feature extraction on the at least two images respectively to obtain image features of any one of the images; determining a target displacement vector field of the object to be analyzed based on the extracted image features; and performing image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result.

[0009] According to one aspect of an embodiment of the present disclosure, an image registration method is also provided, including: acquiring at least two medical images, wherein any one of the medical images contains scanning results of a target part of the same biological object under different conditions using computed tomography technology; performing feature extraction on the at least two medical images respectively to obtain image features of any one of the images, wherein the image features include: global features and local features; determining a target displacement vector field of the target part based on the extracted image features; and performing image registration on the at least two medical images based on the target displacement vector field of the target part to obtain an image registration result.

[0010] According to one aspect of an embodiment of the present disclosure, an image registration method is also provided, including: responding to an input instruction on an operation interface, displaying at least two images on the operation interface, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; responding to the image registration instruction on the operation interface, displaying the image registration result on the operation interface, wherein the image registration result is obtained by performing image registration on the at least two images based on the target displacement vector field of the object to be analyzed, and the target displacement vector field is determined based on extracted image features, and the extracted image features include: global features and local features, and the extracted image features are obtained by performing feature extraction on at least two images respectively.

[0011] According to one aspect of an embodiment of the present disclosure, an image registration method is also provided, including: displaying at least two images on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device, wherein any one of the images contains shooting results of an object to be analyzed under different conditions; performing feature extraction on the at least two images respectively to obtain image features of any one of the images; determining a target displacement vector field of the object to be analyzed based on the extracted image features; performing image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result; and driving the VR device or AR device to render and display the image registration result.

[0012] According to one aspect of an embodiment of the present disclosure, an image registration method is also provided, including: acquiring at least two images by calling a first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter is at least two images, and any one of the images contains the shooting results of the object to be analyzed under different conditions; performing feature extraction on the at least two images respectively to obtain image features of any one of the images; determining the target displacement vector field of the object to be analyzed based on the extracted image features; performing image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result; outputting the image registration result by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the image registration result.

[0013] According to one aspect of an embodiment of the present disclosure, an electronic device is further provided, including: a memory configured to store an executable program; and a processor configured to run the program, wherein the program executes any one of the methods in the above embodiments when it is run.

[0014] According to one aspect of an embodiment of the present disclosure, a computer program product is further provided, including a computer program, which implements the method described in any of the above aspects when executed by a processor.

[0015] According to one aspect of an embodiment of the present disclosure, a computer program product is further provided, including a non-volatile computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method described in any of the above aspects is implemented.

[0016] According to one aspect of an embodiment of the present disclosure, a computer program is further provided. When the computer program is executed by a processor, the method as described in any one of the aforementioned aspects is implemented.

[0017] According to one aspect of an embodiment of the present disclosure, a computer-readable storage medium is further provided, which includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the methods in the above embodiments.

[0018] In an embodiment of the present disclosure, at least two images are acquired, wherein any one of the images includes the photographic results of the object to be analyzed under different conditions; feature extraction is performed on the at least two images respectively to obtain image features of any one of the images; based on the extracted image features, a target displacement vector field of the object to be analyzed is determined; based on the target displacement vector field of the object to be analyzed, at least two images are image registered to obtain an image registration result, thereby achieving the purpose of improving the accuracy of image registration. It is easy to notice that feature extraction can be performed on the at least two images respectively to obtain image features including global features and local features, the receptive field of the image can be expanded based on the global features and the local features to obtain a more accurate target displacement vector field, and the at least two images can be registered based on the target displacement vector field to obtain an image registration result with higher accuracy, which can be applied to scenarios for processing complex registration tasks, thereby solving the technical problems of low accuracy and limited application scenarios of image registration methods in related technologies.

[0019] It is easy to note that the above general description and the following detailed description are only for the purpose of exemplifying and explaining the present disclosure, and do not constitute a limitation of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The exemplary embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the drawings:

[0021] FIG1 is a schematic diagram of a hardware environment of a virtual reality device according to an image registration method according to an embodiment of the present disclosure;

[0022] FIG2 is a structural block diagram of a computing environment of an image registration method according to an embodiment of the present disclosure;

[0023] FIG3 is a flow chart of an image registration method according to Embodiment 1 of the present disclosure;

[0024] FIG4a is a structural diagram of an image registration process according to an embodiment of the present disclosure;

[0025] FIG4 b is a structural diagram of a convex optimization process according to an embodiment of the present disclosure;

[0026] FIG5 is a flowchart of an image configuration method according to Embodiment 2 of the present disclosure;

[0027] FIG6 is a flowchart of an image configuration method according to Embodiment 3 of the present disclosure;

[0028] FIG7 is a flow chart of an image registration method according to Embodiment 4 of the present disclosure;

[0029] FIG8 is a flow chart of an image registration method according to Embodiment 5 of the present disclosure;

[0030] FIG9 is a schematic diagram of an image registration device according to Embodiment 6 of the present disclosure;

[0031] FIG10 is a schematic diagram of an image registration device according to Embodiment 7 of the present disclosure;

[0032] FIG11 is a schematic diagram of an image registration device according to Embodiment 8 of the present disclosure;

[0033] FIG12 is a schematic diagram of an image registration device according to Embodiment 9 of the present disclosure;

[0034] FIG13 is a schematic diagram of an image registration device according to Embodiment 10 of the present disclosure;

[0035] FIG14 is a structural block diagram of a computer terminal according to an embodiment of the present disclosure;

[0036] FIG15 is a structural block diagram of a processor according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0037] In order to enable those skilled in the art to better understand the solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the embodiments described are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present disclosure.

[0038] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way are interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0039] First, some nouns or terms that appear in the description of the embodiments of the present disclosure are subject to the following explanations:

[0040] Registration: refers to the matching of geographic coordinates of different image graphics obtained by different imaging methods in the same area. Image registration is a typical problem and technical difficulty in the field of image processing research. Its purpose is to compare or fuse images of the same object acquired under different conditions. For example, images may come from different acquisition devices, taken at different times, different shooting angles, etc. Sometimes, image registration problems for different objects are also needed. Specifically, for two images in a set of image data sets, a spatial transformation is found to map one image to the other, so that the points corresponding to the same position in the space in the two images correspond one to one.

[0041] Cost volume (or correlation volume): It is a common intermediate representation in optical flow estimation or binocular matching, which is used to represent the pixel-level similarity between feature maps to find the corresponding relationship.

[0042] Convex optimization: also known as convex optimization or convex minimization, is a subfield of mathematical optimization that studies the problem of minimizing convex functions defined in convex sets. In a sense, convex optimization is simpler than general mathematical optimization problems. For example, in convex optimization, the local optimal value is also the global optimal value.

[0043] Deformable image registration is a fundamental task in medical image analysis, traditionally viewed as a continuous update problem involving a dense displacement field between image pairs. This problem results in inefficient iterative image registration. To address this issue, a method has been proposed that uses a learning-based deep network to predict the displacement field and iterate to improve its efficiency. However, since different types of registration tasks require training, this image registration method is difficult to adapt to diverse application scenarios.

[0044] In addition, collecting sufficient training data takes a lot of time. Both iterative updates and learning-based methods rely on similarity metrics calculated on intensity, which are difficult to apply to anatomical correspondences. Some studies use feature descriptors that provide modality and contrast invariant information for processing, but generally can only represent local information without containing global semantic information. Therefore, image registration methods face challenges in environments with large deformations or complex anatomical differences (e.g., abdomens between patients).

[0045] Registration has also been formulated as a discrete update problem, which can be handled using a dense set of discrete displacements as a cost volume. The main challenge of this type of approach is the huge size of the search space, as there are millions of voxels in a typical 3D computed tomography (CT) scan, and each voxel in a moving scan can be reasonably paired with thousands of points in other scans, resulting in a high computational burden. To achieve fast registration via discrete updates, the search space can be pruned by constructing a cost volume within the neighborhood of different voxels. However, the range of deformation amplitudes that can be resolved by this approach is limited by the size of the neighborhood window and relies on accurate pre-alignment, making the implementation process more difficult.

[0046] Widely used deformable registration methods with relatively good performance may include: Deeds, ConvexAdam, among which Deeds and ConvexAdam use feature descriptors that provide modality and contrast invariant information, but still represent local information without containing global semantic information. Therefore, it is still difficult to apply in environments with large deformations or complex anatomical differences (for example, the abdomen between patients); in addition, ConvexAdam relies on precise pre-alignment, which is more difficult to implement.

[0047] Handling complex registration tasks relies on unique features that are robust to inter-subject variations, organ deformation, contrast agent injection and pathological changes, as well as global / contextual information that is beneficial to improving the accuracy of complex deformation registration. To achieve these goals, the present disclosure adopts features embedded in a self-supervised anatomy-based manner to encode global and local embeddings; in the process of calculating cost loss within a neighborhood, it will be limited by the size of the neighborhood. To solve this problem, the present disclosure proposes to implement the image processing process from small-resolution images to large-resolution images through a pyramid model. Different levels have a smaller search range, rather than a large search range at one resolution. This is conducive to improving efficiency such as low computational burden and fast running time, expanding the receptive field, and improving registration accuracy.

[0048] Example 1

[0049] According to an embodiment of the present disclosure, an image registration method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0050] Figure 1 is a schematic diagram of the hardware environment of a virtual reality device for an image registration method according to an embodiment of the present disclosure. As shown in Figure 1, virtual reality device 104 is connected to terminal 106, which is connected to server 102 via a network. Virtual reality device 104 is not limited to virtual reality helmets, virtual reality glasses, or all-in-one virtual reality devices. Terminal 104 is not limited to PCs, mobile phones, or tablet computers. Server 102 can be a server corresponding to a media file operator. The network includes, but is not limited to, a wide area network, a metropolitan area network, or a local area network.

[0051] Optionally, the virtual reality device 104 of this embodiment includes: a memory, a processor, and a transmission device. The memory is used to store an application program, which can be used to execute the following: acquiring at least two images, wherein any one of the images contains photographic results of an object to be analyzed under different conditions; performing feature extraction on each of the at least two images to obtain image features of any one of the images; determining a target displacement vector field of the object to be analyzed based on the extracted image features; and performing image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result, thereby resolving the technical issues of low accuracy and limited application scenarios of image registration methods in related technologies.

[0052] The terminal of this embodiment can be used to display the image registration result on the presentation screen of a virtual reality (VR) device or an augmented reality (AR) device, and send the image registration result to the virtual reality device 104. After receiving the image registration result, the virtual reality device 104 displays it at the target delivery location.

[0053] Optionally, the virtual reality device 104 of this embodiment includes an eye-tracking head-mounted display (HMD) and an eye-tracking module that function similarly to those in the aforementioned embodiment. Specifically, the screen in the HMD displays real-time images, and the eye-tracking module in the HMD captures the real-time trajectory of the user's eye movements. The terminal of this embodiment uses a tracking system to obtain the user's position and motion information in real three-dimensional space, and calculates the three-dimensional coordinates of the user's head in virtual three-dimensional space, as well as the user's field of view in the virtual three-dimensional space.

[0054] The hardware block diagram shown in FIG1 can serve not only as an exemplary block diagram of the aforementioned AR / VR device (or mobile device), but also as an exemplary block diagram of the aforementioned server. In an optional embodiment, FIG2 shows, as a block diagram, an embodiment using the AR / VR device (or mobile device) shown in FIG1 as a computing node in a computing environment 201. FIG2 is a block diagram of the computing environment for an image registration method according to an embodiment of the present disclosure. As shown in FIG2, computing environment 201 includes multiple computing nodes (e.g., servers) (illustrated as 210-1, 210-2, ...) running on a distributed network. Different computing nodes include local processing and memory resources, and end users 202 can remotely run applications or store data in computing environment 201. Applications can be provided as multiple services 220-1, 220-2, 220-3, and 220-4 in computing environment 201, representing services "A," "D," "E," and "H," respectively.

[0055] End user 202 can provide and access services through a web browser or other software application on a client. In some embodiments, the provisioning and / or request of end user 202 can be provided to the ingress gateway 230. The ingress gateway 230 may include a corresponding agent to handle the provisioning and / or request for services (one or more services provided in the computing environment 201).

[0056] Services are provided or deployed based on various virtualization technologies supported by the computing environment 201. In some embodiments, services can be provided based on virtual machine (VM)-based virtualization, container-based virtualization, and / or similar methods. Virtual machine-based virtualization can simulate a real computer by initializing a virtual machine, executing programs and applications without directly contacting any actual hardware resources. While the virtual machine virtualizes the machine, according to container-based virtualization, a container can be started to virtualize the entire operating system (OS) so that multiple workloads can run on a single operating system instance.

[0057] In one embodiment based on container virtualization, several containers of a service can be assembled into a Pod (e.g., a Kubernetes Pod). For example, as shown in Figure 2, service 220-2 can be equipped with one or more Pods 240-1, 240-2, ..., 240-N (collectively referred to as Pods). The Pod may include a proxy 245 and one or more containers 242-1, 242-2, ..., 242-M (collectively referred to as containers). One or more containers in the Pod process requests related to one or more corresponding functions of the service, and the proxy 245 generally controls network functions related to the service, such as routing, load balancing, etc. Other services can also be equipped with similar Pods.

[0058] During operation, executing a user request from end user 202 may require invoking one or more services in computing environment 201. Executing one or more functions of one service may require invoking one or more functions of another service. As shown in FIG2 , service "A" 220-1 receives a user request from end user 202 from ingress gateway 230. Service "A" 220-1 may invoke service "D" 220-2, and service "D" 220-2 may request service "E" 220-3 to execute one or more functions.

[0059] This computing environment can be a cloud computing environment, where resource allocation is managed by the cloud service provider, allowing for feature development without having to worry about implementing, adjusting, or scaling servers. This computing environment allows developers to execute code in response to events without building or maintaining complex infrastructure. Services can be partitioned to perform a set of functions that can scale independently and automatically, rather than scaling a single hardware device to handle the potential load.

[0060] In the above operating environment, the present disclosure provides an image registration method as shown in FIG3. It should be noted that the image registration method of this embodiment can be executed by the mobile terminal of the embodiment shown in FIG1. ​​FIG3 is a flow chart of the image registration method according to embodiment 1 of the present disclosure. As shown in FIG3, the method may include the following steps:

[0061] Step S302: Acquire at least two images.

[0062] Among them, any image contains the shooting results of the object to be analyzed under different conditions.

[0063] The at least two images mentioned above may be CT images belonging to the medical field, or images from other fields. There is no limitation on the type of images.

[0064] The objects to be analyzed contained in the aforementioned images may be portions of the two images to be compared. In the medical field, the at least two images may be abdominal CT images of a patient, and the objects to be analyzed may be abnormal areas of the abdomen, such as lesions in the patient's abdomen. It should be noted that the objects to be analyzed are not limited in this context and can be determined based on the image analysis requirements. This is merely an example.

[0065] The above-mentioned different conditions may be different time periods, different environments, different types of shooting equipment, and different shooting angles. The different conditions here can be set according to actual scenarios.

[0066] In an optional embodiment, the object to be analyzed or the area where the object to be analyzed is located can be photographed at different time periods to obtain at least two images; the object to be analyzed or the area where the object to be analyzed is located can also be photographed using different shooting methods to obtain at least two images; the object to be analyzed or the area where the object to be analyzed is located can also be photographed under other different conditions to obtain at least two images.

[0067] It should be noted that, in the embodiment of the present disclosure, the image registration is performed by obtaining two images as an example, but the invention is not limited thereto. The processing method for obtaining other numbers of images is similar and will not be elaborated here.

[0068] Step S304: performing feature extraction on at least two images respectively to obtain image features of any one of the images.

[0069] The image features include at least global features and local features.

[0070] The image feature of any one of the above images may be the image feature of each of the at least two images.

[0071] In an optional embodiment, feature extraction can be performed on at least two images by a feature extractor to obtain image features of each image. The feature extractor can be a self-supervised anatomical embedding feature extractor (Self-supervised Autoencoder with Mosaicked Training, abbreviated as SAM), but is not limited to this. It can be other types of feature extractors.

[0072] It should be noted that SAM is usually used to extract features from images. The algorithm uses a self-supervised learning method to train an anatomical embedding model through self-learning. The model can learn the global semantic features and local semantic features of different points on the image, so it can obtain the image features of any image, and the image features include at least global features and local features.

[0073] Step S306: determining the target displacement vector field of the object to be analyzed based on the extracted image features.

[0074] The above-mentioned target displacement vector field can represent the spatial change of the object to be analyzed in at least two images. The displacement vector field can be estimated by the position of the object to be analyzed. The vectors in the target displacement vector field are used to represent the displacement direction and size of the object to be analyzed in the image.

[0075] In geometry and mechanics, a displacement is a vector whose length is the shortest distance from an initial position to the final position of a point undergoing motion. It quantifies the distance and direction of the net or total motion of a point's trajectory along a straight line from its initial position to its final position. A displacement vector field can be described as a relative position, that is, as the final position of a point relative to its initial position, and can be defined as the difference between the final and initial positions.

[0076] In an optional embodiment, a cost volume can be constructed based on the extracted image features, and the target displacement vector field can be determined based on the cost volume. The cost volume is a common intermediate representation in optical flow estimation or binocular matching, which can be used to represent the pixel-level similarity between features to find the correspondence between pixels. The target displacement vector field of the object to be analyzed can be determined based on the correspondence.

[0077] The above-mentioned cost volume may be a 6D cost volume, but is not limited thereto, wherein the 6D cost volume stores data matching costs for associating pixels with corresponding pixels in another image.

[0078] It should be noted that the cost volume (or correlation volume) is a common intermediate representation in optical flow estimation or binocular matching. It is used to represent the pixel-level similarity between feature maps, so that the corresponding relationship between pixels can be found based on this similarity. Generally, a dense cost volume is calculated. In optical flow problems, its complexity is the square of the number of pixels (H×W×H×W); in 3D medical image registration problems, its complexity is the square of the number of pixels (H×W×D×H×W×D), where the size of the 3D image is H×W×D.

[0079] Constructing a 6D cost volume based on SAM features is a method that uses features from two images to construct a 6D cost volume, which is used to measure the similarity between two 3D objects. By extracting features from the image using SAM, the corresponding 6D cost volume can be obtained.

[0080] Optionally, a convex optimization process may be performed based on the cost volume to obtain the target displacement vector field, wherein the convex optimization process refers to a type of optimization problem in which the objective function is a convex function.

[0081] Step S308 : performing image registration on at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result.

[0082] The above-mentioned image registration refers to matching the geographic coordinates of different images acquired by different imaging methods in the same area. Optionally, image registration is to map one image to another image by finding a spatial transformation so that points at the same spatial position in the two images correspond one to one.

[0083] In an optional embodiment, any one of the at least two images can be mapped to the other image based on the target displacement vector field of the object to be analyzed, so that the changes of the object to be analyzed can be observed in one image, thereby facilitating the user to analyze the object to be analyzed and improving the efficiency of the analysis.

[0084] In the medical field, the object to be analyzed can be the patient's abdominal area. The target displacement vector field of the abdominal area can be determined based on at least two CT scans of the patient's abdominal area. The two CT scans can be aligned based on the target displacement vector field to obtain an image alignment result, which is convenient for doctors to analyze problems in the patient's abdominal area based on the image alignment results.

[0085] Through the above steps, at least two images are obtained, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; feature extraction is performed on the at least two images respectively to obtain image features of any one of the images; based on the extracted image features, the target displacement vector field of the object to be analyzed is determined; based on the target displacement vector field of the object to be analyzed, the at least two images are image-aligned to obtain image alignment results, thereby achieving the purpose of improving the accuracy of image alignment. It is easy to notice that feature extraction can be performed on the at least two images respectively to obtain image features containing global features and local features, the receptive field of the image can be expanded based on the global features and the local features to obtain a more accurate target displacement vector field, and the at least two images can be aligned based on the target displacement vector field to obtain an image alignment result with higher accuracy, which can be applied to scenarios for processing complex alignment tasks, thereby solving the technical problems of low accuracy and limited application scenarios of image alignment methods in related technologies.

[0086] In the above-mentioned embodiment of the present disclosure, feature extraction is performed on at least two images respectively to obtain image features of any one of the images, including: deforming the first image based on the displacement vector field corresponding to the first image of the at least two images to obtain a deformed image; and feature extraction is performed on the deformed image and the second image of the at least two images respectively to obtain image features of any one of the images, wherein the second image is used to represent the image other than the first image of the at least two images.

[0087] The above-mentioned first image and second image can be images of the same area captured at different time points. The above-mentioned first image can be the image captured first, and the second image can be the image captured later. The above-mentioned second image can be the image captured first, and the above-mentioned first image can be the image captured later. There is no limitation on the first image and the second image here.

[0088] The first image may also be any one of the at least two images, and the second image may be any other image of the at least two images except the one. Optionally, when the at least two images only include two images, the first image may be the source image of the two images, and the second image may be the target image.

[0089] The displacement vector field mentioned above can also be called an up-sampled field.

[0090] The displacement vector field corresponding to the above-mentioned first image can be determined based on the scale information of the first image. The scale information of the first image can be divided into multiple levels. The higher the level, the higher the resolution of the image, and the lower the level, the smaller the resolution of the image. Therefore, if the scale of the first image is the smallest, that is, the lowest level, the displacement vector field can be a preset vector field. If the scale of the first image is not the smallest scale, that is, not the lowest level, the displacement vector field can be a target displacement vector field determined by the image of the previous level.

[0091] In an optional embodiment, the above content can be described according to three levels, the first level is the level with the smallest image scale, the second level is the level with the second largest image scale, and the third level is the level with the largest image scale. When the first image is at the first level, it means that the resolution of the first image is the lowest. At this time, the first displacement vector field between the deformed image and the second image of the first level can be directly obtained; when the first image is at the second level, the first image can be deformed based on the displacement vector field obtained after upsampling the first displacement vector field to obtain a deformed image, and the second displacement vector field between the deformed image and the second image of the second level is obtained; when the first image is at the third level, it means that the resolution of the first image is the largest. At this time, the first image can be deformed based on the displacement vector field obtained after upsampling the second displacement vector field to obtain a deformed image, and the third displacement vector field between the deformed image and the second image of the largest level is obtained.

[0092] In another optional embodiment, feature extraction can be performed on the deformed image and the second image obtained at any level to obtain the image features of any image. Feature extraction can also be performed on the deformed image and the second image obtained at the above three levels to obtain the image features of any image.

[0093] In the above-mentioned embodiment of the present disclosure, feature extraction is performed on the deformed image and the second image of at least two images respectively to obtain image features of any one of the images, including: feature extraction on the deformed image and the second image to obtain global features and local features; and global features and local features are superimposed to obtain image features.

[0094] In an optional embodiment, the global features and local features in the deformed image and the second image can be captured simultaneously by a feature extractor, and the global features and local features can be further superimposed to obtain the image features of any image.

[0095] In the above embodiment of the present disclosure, based on the extracted image features, the target displacement vector field of the object to be analyzed is determined, including: obtaining the dot product of the extracted image features to obtain the target cost volume; and performing convex optimization processing on the target cost volume to obtain the target displacement vector field.

[0096] The target cost volume mentioned above is used to find the correspondence between image features based on the similarity of image features.

[0097] In an optional embodiment, the dot product of the extracted image features can be obtained to measure the similarity between the first image and the second image, and a target cost volume can be constructed based on the similarity between the two images. Convex optimization can be performed on the target cost volume to obtain a target displacement vector field. The target cost volume can be a 6D cost volume.

[0098] In the above embodiment of the present disclosure, a target cost volume is subjected to convex optimization processing to obtain a target displacement vector field, including: performing parameter minimization processing on the target cost volume to obtain an initial displacement vector field; and performing an average pooling operation on the initial displacement vector field to obtain a target displacement vector field.

[0099] In an optional embodiment, parameter minimization processing can be performed on the target cost volume, that is, parameter minimization processing can be performed on the target cost volume using Argmin(.) to obtain a variable corresponding to the target cost volume when the output value of Argmin(.) is minimum. The initial displacement vector field is determined based on the variable, and the initial displacement vector field can be averaged and pooled so that the initial displacement vector field can be divided into several regions of the same size. The eigenvalues ​​in different regions are then averaged to obtain the target displacement vector field.

[0100] In the above-mentioned embodiment of the present disclosure, feature extraction is performed on at least two images respectively to obtain image features of any one image, including: downsampling the at least two images multiple times respectively to obtain multiple downsampled images of any one image, wherein the resolutions of different downsampled images are different; deforming the downsampled image of the first image based on the sampling displacement vector field corresponding to the downsampled image of the first image of the at least two images to obtain a sampled deformed image; feature extraction is performed on the sampled deformed image and the downsampled image of the second image of the at least two images respectively to obtain downsampled image features of the downsampled image of any one image; and the features of the multiple downsampled images are aggregated to obtain the image features of any one image.

[0101] In an optional embodiment, at least two images can be downsampled multiple times to obtain representations of the at least two images at different scales, that is, multiple downsampled images of any image can be obtained, and the resolutions of different downsampled images are different. The downsampled image of the first image can be deformed according to the sampling displacement vector field corresponding to the downsampled image of the first image to obtain a sampled deformed image. Sampled deformed images of different resolutions can be obtained. Feature extraction can be performed on the sampled deformed image and the downsampled image of the second image to obtain downsampled image features of the downsampled image of any image. The features of multiple downsampled images with different resolutions can be summarized to obtain an image feature pyramid of any image.

[0102] In another optional embodiment, the largest-scale downsampled image can be processed to obtain the largest-scale sampled deformed image. Feature extraction can be performed on the sampled deformed image and the downsampled image of the second image respectively to obtain the downsampled image features of the downsampled image of any image. The sampling displacement vector field required for the next-scale image processing process can be constructed based on the downsampled image features. Optionally, a cost volume can be constructed based on the downsampled image features and convex optimization processing can be performed to obtain the sampling displacement vector field required for the next-scale image processing process. The sampling displacement vector fields corresponding to images of each scale are obtained cyclically, thereby obtaining multiple sampled deformed images in sequence based on the multiple sampling displacement vector fields, and obtaining multiple downsampled image features in sequence based on the multiple sampled deformed images. The multiple downsampled image features can be summarized to obtain the image features of any image.

[0103] The first image is deformed at different scales in a coarse-to-fine manner to obtain a sampled deformed image, and features are extracted from downsampled images of the sampled deformed image and the second image to obtain features of the downsampled image.

[0104] In the above embodiment of the present disclosure, based on the extracted image features, the target displacement vector field of the object to be analyzed is determined, including: obtaining the dot product of the downsampled image features of the downsampled image of any image to obtain a sampling cost volume; performing convex optimization processing based on the sampling cost volume to obtain a sampling displacement vector field; and superimposing multiple sampling displacement vector fields to obtain a target displacement vector field.

[0105] In an optional embodiment, the dot product of the downsampled image features of any image can be obtained to measure the similarity between the downsampled image of the first image and the downsampled image of the second image, and a sampling cost volume can be constructed based on the similarity between the two downsampled images. Convex optimization processing can be performed based on the sampling cost volume to obtain a sampling displacement vector field.

[0106] The above operation can be performed cyclically on downsampled images of different scales. It should be noted that the sampling displacement vector field obtained at the current scale can be deformed according to the sampling displacement vector field obtained at the previous scale to improve the accuracy of the deformation. After obtaining multiple sampling displacement vector fields at different scales, the multiple sampling displacement vector fields can be superimposed to obtain the target displacement vector field.

[0107] In the above embodiment of the present disclosure, the method further includes: when the downsampled image of the first image is a preset downsampled image, determining that the sampling displacement vector field corresponding to the downsampled image of the first image is the preset displacement vector field, wherein the preset downsampled image is used to represent the maximum-scale downsampled image of the first image; when the downsampled image of the first image is not the preset downsampled image, upsampling the sampling displacement vector field corresponding to the downsampled image of the next scale of the first image to obtain the sampling displacement vector field corresponding to the downsampled image of the first image.

[0108] The above-mentioned preset downsampled image can be a downsampled image at the maximum scale. Since the resolution of the downsampled image at the maximum scale is low, the number of features that need to be processed is small. Therefore, the global features of the downsampled image can be efficiently extracted, and the downsampled images can be processed in sequence from large resolution to small resolution. While ensuring that global features and local features can be extracted, the feature extraction efficiency can be further improved.

[0109] In an optional embodiment, when the downsampled image of the first image is the downsampled image of the maximum scale, since it is the first time to deform, the displacement vector field has not yet been obtained. At this time, the pre-set preset displacement vector field can be determined as the sampling displacement vector field corresponding to the downsampled image of the first image; after obtaining the downsampled image features based on the downsampled image of the maximum scale, the dot product of the downsampled image features can be obtained to obtain the sampling cost volume, and convex optimization processing can be performed based on the sampling cost volume to obtain the sampling displacement vector field required for the deformation of the downsampled image of the next scale. Based on this, downsampled images of different scales can be processed in sequence to obtain sampling displacement vector fields obtained at different scales.

[0110] The following describes a preferred embodiment of the present disclosure in detail, taking the scenario of two-image registration as an example. Figure 4a is a structural diagram of an image registration process according to an embodiment of the present disclosure. As shown in Figure 4a, the images to be registered include a first image IS and a second image IT. The first image and the second image can be downsampled to obtain an image pyramid (Pyramid Source) for the first image and an image pyramid (Pyramid Target) for the second image. The top image of the two sampling pyramids is the downsampled image with the largest scale, and the bottom image of the sampling pyramid is the downsampled image with the smallest scale. The processing flow for downsampled images at each scale is similar. Here, the processing flow for the downsampled image at the smallest scale is used as an example for explanation: when performing feature processing on the downsampled image, the registration field obtained at the previous scale can be upsampled (Up-Sampled Field), that is, the above-mentioned sampling displacement vector field is used to perform a warping operation on the downsampled image corresponding to the first image to obtain a sampled deformed image. Feature extraction can be performed on the sampled deformed image and the downsampled image of the second image to obtain downsampled image features. Convex optimization can be performed on the downsampled image features to obtain a sampling displacement vector field, that is, to obtain the registration field at the current scale. Finally, multiple sampling displacement vector fields can be superimposed to obtain a target displacement vector field. The first image and the second image can be registered based on the target displacement vector field to obtain an image registration result. It should be noted that for the processing flow for the downsampled image at the smallest scale, the downsampled image corresponding to the first image can be warped (Warping) based on a pre-set registration field to obtain a sampled deformed image.

[0111] Figure 4b is a structural diagram of a convex optimization processing process according to an embodiment of the present disclosure. As shown in Figure 4b, the local features and global features of the first image can be superimposed to obtain image features of the first image and the second image. The image features can be subjected to a dot product operation (Dot Product) to obtain a target cost volume. The preset function [Argmin(.)] can be used to determine the value of the minimum independent variable of the function of the target cost volume to obtain an initial displacement vector field, that is, the first registration field u shown in the figure. The initial displacement vector field can be subjected to an average pooling operation (Mean Conv) to obtain a target displacement vector field, that is, the second registration field ψ shown in the figure.

[0112] The image registration method disclosed in the present invention provides a self-supervised anatomical embedding network (SAMConvex), which can be quickly used for a discrete optimization method for image registration. The method includes a decoupled convex optimization program, which is used to obtain a displacement vector field based on a self-supervised anatomical embedding feature extractor. The feature extractor can capture global features and local features at the same time. Optionally, SAMConvex can extract features of different scales and construct a 6D cost volume based on the features. The displacement vector field can be iteratively updated by using a processing process from large scale to small scale.

[0113] It should be noted that the 6D cost volumes of different levels are calculated by taking inputs of different resolutions, and the cost volume of each level is calculated by taking inputs of different resolutions.

[0114] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0115] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the order of the actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the present disclosure are all preferred embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0116] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present disclosure.

[0117] Example 2

[0118] According to an embodiment of the present disclosure, an image registration method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0119] FIG5 is a flow chart of an image configuration method according to Embodiment 2 of the present disclosure. As shown in FIG5 , the method includes the following steps:

[0120] Step S502: Acquire at least two medical images.

[0121] Among them, any medical image contains the scanning results of the target part of the same biological object under different conditions using computer tomography technology.

[0122] The above-mentioned medical images include but are not limited to CT images and patient skin images, which are not limited here and the type of medical image can be set according to actual conditions.

[0123] Step S504 : performing feature extraction on at least two medical images respectively to obtain image features of any one of the images.

[0124] Among them, image features include: global features and local features.

[0125] Step S506: determining a target displacement vector field of the target part based on the extracted image features.

[0126] Step S508 : performing image registration on at least two medical images based on the target displacement vector field of the target part to obtain an image registration result.

[0127] Through the above steps, at least two medical images are obtained, wherein any one of the medical images includes scan results of a target part of the same biological object under different conditions using computed tomography technology; feature extraction is performed on the at least two medical images to obtain image features of any one of the images, wherein the image features include: global features and local features; based on the extracted image features, a target displacement vector field of the target part is determined; based on the target displacement vector field of the target part, the at least two medical images are image registered to obtain an image registration result, thereby achieving the purpose of improving the accuracy of image registration. It is easy to note that feature extraction can be performed on the at least two images to obtain image features including global features and local features, the receptive field of the image can be expanded based on the global features and the local features to obtain a more accurate target displacement vector field, and the at least two images can be registered based on the target displacement vector field to obtain an image registration result with higher accuracy. This can be applied to scenarios where complex registration tasks are processed, thereby solving the technical problems of low accuracy and limited application scenarios of image registration methods in related technologies.

[0128] It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the solution provided in Example 1, as well as the application scenario and implementation process, but is not limited to the solution provided in Example 1.

[0129] Example 3

[0130] According to an embodiment of the present disclosure, an image registration method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0131] FIG6 is a flow chart of an image configuration method according to Embodiment 3 of the present disclosure. As shown in FIG6 , the method includes the following steps:

[0132] Step S602 : Displaying at least two images on the operation interface in response to an input instruction on the operation interface.

[0133] Among them, any image contains the shooting results of the object to be analyzed under different conditions.

[0134] The above-mentioned operation interface may be a display interface capable of displaying at least two images, and at least two images may be displayed by performing relevant touch operations on the display interface.

[0135] Step S604 , responding to the image registration instruction applied to the operation interface, and displaying the image registration result on the operation interface.

[0136] Among them, the image registration result is obtained by performing image registration on at least two images based on the target displacement vector field of the object to be analyzed. The target displacement vector field is determined based on the extracted image features. The extracted image features include: global features and local features. The extracted image features are obtained by performing feature extraction on at least two images respectively.

[0137] The above-mentioned image registration instruction may be an instruction generated by touching a related control on the operation interface when at least two images need to be registered. The image registration result may be displayed on the operation interface according to the image registration instruction.

[0138] Through the above steps, in response to an input instruction on the operation interface, at least two images are displayed on the operation interface, wherein any one of the images contains the results of shooting the object to be analyzed under different conditions; in response to an image registration instruction on the operation interface, an image registration result is displayed on the operation interface, wherein the image registration result is obtained by performing image registration on the at least two images based on the target displacement vector field of the object to be analyzed, the target displacement vector field is determined based on extracted image features, and the extracted image features include: global features and local features. The extracted image features are obtained by performing feature extraction on the at least two images separately, thereby achieving the purpose of improving the accuracy of image registration. It is easy to notice that feature extraction can be performed on the at least two images separately to obtain image features containing global features and local features, the receptive field of the image can be expanded based on the global features and the local features to obtain a more accurate target displacement vector field, and the at least two images can be registered based on the target displacement vector field to obtain an image registration result with higher accuracy, which can be applied to scenarios for processing complex registration tasks, thereby solving the technical problems of low accuracy and limited application scenarios of image registration methods in related technologies.

[0139] It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the solution provided in Example 1, as well as the application scenario and implementation process, but is not limited to the solution provided in Example 1.

[0140] Example 4

[0141] According to an embodiment of the present disclosure, an image registration method that can be applied to virtual reality scenarios such as virtual reality VR devices and augmented reality AR devices is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0142] FIG7 is a flow chart of an image registration method according to Embodiment 4 of the present disclosure. As shown in FIG7 , the method may include the following steps:

[0143] Step S702: Display at least two images on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device.

[0144] Among them, any image contains the shooting results of the object to be analyzed under different conditions.

[0145] Step S704: performing feature extraction on at least two images respectively to obtain image features of any one of the images.

[0146] The image features include at least global features and local features.

[0147] Step S706: determining the target displacement vector field of the object to be analyzed based on the extracted image features.

[0148] Step S708 : performing image registration on at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result.

[0149] Step S710: driving the VR device or AR device to render and display the image registration result.

[0150] Through the above steps, at least two images are displayed on the presentation screen of a virtual reality (VR) device or an augmented reality (AR) device, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; feature extraction is performed on the at least two images respectively to obtain image features of any one of the images; based on the extracted image features, the target displacement vector field of the object to be analyzed is determined; based on the target displacement vector field of the object to be analyzed, the at least two images are image-aligned to obtain an image alignment result; the VR device or AR device is driven to render and display the image alignment result, thereby achieving the purpose of improving the accuracy of image alignment. It is easy to notice that feature extraction can be performed on the at least two images respectively to obtain image features containing global features and local features, the receptive field of the image can be expanded based on the global features and the local features to obtain a more accurate target displacement vector field, and the at least two images can be aligned based on the target displacement vector field to obtain an image alignment result with higher accuracy, which can be applied to scenarios for processing complex alignment tasks, thereby solving the technical problems of low accuracy and limited application scenarios of image alignment methods in related technologies.

[0151] Optionally, in this embodiment, the image registration method can be applied to a hardware environment consisting of a server and a virtual reality device. The image registration result is displayed on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device. The server can be a server corresponding to a media file operator. The network includes, but is not limited to, a wide area network, a metropolitan area network, or a local area network. The virtual reality device is not limited to a virtual reality helmet, virtual reality glasses, or an all-in-one virtual reality device.

[0152] Optionally, the virtual reality device includes: a memory, a processor, and a transmission device. The memory is used to store an application, which can be used to execute the following: displaying at least two images on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device, wherein any one of the images contains the results of photographing an object to be analyzed under different conditions; performing feature extraction on the at least two images to obtain image features of any one of the images, wherein the image features include at least global features and local features; determining a target displacement vector field of the object to be analyzed based on the extracted image features; performing image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result; and driving the VR device or AR device to render and display the image registration result.

[0153] It should be noted that the above-mentioned image registration method applied in the VR device or AR device of this embodiment may include the method of the embodiment shown in Figure 7, so as to achieve the purpose of driving the VR device or AR device and displaying the image registration results.

[0154] Optionally, the processor of this embodiment can call the application stored in the memory to perform the above steps through a transmission device. The transmission device can receive media files sent by the server through the network, and can also be used for data transmission between the processor and the memory.

[0155] It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the solution provided in Example 1, as well as the application scenario and implementation process, but is not limited to the solution provided in Example 1.

[0156] Example 5

[0157] According to an embodiment of the present disclosure, a target tracking method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0158] FIG8 is a flow chart of an image registration method according to Embodiment 5 of the present disclosure. As shown in FIG8 , the method includes the following steps:

[0159] Step S802: Acquire at least two images by calling the first interface.

[0160] The first interface includes a first parameter, the parameter value of the first parameter is at least two images, and any one of the images contains the shooting results of the object to be analyzed under different conditions.

[0161] The first interface in the above steps can be an interface for data interaction between the cloud server and the client. The client can pass at least two images into the interface function as the first parameter of the interface function to achieve the purpose of uploading at least two images to the cloud server.

[0162] Step S804: performing feature extraction on at least two images respectively to obtain image features of any one of the images.

[0163] The image features include at least global features and local features.

[0164] Step S806: determining the target displacement vector field of the object to be analyzed based on the extracted image features.

[0165] Step S808 : performing image registration on at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result.

[0166] Step S810: Output the image registration result by calling the second interface.

[0167] The second interface includes a second parameter, and the parameter value of the second parameter is the image registration result.

[0168] The above-mentioned second interface can be an interface for data interaction between the cloud server and the client. The cloud server can pass the image registration result into the interface function as the second parameter of the interface function to achieve the purpose of sending the image registration result to the client.

[0169] Through the above steps, at least two images are acquired by calling a first interface, wherein the first interface includes a first parameter, the parameter value of the first parameter is at least two images, and any one of the images contains the shooting results of the object to be analyzed under different conditions; feature extraction is performed on the at least two images respectively to obtain image features of any one of the images; based on the extracted image features, a target displacement vector field of the object to be analyzed is determined; based on the target displacement vector field of the object to be analyzed, at least two images are image registered to obtain an image registration result; and the image registration result is output by calling a second interface, wherein the second interface includes a second parameter, the parameter value of the second parameter is the image registration result, thereby achieving the purpose of improving the accuracy of image registration. It is easy to notice that feature extraction can be performed on at least two images respectively to obtain image features including global features and local features, the receptive field of the image can be expanded according to the global features and the local features to obtain a more accurate target displacement vector field, and the at least two images can be registered according to the target displacement vector field to obtain an image registration result with higher accuracy, which can be applied to scenarios for processing complex registration tasks, thereby solving the technical problems of low accuracy and limited application scenarios of image registration methods in related technologies.

[0170] It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the solution provided in Example 1, as well as the application scenario and implementation process, but is not limited to the solution provided in Example 1.

[0171] Example 6

[0172] According to an embodiment of the present disclosure, an image registration device for implementing the above-mentioned image configuration method is also provided. Figure 9 is a schematic diagram of an image registration device according to Example 6 of the present disclosure. As shown in Figure 9, the device 900 includes: an acquisition module 902, an extraction module 904, a determination module 906, and a registration module 908.

[0173] Among them, the acquisition module is configured to acquire at least two images, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; the extraction module is configured to perform feature extraction on at least two images respectively to obtain the image features of any one of the images; the determination module is configured to determine the target displacement vector field of the object to be analyzed based on the extracted image features; the registration module is configured to perform image registration on at least two images based on the target displacement vector field of the object to be analyzed to obtain the image registration result.

[0174] It should be noted that the acquisition module 902, extraction module 904, determination module 906, and registration module 908 correspond to steps S302 to S308 in Example 1. The examples and application scenarios implemented by the four modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above modules or units can be hardware components or software components stored in a memory (e.g., memory 1402) and processed by one or more processors (e.g., processors 1401a, 1401b, ..., 1401n). The above modules can also be part of the device and can be run in the computer terminal 1400A provided in Example 1.

[0175] In the above embodiment of the present disclosure, the extraction module is further configured to deform the first image based on the displacement vector field corresponding to the first image of at least two images to obtain a deformed image; and perform feature extraction on the deformed image and the second image of at least two images respectively to obtain image features of any one of the images, wherein the second image is used to represent the image other than the first image of the at least two images.

[0176] In the above embodiment of the present disclosure, the extraction module is further configured to extract features from the deformed image and the second image to obtain global features and local features; and to superimpose the global features and the local features to obtain image features.

[0177] In the above embodiment of the present disclosure, the determination module is further configured to obtain the dot product of the extracted image features to obtain a target cost volume; and perform convex optimization processing on the target cost volume to obtain a target displacement vector field.

[0178] In the above embodiment of the present disclosure, the determination module is further configured to perform parameter minimization processing on the target cost volume to obtain an initial displacement vector field; and perform an average pooling operation on the initial displacement vector field to obtain a target displacement vector field.

[0179] In the above embodiment of the present disclosure, the extraction module is further configured to perform multiple downsampling on at least two images respectively to obtain multiple downsampled images of any one image, wherein the resolutions of different downsampled images are different; based on the sampling displacement vector field corresponding to the downsampled image of the first image of the at least two images, the downsampled image of the first image is deformed to obtain a sampled deformed image; feature extraction is performed on the sampled deformed image and the downsampled image of the second image of the at least two images respectively to obtain downsampled image features of the downsampled image of any one image; and the features of multiple downsampled images are summarized to obtain image features of any one image.

[0180] In the above embodiment of the present disclosure, the determination module is also configured to obtain the dot product of the downsampled image features of the downsampled image of any image to obtain a sampling cost volume; perform convex optimization processing based on the sampling cost volume to obtain a sampling displacement vector field; and superimpose multiple sampling displacement vector fields to obtain a target displacement vector field.

[0181] In the above-mentioned embodiment of the present disclosure, the device is further configured to, when the downsampled image of the first image is a preset downsampled image, determine that the sampling displacement vector field corresponding to the downsampled image of the first image is the preset displacement vector field, wherein the preset downsampled image is used to represent the downsampled image of the maximum scale of the first image; when the downsampled image of the first image is not the preset downsampled image, perform upsampling processing on the sampling displacement vector field corresponding to the downsampled image of the next scale of the first image to obtain the sampling displacement vector field corresponding to the downsampled image of the first image.

[0182] It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the solution provided in Example 1, as well as the application scenario and implementation process, but is not limited to the solution provided in Example 1.

[0183] Example 7

[0184] According to an embodiment of the present disclosure, a target tracking device for implementing the above-mentioned target tracking method is also provided. Figure 10 is a schematic diagram of an image registration device according to Example 7 of the present disclosure. As shown in Figure 10, the device 1000 includes: an acquisition module 1002, an extraction module 1004, a determination module 1006, and a registration module 1008.

[0185] The acquisition module is configured to acquire at least two medical images, wherein any one of the medical images includes scan results of a target part of the same biological object under different conditions using computed tomography technology; the extraction module is configured to perform feature extraction on the at least two medical images to obtain image features of the at least one image, wherein the image features include global features and local features; the determination module is configured to determine a target displacement vector field of the target part based on the extracted image features; and the registration module is configured to perform image registration on the at least two medical images based on the target displacement vector field of the target part to obtain an image registration result.

[0186] It should be noted that the acquisition module 1002, extraction module 1004, determination module 1006, and registration module 1008 correspond to steps S502 to S508 in Example 2. The examples and application scenarios implemented by the four modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above modules or units can be hardware components or software components stored in a memory (e.g., memory 1402) and processed by one or more processors (e.g., processors 1401a, 1401b, ..., 1401n). The above modules can also be part of the device and can be run in the computer terminal 1400A provided in Example 1.

[0187] It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the solution provided in Example 1, as well as the application scenario and implementation process, but is not limited to the solution provided in Example 1.

[0188] Example 8

[0189] According to an embodiment of the present disclosure, a target tracking device for implementing the above-mentioned target tracking method is also provided. Figure 11 is a schematic diagram of an image registration device according to Example 8 of the present disclosure. As shown in Figure 11, the device 1100 includes: a first display module 1102 and a second display module 1104.

[0190] In which, the first display module is configured to respond to an input instruction on the operation interface and display at least two images on the operation interface, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; the second display module is configured to respond to an image registration instruction on the operation interface and display the image registration result on the operation interface, wherein the image registration result is obtained by performing image registration on at least two images based on the target displacement vector field of the object to be analyzed, and the target displacement vector field is determined based on the extracted image features, and the extracted image features include: global features and local features, and the extracted image features are obtained by performing feature extraction on at least two images respectively.

[0191] It should be noted that the first display module 1102 and the second display module 1104 correspond to steps S602 to S604 in Example 3. The examples and application scenarios implemented by the two modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above modules or units can be hardware components or software components stored in a memory (e.g., memory 1402) and processed by one or more processors (e.g., processors 1401a, 1401b, ..., 1401n). The above modules can also be part of the device and can be run in the computer terminal 1400A provided in Example 1.

[0192] It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the solution provided in Example 1, as well as the application scenario and implementation process, but is not limited to the solution provided in Example 1.

[0193] Example 9

[0194] According to an embodiment of the present disclosure, a target tracking device for implementing the above-mentioned target tracking method is also provided. Figure 12 is a schematic diagram of an image registration device according to Example 9 of the present disclosure. As shown in Figure 12, the device 1200 includes: a display module 1202, an extraction module 1204, a determination module 1206, a registration module 1208, and a driving module 1210.

[0195] Among them, the display module is configured to display at least two images on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; the extraction module is configured to perform feature extraction on at least two images respectively to obtain image features of any one of the images; the determination module is configured to determine the target displacement vector field of the object to be analyzed based on the extracted image features; the registration module is configured to perform image registration on at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result; the driving module is used to drive the VR device or AR device to render and display the image registration result.

[0196] It should be noted that the above-mentioned display module 1202, extraction module 1204, determination module 1206, registration module 1208, and driving module 1210 correspond to steps S702 to S710 in Example 4. The examples and application scenarios implemented by the five modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules or units can be hardware components or software components stored in a memory (e.g., memory 1402) and processed by one or more processors (e.g., processors 1401a, 1401b, ..., 1401n). The above-mentioned modules can also be part of the device and can be run in the computer terminal 1400A provided in Example 1.

[0197] It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the solution provided in Example 1, as well as the application scenario and implementation process, but is not limited to the solution provided in Example 1.

[0198] Example 10

[0199] According to an embodiment of the present disclosure, a target tracking device for implementing the above-mentioned target tracking method is also provided. Figure 13 is a schematic diagram of an image registration device according to embodiment 10 of the present disclosure. As shown in Figure 13, the device 1300 includes: an acquisition module 1302, an extraction module 1304, a determination module 1306, a registration module 1308, and a calling module 1310.

[0200] Among them, the acquisition module is configured to acquire at least two images by calling a first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter is at least two images, and any one of the images contains the shooting results of the object to be analyzed under different conditions; the extraction module is configured to perform feature extraction on at least two images respectively to obtain the image features of any one of the images; the determination module is configured to determine the target displacement vector field of the object to be analyzed based on the extracted image features; the registration module is used to perform image registration on at least two images based on the target displacement vector field of the object to be analyzed to obtain the image registration result; the calling module is configured to output the image registration result by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the image registration result.

[0201] It should be noted that the acquisition module 1302, extraction module 1304, determination module 1306, registration module 1308, and calling module 1310 correspond to steps S802 to S810 in Example 5. The examples and application scenarios implemented by the five modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above modules or units can be hardware components or software components stored in a memory (e.g., memory 1402) and processed by one or more processors (e.g., processors 1401a, 1401b, ..., 1401n). The above modules can also be run as part of the device in the computer terminal 10 provided in Example 1.

[0202] It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the solution provided in Example 1, as well as the application scenario and implementation process, but is not limited to the solution provided in Example 1.

[0203] Example 11

[0204] The embodiments of the present disclosure may provide an electronic device, which may be an AR / VR device, and the AR / VR device may be any AR / VR device in an AR / VR device group. Optionally, in this embodiment, the AR / VR device may be replaced by a terminal device such as a mobile terminal.

[0205] Optionally, in this embodiment, the AR / VR device may be located in at least one of a plurality of network devices of a computer network.

[0206] In this embodiment, the above-mentioned AR / VR device can execute the program code of the following steps in the image registration method: obtaining at least two images, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; performing feature extraction on the at least two images respectively to obtain the image features of any one of the images; determining the target displacement vector field of the object to be analyzed based on the extracted image features; and performing image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result.

[0207] Optionally, Figure 14 is a block diagram of a computer terminal according to an embodiment of the present disclosure. As shown in Figure 14, the computer terminal 1400A may include: one or more (only one is shown in the figure) processors 1401, a memory 1402, a storage controller, and a peripheral interface, wherein the peripheral interface is connected to a radio frequency module, an audio module, and a display.

[0208] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the image registration method and device in the embodiment of the present disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned image registration method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely located relative to the processor, and these remote memories may be connected to the terminal A via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0209] The present disclosure also provides a processor. Figure 15 shows a block diagram of a processor for implementing the present disclosure. As shown in Figure 15, the processor 1500 is configured to run a program, wherein the program, when executed by the processor, executes the method in the above embodiment.

[0210] Optionally, the processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain at least two images, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; perform feature extraction on the at least two images respectively to obtain the image features of any one of the images; determine the target displacement vector field of the object to be analyzed based on the extracted image features; and perform image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result.

[0211] Optionally, the processor may also execute the program code of the following steps: deforming the first image based on the displacement vector field corresponding to the first image of at least two images to obtain a deformed image; performing feature extraction on the deformed image and the second image of at least two images respectively to obtain image features of any one image, wherein the second image is used to represent the image other than the first image of at least two images.

[0212] Optionally, the processor may further execute program codes of the following steps: extracting features from the deformed image and the second image to obtain global features and local features; and superimposing the global features and the local features to obtain image features.

[0213] Optionally, the processor may further execute program codes of the following steps: obtaining the dot product of the extracted image features to obtain a target cost volume; and performing convex optimization processing on the target cost volume to obtain a target displacement vector field.

[0214] Optionally, the processor may further execute program codes of the following steps: performing parameter minimization processing on the target cost volume to obtain an initial displacement vector field; and performing an average pooling operation on the initial displacement vector field to obtain a target displacement vector field.

[0215] Optionally, the processor may also execute the program code of the following steps: downsampling at least two images multiple times to obtain multiple downsampled images of any one image, wherein the resolutions of different downsampled images are different; deforming the downsampled image of the first image based on the sampling displacement vector field corresponding to the downsampled image of the first image of at least two images to obtain a sampled deformed image; extracting features from the sampled deformed image and the downsampled image of the second image of at least two images to obtain downsampled image features of the downsampled image of any one image; and summarizing the features of multiple downsampled images to obtain image features of any one image.

[0216] Optionally, the processor may also execute the program code of the following steps: obtaining the dot product of the downsampled image features of any image to obtain a sampling cost volume; performing convex optimization processing based on the sampling cost volume to obtain a sampling displacement vector field; and superimposing multiple sampling displacement vector fields to obtain a target displacement vector field.

[0217] Optionally, the processor may also execute the program code of the following steps: when the downsampled image of the first image is a preset downsampled image, determining that the sampling displacement vector field corresponding to the downsampled image of the first image is a preset displacement vector field, wherein the preset downsampled image is used to represent the maximum-scale downsampled image of the first image; when the downsampled image of the first image is not the preset downsampled image, performing upsampling processing on the sampling displacement vector field corresponding to the downsampled image of the next scale of the first image to obtain the sampling displacement vector field corresponding to the downsampled image of the first image.

[0218] The processor can call information and applications stored in the memory through a transmission device to execute the following steps: obtaining at least two medical images, wherein any one of the medical images contains scan results of a target part of the same biological object under different conditions using computed tomography technology; performing feature extraction on the at least two medical images respectively to obtain image features of any one of the images, wherein the image features include: global features and local features; determining a target displacement vector field of the target part based on the extracted image features; and performing image registration on the at least two medical images based on the target displacement vector field of the target part to obtain an image registration result.

[0219] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: responding to the input instruction on the operation interface, displaying at least two images on the operation interface, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; responding to the image registration instruction on the operation interface, displaying the image registration result on the operation interface, wherein the image registration result is obtained by performing image registration on the at least two images based on the target displacement vector field of the object to be analyzed, and the target displacement vector field is determined based on the extracted image features, and the extracted image features include: global features and local features, and the extracted image features are obtained by performing feature extraction on at least two images respectively.

[0220] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: display at least two images on the presentation screen of a virtual reality (VR) device or an augmented reality (AR) device, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; perform feature extraction on the at least two images respectively to obtain image features of any one of the images; determine the target displacement vector field of the object to be analyzed based on the extracted image features; perform image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result; drive the VR device or AR device to render and display the image registration result.

[0221] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain at least two images by calling the first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter is at least two images, and any one of the images contains the shooting results of the object to be analyzed under different conditions; perform feature extraction on the at least two images respectively to obtain the image features of any one of the images; determine the target displacement vector field of the object to be analyzed based on the extracted image features; perform image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result; output the image registration result by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the image registration result.

[0222] According to the embodiment of the present disclosure, at least two images are acquired, wherein any one of the images includes the photographic results of the object to be analyzed under different conditions; feature extraction is performed on each of the at least two images to obtain image features of each image; based on the extracted image features, a target displacement vector field of the object to be analyzed is determined; based on the target displacement vector field of the object to be analyzed, at least two images are registered to obtain an image registration result, thereby achieving the purpose of improving the accuracy of image registration. It is easy to notice that feature extraction can be performed on each of the at least two images to obtain image features containing global features and local features, the receptive field of the image can be expanded based on the global features and the local features to obtain a more accurate target displacement vector field, and the at least two images can be registered based on the target displacement vector field to obtain an image registration result with higher accuracy, thereby solving the technical problem of low accuracy in processing the registration task in the related art.

[0223] Those skilled in the art will appreciate that the structure shown in FIG14 is merely illustrative, and the computer terminal may also be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile internet device (MID), a PAD, or other terminal device. FIG14 does not limit the structure of the above-mentioned electronic devices. For example, the computer terminal A may include more or fewer components (such as a network interface, a display device, etc.) than those shown in FIG14, or may have a configuration different from that shown in FIG14.

[0224] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0225] Example 12

[0226] The embodiment of the present disclosure further provides a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store the program code executed by the image registration method provided in the embodiment 1 above.

[0227] Computer-readable storage media, also known as computer storage media, may include data signals transmitted in baseband or as part of a carrier wave, carrying readable program code. Such transmitted data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable storage media may transmit, propagate, or transfer programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0228] The program code contained in the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, radio frequency, etc., or any suitable combination of the foregoing.

[0229] Optionally, in this embodiment, the above-mentioned computer-readable storage medium can be located in any computer terminal in the AR / VR device terminal group in the AR / VR device network, or in any mobile terminal in the mobile terminal group.

[0230] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: acquiring at least two images, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; performing feature extraction on the at least two images respectively to obtain image features of any one of the images; determining the target displacement vector field of the object to be analyzed based on the extracted image features; and performing image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result.

[0231] Optionally, the above-mentioned storage medium is also configured to store program code for executing the following steps: deforming the first image based on the displacement vector field corresponding to the first image of at least two images to obtain a deformed image; performing feature extraction on the deformed image and the second image of at least two images respectively to obtain image features of any one of the images, wherein the second image is used to represent the image other than the first image of the at least two images.

[0232] Optionally, the storage medium is further configured to store program codes for executing the following steps: extracting features from the deformed image and the second image to obtain global features and local features; and superimposing the global features and the local features to obtain image features.

[0233] Optionally, the storage medium is further configured to store program codes for executing the following steps: obtaining the dot product of the extracted image features to obtain a target cost volume; and performing convex optimization on the target cost volume to obtain a target displacement vector field.

[0234] Optionally, the storage medium is further configured to store program codes for executing the following steps: performing parameter minimization processing on the target cost volume to obtain an initial displacement vector field; performing an average pooling operation on the initial displacement vector field to obtain a target displacement vector field.

[0235] Optionally, the storage medium is further configured to store program codes for executing the following steps: downsampling at least two images multiple times to obtain multiple downsampled images of any one image, wherein the resolutions of different downsampled images are different; deforming the downsampled image of the first image based on the sampling displacement vector field corresponding to the downsampled image of the first image of at least two images to obtain a sampled deformed image; extracting features from the sampled deformed image and the downsampled image of the second image of at least two images to obtain downsampled image features of the downsampled image of any one image; and summarizing the features of multiple downsampled images to obtain image features of any one image.

[0236] Optionally, the above-mentioned storage medium is also configured to store program code for executing the following steps: obtaining the dot product of the downsampled image features of the downsampled image of any image to obtain a sampling cost volume; performing convex optimization processing based on the sampling cost volume to obtain a sampling displacement vector field; and superimposing multiple sampling displacement vector fields to obtain a target displacement vector field.

[0237] Optionally, the above-mentioned storage medium is also configured to store program code for executing the following steps: when the downsampled image of the first image is a preset downsampled image, determining that the sampling displacement vector field corresponding to the downsampled image of the first image is a preset displacement vector field, wherein the preset downsampled image is used to represent the maximum-scale downsampled image of the first image; when the downsampled image of the first image is not the preset downsampled image, upsampling the sampling displacement vector field corresponding to the downsampled image of the next scale of the first image to obtain the sampling displacement vector field corresponding to the downsampled image of the first image.

[0238] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: acquiring at least two medical images, wherein any one of the medical images contains scanning results of a target part of the same biological object under different conditions using computed tomography technology; performing feature extraction on the at least two medical images respectively to obtain image features of any one of the images, wherein the image features include: global features and local features; determining a target displacement vector field of the target part based on the extracted image features; and performing image registration on the at least two medical images based on the target displacement vector field of the target part to obtain an image registration result.

[0239] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: in response to an input instruction on the operation interface, displaying at least two images on the operation interface, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; in response to an image registration instruction on the operation interface, displaying the image registration result on the operation interface, wherein the image registration result is obtained by performing image registration on the at least two images based on the target displacement vector field of the object to be analyzed, and the target displacement vector field is determined based on the extracted image features, and the extracted image features include: global features and local features, and the extracted image features are obtained by performing feature extraction on at least two images respectively.

[0240] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: displaying at least two images on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; performing feature extraction on the at least two images respectively to obtain image features of any one of the images; determining the target displacement vector field of the object to be analyzed based on the extracted image features; performing image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result; and driving the VR device or AR device to render and display the image registration result.

[0241] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: acquiring at least two images by calling a first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter is at least two images, and any one of the images contains the shooting results of the object to be analyzed under different conditions; performing feature extraction on the at least two images respectively to obtain image features of any one of the images; determining the target displacement vector field of the object to be analyzed based on the extracted image features; performing image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result; and outputting the image registration result by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the image registration result.

[0242] According to the embodiment of the present disclosure, at least two images are acquired, wherein any one of the images includes the photographic results of the object to be analyzed under different conditions; feature extraction is performed on each of the at least two images to obtain image features of each image; based on the extracted image features, a target displacement vector field of the object to be analyzed is determined; based on the target displacement vector field of the object to be analyzed, at least two images are registered to obtain an image registration result, thereby achieving the purpose of improving the accuracy of image registration. It is easy to notice that feature extraction can be performed on each of the at least two images to obtain image features containing global features and local features, the receptive field of the image can be expanded based on the global features and the local features to obtain a more accurate target displacement vector field, and the at least two images can be registered based on the target displacement vector field to obtain an image registration result with higher accuracy, thereby solving the technical problem of low accuracy in processing the registration task in the related art.

[0243] The serial numbers of the above-mentioned embodiments of the present disclosure are for description only and do not represent the advantages or disadvantages of the embodiments.

[0244] In the above embodiments of the present disclosure, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0245] In the several embodiments provided in the present disclosure, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0246] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0247] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0248] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM, abbreviated as Read-Only Memory), a random access memory (RAM, abbreviated as Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk.

[0249] The above is only a preferred embodiment of the present disclosure. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present disclosure. These improvements and modifications should also be regarded as within the scope of protection of the present disclosure. Industrial Applicability

[0250] The solution provided by the embodiments of the present disclosure can be applied to the process of image registration, obtaining at least two images, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; performing feature extraction on the at least two images respectively to obtain image features of any one of the images; determining the target displacement vector field of the object to be analyzed based on the extracted image features; and performing image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result, thereby solving the technical problems of low accuracy and limited application scenarios of image registration methods in related technologies.

Claims

1. An image registration method, characterized in that: include: Acquire at least two images, wherein any one of the images contains shooting results of the object to be analyzed under different conditions; Performing feature extraction on the at least two images respectively to obtain image features of any one of the images; Determining a target displacement vector field of the object to be analyzed based on the extracted image features; Based on the target displacement vector field of the object to be analyzed, the at least two images are registered to obtain an image registration result.

2. The method according to claim 1, characterized in that Extracting features from the at least two images respectively to obtain image features of any one of the images comprises: Based on the displacement vector field corresponding to the first image of the at least two images, deforming the first image to obtain a deformed image; Feature extraction is performed on the deformed image and a second image of the at least two images to obtain image features of any one of the images, wherein the second image is used to represent images of the at least two images other than the first image.

3. The method according to claim 2, characterized in that Extracting features from the deformed image and the second image of the at least two images respectively to obtain image features of any one of the images includes: Performing feature extraction on the deformed image and the second image to obtain global features and local features; The global feature and the local feature are superimposed to obtain the image feature.

4. The method according to claim 1, characterized in that Determining a target displacement vector field of the object to be analyzed based on the extracted image features includes: Obtaining the dot product of the extracted image features to obtain a target cost volume; The target cost volume is subjected to convex optimization processing to obtain the target displacement vector field.

5. The method according to claim 4, characterized in that The target cost volume is subjected to convex optimization processing to obtain a target displacement vector field, including: Performing parameter minimization processing on the target cost volume to obtain an initial displacement vector field; An average pooling operation is performed on the initial displacement vector field to obtain the target displacement vector field.

6. The method according to any one of claims 1 to 5, characterized in that Extracting features from the at least two images respectively to obtain image features of any one of the images comprises: Downsampling the at least two images multiple times respectively to obtain multiple downsampled images of the any one image, wherein different downsampled images have different resolutions; Based on a sampling displacement vector field corresponding to a downsampled image of a first image of the at least two images, deforming the downsampled image of the first image to obtain a sampled deformed image; Performing feature extraction on the sampled deformed image and the downsampled image of the second image of the at least two images respectively, to obtain downsampled image features of the downsampled image of any one of the images; A plurality of the downsampled image features are aggregated to obtain image features of the arbitrary image.

7. The method according to claim 6, characterized in that Determining a target displacement vector field of the object to be analyzed based on the extracted image features includes: Obtaining a dot product of downsampled image features of a downsampled image of any one of the images to obtain a sampling cost volume; Performing convex optimization processing on the sampling cost volume to obtain a sampling displacement vector field; A plurality of the sampled displacement vector fields are superimposed to obtain the target displacement vector field.

8. The method according to claim 6, characterized in that The method further comprises: In a case where the downsampled image of the first image is a preset downsampled image, determining that a sampling displacement vector field corresponding to the downsampled image of the first image is a preset displacement vector field, wherein the preset downsampled image is used to represent a downsampled image of a maximum scale of the first image; When the downsampled image of the first image is not the preset downsampled image, upsampling is performed on the sampling displacement vector field corresponding to the downsampled image of the next scale of the first image to obtain the sampling displacement vector field corresponding to the downsampled image of the first image.

9. An image registration method, characterized in that: include: Acquire at least two medical images, wherein any one of the medical images contains scan results of a target part of a same biological object under different conditions using a computer tomography technique; Performing feature extraction on the at least two medical images respectively to obtain image features of any one of the images; Determining a target displacement vector field of the target part based on the extracted image features; Based on the target displacement vector field of the target part, the at least two medical images are registered to obtain an image registration result.

10. An image registration method, characterized in that: include: In response to an input instruction acting on the operation interface, at least two images are displayed on the operation interface, wherein any one of the images contains the photographing results of the object to be analyzed under different conditions; In response to an image registration instruction applied on the operation interface, an image registration result is displayed on the operation interface, wherein the image registration result is obtained by performing image registration on the at least two images based on a target displacement vector field of the object to be analyzed, the target displacement vector field is determined based on extracted image features, and the extracted image features are obtained by performing feature extraction on the at least two images respectively.

11. An image registration method, characterized in that: include: Displaying at least two images on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device, wherein any one of the images includes shooting results of the object to be analyzed under different conditions; Performing feature extraction on the at least two images respectively to obtain image features of any one of the images; Determining a target displacement vector field of the object to be analyzed based on the extracted image features; Based on the target displacement vector field of the object to be analyzed, performing image registration on the at least two images to obtain an image registration result; The VR device or the AR device is driven to render and display the image registration result.

12. The method according to claim 11, characterized in that Extracting features from the at least two images respectively to obtain image features of any one of the images comprises: Based on the displacement vector field corresponding to the first image of the at least two images, deforming the first image to obtain a deformed image; Feature extraction is performed on the deformed image and a second image of the at least two images to obtain image features of any one of the images, wherein the second image is used to represent images of the at least two images other than the first image.

13. The method according to claim 12, characterized in that Extracting features from the deformed image and the second image of the at least two images respectively to obtain image features of any one of the images includes: Performing feature extraction on the deformed image and the second image to obtain global features and local features; The global feature and the local feature are superimposed to obtain the image feature.

14. The method according to claim 11, characterized in that Determining a target displacement vector field of the object to be analyzed based on the extracted image features includes: Obtaining the dot product of the extracted image features to obtain a target cost volume; The target cost volume is subjected to convex optimization processing to obtain the target displacement vector field.

15. An image registration method, characterized in that: include: Acquire at least two images by calling a first interface, wherein the first interface includes a first parameter, a parameter value of the first parameter is the at least two images, and any one of the images includes shooting results of the object to be analyzed under different conditions; Performing feature extraction on the at least two images respectively to obtain image features of any one of the images; Determining a target displacement vector field of the object to be analyzed based on the extracted image features; Based on the target displacement vector field of the object to be analyzed, performing image registration on the at least two images to obtain an image registration result; The image registration result is output by calling a second interface, wherein the second interface includes a second parameter, and a parameter value of the second parameter is the image registration result.

16. An electronic device, characterized in that: include: A memory configured to store an executable program; A processor configured to run the program, wherein when the program is run, the program executes claim 1 The method according to any one of claims 1 to 15.

17. A computer program product, wherein: The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 15.

18. A computer program product, wherein: The invention comprises a non-volatile computer-readable storage medium storing a computer program, wherein the computer program implements the method according to any one of claims 1 to 15 when executed by a processor.

19. A computer program, wherein When the computer program is executed by a processor, the method according to any one of claims 1 to 15 is implemented.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 15.

Images