Image registration method and device, computer device and storage medium

Abstract

Embodiments of the present application disclose a kind of image registration method, device, computer equipment and storage medium, belong to image processing technical field.The method comprises: obtaining first pathological image and second pathological image, first pathological image and second pathological image are the pathological image of same target object;First pathological image and second pathological image are profiled registration, obtain first registration parameter, first registration parameter is the parameter for adjusting the profile of target object in second pathological image;First pathological image and second pathological image are registered in target site, obtain second registration parameter, second registration parameter is the parameter for adjusting the position of target site in second pathological image;Based on first registration parameter and second registration parameter, adjust second pathological image, obtain the registration image of second pathological image.The method adopts the registration mode of first overall registration then local accurate registration, reduces the registration difficulty, improves the accuracy of image registration.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to an image registration method, apparatus, computer equipment, and storage medium. Background Art

[0002] Image registration refers to the process of matching different images of the same object. Image registration is widely used in image three-dimensional reconstruction, remote sensing image analysis, medical image processing and other fields.

[0003] In related technologies, image registration is achieved by extracting feature points from the two images to be processed and matching the corresponding feature points in the two images. However, the process of extracting and matching feature points is very time-consuming, resulting in low registration efficiency. Summary of the Invention

[0004] The embodiments of the present application provide an image registration method, apparatus, computer device, and storage medium, which reduce the difficulty of registration and improve the efficiency and accuracy of image registration. The technical solution is as follows:

[0005] In one aspect, a method for image registration is provided, the method comprising:

[0006] Acquiring a first pathological image and a second pathological image, where the first pathological image and the second pathological image are pathological images of the same target object;

[0007] Performing contour registration on the first pathological image and the second pathological image to obtain a first registration parameter, where the first registration parameter is a parameter used to adjust the contour of the target object in the second pathological image;

[0008] performing target part registration on the first pathological image and the second pathological image to obtain a second registration parameter, wherein the target part is a part of the target object in a target state, and the second registration parameter is a parameter used to adjust the position of the target part in the second pathological image;

[0009] Based on the first registration parameter and the second registration parameter, the second pathological image is adjusted to obtain a registered image of the second pathological image, wherein the registered image is consistent with the contour of the target object in the first pathological image and the position of the target part is consistent.

[0010] In another aspect, an image registration apparatus is provided, the apparatus comprising:

[0011] An image acquisition module, configured to acquire a first pathological image and a second pathological image, wherein the first pathological image and the second pathological image are pathological images of the same target object;

[0012] a first registration module, configured to perform contour registration on the first pathological image and the second pathological image to obtain a first registration parameter, wherein the first registration parameter is a parameter for adjusting the contour of the target object in the second pathological image;

[0013] a second registration module, configured to register a target part of the first pathological image and the second pathological image to obtain a second registration parameter, wherein the target part is a part of the target object in a target state, and the second registration parameter is a parameter for adjusting a position of the target part in the second pathological image;

[0014] A registration image acquisition module is used to adjust the second pathological image based on the first registration parameter and the second registration parameter to obtain a registration image of the second pathological image, wherein the registration image is consistent with the outline of the target object in the first pathological image and the position of the target part is consistent.

[0015] In a possible implementation, the first registration module includes:

[0016] a contour image acquiring unit, configured to acquire a first contour image of the first pathological image and a second contour image of the second pathological image;

[0017] The first registration unit is configured to perform contour registration on the first contour image and the second contour image to obtain the first registration parameter.

[0018] In another possible implementation, the contour image acquisition unit is configured to:

[0019] performing binarization processing on the first pathological image and the second pathological image respectively to obtain the first contour image and the second contour image; or,

[0020] Grayscale processing is performed on the first pathological image and the second pathological image respectively to obtain the first contour image and the second contour image.

[0021] In another possible implementation, the second registration module is configured to:

[0022] adjusting the second pathological image based on the first registration parameter to obtain a third pathological image, wherein the third pathological image is consistent with the contour of the target object in the first pathological image;

[0023] The first pathological image and the third pathological image are registered at the target site to obtain the second registration parameter.

[0024] In another possible implementation, the second registration module includes:

[0025] a region segmentation unit, configured to perform region segmentation on the first pathological image and the third pathological image, respectively, to obtain a first region image and a second region image, wherein the first region image is an image of the region where the target part is located in the first pathological image, and the second region image is an image of the region where the target part is located in the third pathological image;

[0026] The second registration unit is configured to perform target part registration on the first region image and the second region image to obtain the second registration parameter.

[0027] In another possible implementation, the region segmentation unit is used to call a region segmentation model to perform region segmentation on the first pathological image and the third pathological image respectively to obtain the first region image and the second region image, and the region segmentation model is used to segment the image of the region where the part in the target state is located from the image.

[0028] In another possible implementation, the first region image includes multiple sub-regions, the second region image includes multiple sub-regions, and the second registration unit is configured to:

[0029] Using the positions of the plurality of sub-regions in the first region image as a reference, moving the positions of the corresponding sub-regions in the second region image so that the positions of the plurality of sub-regions in the second region image are respectively the same as the positions of the plurality of sub-regions in the first region image;

[0030] The second registration parameter is determined based on the movement distances of the plurality of sub-regions in the second region image.

[0031] In another possible implementation, the registration image acquisition module is configured to:

[0032] combining the first registration parameter and the second registration parameter to obtain a third registration parameter;

[0033] Based on the third registration parameter, the second pathological image is adjusted to obtain a registered image of the second pathological image.

[0034] In another possible implementation, the registration image acquisition module is configured to:

[0035] adjusting the second pathological image based on the first registration parameter to obtain a fourth pathological image, wherein the fourth pathological image is consistent with the contour of the target object in the first pathological image;

[0036] Based on the second registration parameter, the fourth pathological image is adjusted to obtain a registered image of the second pathological image.

[0037] In another possible implementation, the first registration parameter includes at least one of a translation parameter, a rotation parameter, or a scaling parameter, and the first registration module is configured to:

[0038] Using the outline of the target object in the first pathological image as a reference, translating the second pathological image so that the target object in the second pathological image is at the same height as the target object in the first pathological image, and determining the translation distance of the second pathological image as the translation parameter; or

[0039] Using the outline of the target object in the first pathological image as a reference, rotating the second pathological image so that the target object in the second pathological image is at the same angle as the target object in the first pathological image, and determining the angle of rotation of the second pathological image as the rotation parameter; or

[0040] The second pathological image is scaled with reference to the outline of the target object in the first pathological image so that the target object in the second pathological image is the same size as the target object in the first pathological image, and the scaled size of the second pathological image is determined as the scaling parameter.

[0041] In another possible implementation, the target object is lesion tissue, and the image acquisition module is used to downsample the fifth pathological image and the sixth pathological image of the lesion tissue respectively to obtain the first pathological image and the second pathological image.

[0042] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the operations performed by the image registration method described in the above aspects.

[0043] On the other hand, a computer-readable storage medium is provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement the operations performed by the image registration method described in the above aspects.

[0044] On the other hand, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the operations performed by the image registration method according to the above aspects are implemented.

[0045] The image registration method provided in the embodiment of the present application first performs overall registration of the first pathological image and the second pathological image based on the contours of the target objects in the first pathological image and the second pathological image, and then performs local precise registration of the first pathological image and the second pathological image based on the target parts of the target objects in the first pathological image and the second pathological image. There is no need to extract every feature point in the first pathological image and the second pathological image, which reduces the difficulty of registration, improves the registration efficiency, and avoids inaccurate registered images caused by inaccurate feature points, thereby improving the accuracy of image registration. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0047] Figure 1 This is a flow chart of an image registration method provided in an embodiment of the present application;

[0048] Figure 2 is a flowchart of another image registration method provided in an embodiment of the present application;

[0049] Figure 3 Schematic diagram of two pathological images of a lesion tissue provided in an embodiment of the present application;

[0050] Figure 4 is a schematic diagram of a contour image of a pathological image provided in an embodiment of the present application;

[0051] Figure 5 is a schematic diagram of a contour image of a pathological image provided in an embodiment of the present application;

[0052] Figure 6 is a schematic diagram of a pathological image and a regional image provided in an embodiment of the present application;

[0053] Figure 7 is a schematic diagram of another pathological image and regional image provided in an embodiment of the present application;

[0054] Figure 8 is a schematic diagram of a regional image provided in an embodiment of the present application;

[0055] Figure 9 is a schematic diagram of another pathological image and regional image provided in an embodiment of the present application;

[0056] Figure 10 is a schematic diagram of another pathological image and a registered image provided in an embodiment of the present application;

[0057] Figure 11 is a schematic diagram of another pathological image and a registered image provided in an embodiment of the present application;

[0058] Figure 12 is a flowchart of another image registration method provided in an embodiment of the present application;

[0059] Figure 13 is a structural diagram of an image registration device provided in an embodiment of the present application;

[0060] Figure 14 is a structural diagram of another image registration device provided in an embodiment of the present application;

[0061] Figure 15 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present application;

[0062] Figure 16 This is a structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0064] It is understood that the terms "first," "second," and so forth, used herein may be used to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are used solely to distinguish one concept from another. For example, a first pathology image may be referred to as a second pathology image, and a second pathology image may be referred to as a first pathology image, without departing from the scope of this application.

[0065] As used herein, the terms "at least one," "plurality," "each," and "any" include one, two, or more, "plurality" includes two or more, "each" refers to each of the corresponding plurality, and "any" refers to any one of the plurality. For example, if a plurality of regions includes three regions, then "each" refers to each of the three regions, and "any" refers to any one of the three regions, which can be the first, second, or third.

[0066] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0067] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and smart transportation.

[0068] Computer vision (CV) is the science of making machines "see." Specifically, it refers to the use of cameras and computers to replace the human eye in identifying, tracking, and measuring objects. Furthermore, it involves image processing, which transforms the computer's image into an image more suitable for human observation or transmission to instrumentation. As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, autonomous driving, and smart transportation. It also includes common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0069] Machine learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning through demonstration.

[0070] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, Internet of Vehicles, automatic driving, smart transportation, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0071] The solution provided in the embodiment of the present application aligns the acquired first pathological image and the second pathological image based on artificial intelligence computer vision, machine learning and other technologies, and obtains a registered image of the second pathological image with the first pathological image as a reference.

[0072] The image registration method provided in the embodiment of the present application can be executed by a computer device. Optionally, the computer device is a terminal or a server. Optionally, the server is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Optionally, the terminal is a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, medical equipment, car terminal, etc., but is not limited to this.

[0073] In one possible implementation, the computer program involved in the embodiments of the present application can be deployed and executed on a computer device, or on multiple computer devices located at one location, or on multiple computer devices distributed at multiple locations and interconnected through a communication network. Multiple computer devices distributed at multiple locations and interconnected through a communication network can constitute a blockchain system.

[0074] In one possible implementation, the computer device used to process images in the embodiment of the present application is a node in a blockchain system, which can obtain a first pathological image and a second pathological image, and obtain first registration parameters and second registration parameters for adjusting the second pathological image by processing the first pathological image and the second pathological image. The second pathological image is registered based on the obtained first registration parameters and second registration parameters to obtain a registered image of the second pathological image.

[0075] Figure 1 This is a flow chart of an image registration method provided by an embodiment of the present application. The execution subject of the embodiment of the present application is a computer device, see Figure 1 , the method comprises the following steps:

[0076] 101. Acquire a first pathological image and a second pathological image.

[0077] Among them, the first pathological image and the second pathological image are pathological images of the same target object, the second pathological image is the image to be registered, the first pathological image is the reference image when registering the second pathological image, and the target object is any object, for example, the target object is a lesion tissue or other object.

[0078] 102. Perform contour registration on the first pathological image and the second pathological image to obtain a first registration parameter, where the first registration parameter is a parameter used to adjust the contour of the target object in the second pathological image.

[0079] Among them, the contour of the target object can reflect the overall position of the target object in the first pathological image and the second target. Based on the contour of the target object, the first pathological image and the second pathological image are overall contour-aligned to obtain first alignment parameters. For example, the first alignment parameters are rotation parameters, translation parameters, scaling parameters, etc.

[0080] 103. Perform registration of the target part on the first pathological image and the second pathological image to obtain a second registration parameter, where the second registration parameter is a parameter used to adjust the position of the target part in the second pathological image.

[0081] The target site is a site in a target object that is in a target state. The target state includes either a normal state or an abnormal state. A normal state refers to a state that has not changed, while an abnormal state refers to a state that has changed. For example, if the target object is a lesion tissue, the normal state refers to the state of the area in the lesion tissue that has not developed a lesion, and the abnormal state refers to the state of the area in the lesion tissue that has developed a lesion. For example, the state of the lesion area in the lesion tissue is an abnormal state.

[0082] Based on the target part of the target object, the first pathological image and the second pathological image are locally registered at the target part to obtain second registration parameters, for example, the second registration parameters are rotation parameters, translation parameters, scaling parameters, etc.

[0083] 104. Adjust the second pathological image based on the first registration parameter and the second registration parameter to obtain a registered image of the second pathological image.

[0084] After the computer device obtains the first registration parameter and the second registration parameter, it can adjust the second pathological image. In the process of adjusting the second pathological image, the position of the target object and the position of the area where the target part is located in the second pathological image can be adjusted at the same time, or the position of the target object can be adjusted first, and then the position of the area where the target part is located can be adjusted to obtain a registered image. The registered image has the same outline as the target object in the first pathological image, and the position of the target part is consistent, that is, the registered image is consistent with the position, angle, and size of the first pathological image.

[0085] An embodiment of the present application provides an image registration method, which first performs overall registration on the first pathological image and the second pathological image based on the contours of the target objects in the first pathological image and the second pathological image, and then performs local precise registration on the first pathological image and the second pathological image based on the target parts of the target objects in the first pathological image and the second pathological image. This method eliminates the need to extract every feature point in the first pathological image and the second pathological image, reduces the difficulty of registration, improves the efficiency of registration, and avoids inaccurate registered images resulting from inaccurate feature points, thereby improving the accuracy of image registration.

[0086] above Figure 1 The embodiment shown is only a brief introduction to the image registration process. Figure 2 The illustrated embodiment provides a detailed description of the implementation method for obtaining registration parameters during image registration.

[0087] Figure 2 This is a flow chart of an image registration method provided by an embodiment of the present application. The execution subject of the embodiment of the present application is a computer device. Figure 2 , the method comprises the following steps:

[0088] 201. Acquire a first pathological image and a second pathological image.

[0089] The first pathological image and the second pathological image are pathological images of the same target object, the second pathological image is the pathological image to be registered, the first pathological image is the pathological image used as a reference for registering the second pathological image, and the target object is any object, for example, a lesion tissue or other object. Optionally, the first pathological image and the second pathological image are pathological images acquired by a computer device, or are pathological images sent to the computer device by another computer device.

[0090] Optionally, the first pathological image and the second pathological image are digital slices (WSI, Whole Slide Image) obtained by scanning pathological slices of lesion tissue using a fully automatic microscope scanning device, or are obtained by other methods. The embodiment of the present application does not limit the method of obtaining pathological images.

[0091] When the target object is a lesion tissue, the computer device obtains a first pathological section and a second pathological section of the lesion tissue, scans the first pathological section and the second pathological section respectively, and obtains a first pathological image corresponding to the first pathological section and a second pathological image corresponding to the second pathological section. The lesion tissue is removed from the human body. The lesion tissue is placed in a fixative to make a pathological section for observation under a microscope. The fixative is used to stabilize the tissue to prevent decay. For the same lesion tissue, in order to facilitate observation under a microscope, different components of the lesion tissue are stained with different stains, and then different staining techniques are applied to display specific tissue components under a microscope. The first pathological section and the second pathological section are obtained using different staining techniques for the same lesion tissue.

[0092] During the preparation and scanning of pathological sections, manual operation may cause different pathological images of the same lesion tissue to have positional offset and direction rotation, which is not convenient for subsequent correspondence with different pathological images. For example, see Figure 3 The first pathological image 301 and the second pathological image 302 shown in the figure have completely different positions and directions of the lesion tissues, so it is necessary to perform registration on the different pathological images.

[0093] The size of the pathological image is relatively large. For example, the size of the pathological image is 10000*10000. In one possible implementation, after the computer device obtains the pathological image, it stores the pathological image in a pyramid form, that is, it downsamples the pathological image to obtain pathological images of different sizes, and stores the pathological images of multiple sizes together. When the pathological image is subsequently processed, the pathological image of the corresponding size is read according to the size of the pathological image to be processed. For example, for the same pathological image, the size of 10000*10000, 8000*8000, and 4000*4000 are stored. When performing image registration, the pathological image size required is 5000*5000. Then, the pathological image with the closest size can be directly read from the stored pathological images, that is, the pathological image of 4000*4000 can be read. The above embodiment can be used to store and read both the first pathological image and the second pathological image.

[0094] In another possible implementation, when registering the pathological images, the original pathological image is downsampled to obtain downsampled pathological images, that is, the fifth pathological image and the sixth pathological image are downsampled to obtain a downsampled first pathological image and a downsampled second pathological image.

[0095] Optionally, the first pathological image and the second pathological image are of any size, the first pathological image and the second pathological image are of the same size, or the first pathological image and the second pathological image are of different sizes.

[0096] In one possible implementation, in order to make the subsequent registration process more accurate, the first pathological image and the second pathological image are preprocessed, for example, filtering and image enhancement are performed on the reference image, and the preprocessed first pathological image and the second pathological image are registered.

[0097] 202. Acquire a first contour image of the first pathological image and a second contour image of the second pathological image.

[0098] Wherein, both the first contour image and the second contour image include the contour of the target object.

[0099] In one possible implementation, a computer device performs grayscale processing on the first and second pathological images to obtain first and second contour images. The grayscale processing of the first and second contour images clearly displays the contour of the target object, avoiding the influence of various colors in the first and second pathological images.

[0100] In another possible implementation, the computer device performs binarization processing on the first pathological image and the second pathological image respectively to obtain a first contour image and a second contour image. Binarization processing refers to setting the grayscale value of the pixel points in the image to 0 or 1 to make the entire image appear to be only black and white. For example, the outline of the target object in the image is set to white, and the area other than the target object is set to black, or the outline of the target object in the image is set to black, and the area other than the target object is set to white. For example, see Figure 4 The first pathological image 401 and the first contour image 402 are shown. The first pathological image 401 is a pathological image of the lesion tissue, and the white area in the first contour image 402 is the contour of the lesion tissue. Figure 5 , for the above Figure 3 As shown in the first pathological image 301 and the second pathological image 302 , the first pathological image 301 is subjected to contour segmentation to obtain a first contour image 501 , and the second pathological image 302 is subjected to contour segmentation to obtain a second contour image 502 .

[0101] In one possible implementation, the computer device sets the grayscale value of a pixel in an image to 1 when the grayscale value of the pixel is greater than a first threshold, and sets the grayscale value of the pixel to 0 when the grayscale value of the pixel is not greater than the first threshold; or sets the grayscale value of the pixel in the image to 0 when the grayscale value of the pixel is greater than a second threshold, and sets the grayscale value of the pixel to 1 when the grayscale value of the pixel is not greater than the second threshold, thereby achieving binarization of the image. Wherein, either the first threshold and the second threshold are positive integers greater than 0 and less than 255, the first threshold and the second threshold are the same, or the first threshold and the second threshold are different. The first pathological image and the second pathological image are binarized using the same standard.

[0102] To improve the accuracy of the contour image, in another possible implementation, the computer device invokes a contour segmentation model to perform binarization processing on the first pathological image and the second pathological image, respectively, to obtain a first contour image and a second contour image. The contour segmentation model is used to identify the contour of the target object in the image and perform contour segmentation on the image. The contour segmentation model may be trained by the computer device or sent to the computer device by another computer device, which is not limited in this embodiment of the present application.

[0103] Optionally, in order to be able to perform contour segmentation on the image more accurately, the image input to the contour segmentation model is preprocessed by random flipping, random cropping, color perturbation, etc. to achieve image enhancement, and then the preprocessed image is contour segmented to obtain the corresponding contour image.

[0104] During the training process for contour segmentation images, a computer device receives a first sample image, manually annotates the contours of the first sample image, and obtains a sample contour image corresponding to the first sample image. A contour segmentation model is then trained based on the first sample image and the sample contour image. Specifically, the first sample image is input into the contour segmentation model, which outputs a predicted contour image corresponding to the first sample image. The sample contour image and the predicted contour image are compared, and the parameters of the contour segmentation model are adjusted based on the difference between the sample contour image and the predicted contour image. For example, the learning rate during the contour segmentation model training process is 0.0001, and the maximum number of iterations for each sample image is 200.

[0105] Optionally, the size of the image that the contour segmentation model can process is fixed, for example, the size of the input image is 512*512.

[0106] In the embodiment of the present application, compared with the contour image obtained by binarization processing, the contour image obtained by grayscale processing has a clearer contour of the target object and a better effect.

[0107] 203. Perform contour registration on the first contour image and the second contour image to obtain first registration parameters.

[0108] In an embodiment of the present application, after the computer device obtains the first contour image and the second contour image, it can, based on the first contour image and the second contour image, perform overall registration on the contours of the target objects in the first contour image and the second contour image to obtain first registration parameters. The first registration parameters are parameters used to adjust the second pathological image with reference to the contour of the target object in the first pathological image, that is, parameters used to adjust the second contour image with reference to the contour of the target object in the first contour image.

[0109] In one possible implementation, the first registration parameter includes at least one of a translation parameter, a rotation parameter, or a scaling parameter. If the height of the target object's outline in the first and second contour images differs, the computer device translates the second contour image using the contour of the target object in the first contour image as a reference so that the target object in the second contour image is at the same height as the target object in the first contour image, and determines the distance of the translation of the second contour image as the translation parameter. If the angle of the target object's outline in the first and second contour images differs, the computer device rotates the second contour image using the contour of the target object in the first contour image as a reference so that the target object in the second contour image is at the same angle as the target object in the first contour image, and determines the angle of rotation of the second contour image as the rotation parameter. If the size of the target object's outline in the first and second contour images differs, the computer device scales the second contour image using the contour of the target object in the first contour image as a reference so that the target object in the second contour image is the same size as the target object in the first contour image, and determines the size of the scaled second contour image as the scaling parameter. The translation parameter indicates the direction and distance of movement, the rotation parameter indicates the angle of rotation, and the scaling parameter indicates the ratio of image enlargement or reduction.

[0110] For example, similarity registration, affine registration, and other registration modes are used to perform contour registration on the first pathological image and the second pathological image.

[0111] It should be noted that, in the above embodiment, only the example of first acquiring the first contour image and the second contour image and then performing contour registration on the first contour image and the second contour image is used for illustration. In another embodiment, the contour registration of the first pathological image and the second pathological image can be performed directly based on the contours of the target objects in the first pathological image and the second pathological image, without the need to acquire the first contour image and the second contour image.

[0112] Accordingly, in one possible implementation, when the heights of the contours of the target object in the first pathological image and the second pathological image are different, the computer device translates the second pathological image with reference to the contour of the target object in the first pathological image so that the target object in the second pathological image is at the same height as the target object in the first pathological image, and determines the distance of the translation of the second pathological image as a translation parameter; when the angles of the contours of the target object in the first pathological image and the second pathological image are different, the computer device rotates the second pathological image with reference to the contour of the target object in the first pathological image so that the target object in the second pathological image is at the same angle as the target object in the first pathological image, and determines the angle of rotation of the second pathological image as a rotation parameter; when the sizes of the contours of the target object in the first pathological image and the second pathological image are different, the computer device scales the second pathological image with reference to the contour of the target object in the first pathological image so that the target object in the second pathological image is the same size as the target object in the first pathological image, and determines the size of the scaled second pathological image as a scaling parameter.

[0113] 204. Adjust the second pathological image based on the first registration parameter to obtain a third pathological image.

[0114] In this embodiment of the present application, after obtaining the first registration parameters, the computer device can perform overall registration on the second pathological image based on the first registration parameters to obtain a third pathological image. The third pathological image has a contour consistent with that of the target object in the first pathological image. Regional registration is then performed based on the third pathological image.

[0115] Among them, adjusting the second pathological image based on the first registration parameter means adjusting the second pathological image according to at least one parameter among a translation parameter, a rotation parameter or a scaling parameter, for example, moving the second pathological image left and right or up and down according to the translation parameter, or rotating the second pathological image by a certain angle according to the rotation parameter, or enlarging or reducing the second pathological image by a certain proportion according to the scaling parameter.

[0116] For example, see Figure 6 , taking the first pathological image 601 as a reference, adjusting the second pathological image based on the first registration parameter to obtain a third pathological image 602, Figure 6 The area framed in the first pathological image 601 and the third pathological image 602 is the lesion area, and the regional image 603 is obtained by overlapping the area framed in the first pathological image 601 and the area framed in the third pathological image 602. It can be seen from the regional image 603 that the positions of the lesion areas are not completely overlapped.

[0117] For example, see Figure 7 , Figure 7This is another form of the first and second pathological images. The first and second pathological images are first processed to obtain a black-and-white first pathological image 701 and a second pathological image 702. The lesion area in the first pathological image 701 and the lesion area in the second pathological image 702 are overlapped to obtain a regional image 703. It can be seen from regional image 703 that the positions of the lesion areas do not completely overlap.

[0118] 205. Perform target site registration on the first pathological image and the third pathological image to obtain a second registration parameter.

[0119] The target site is a site in a target object that is in a target state. The target state includes either a normal state or an abnormal state. A normal state refers to a state that has not changed, while an abnormal state refers to a state that has changed. For example, if the target object is a lesion tissue, the normal state refers to the state of the area in the lesion tissue that has not developed a lesion, and the abnormal state refers to the state of the area in the lesion tissue that has developed a lesion. For example, the state of the lesion area in the lesion tissue is an abnormal state.

[0120] In one possible implementation, it is possible to determine the state of the target part to be regionally registered based on the actual application scenario. For example, if the purpose of obtaining the registered image of the second pathological image is to better observe the part in the second pathological image that is in an abnormal state, then it is necessary to more accurately register the region where the part in the second pathological image that is in an abnormal state is located. In this case, the target part is the part in the abnormal state. Alternatively, if the purpose of obtaining the registered image of the second pathological image is to better observe the part in the second pathological image that is in a normal state, then it is necessary to more accurately register the region where the part in the second pathological image that is in a normal state is located. In this case, the target part is the part in the normal state.

[0121] In one possible implementation, a computer device first segments a region image corresponding to the target site from the first and third pathological images. Specifically, the computer device performs region segmentation on the first and third pathological images to obtain a first region image and a second region image. The first region image is an image of the region in the first pathological image where the target site is located, and the second region image is an image of the region in the third pathological image where the target site is located. The computer device then performs registration of the target site on the first and second region images to obtain second registration parameters.

[0122] Optionally, when the first region image includes multiple sub-regions and the second region image includes multiple sub-regions, the first region image and the second region image are further divided into multiple sub-regions, and target region registration is performed on the multiple sub-regions. That is, the computer device performs target region registration on the first region image and the second region image to obtain second registration parameters, including: using the positions of the multiple sub-regions in the first region image as a reference, moving the positions of the corresponding sub-regions in the second region image so that the multiple sub-regions in the second region image are respectively positioned the same as the multiple sub-regions in the first region image, and determining the second registration parameters based on the movement distances of the multiple sub-regions in the second region image. The multiple sub-regions in the first region image correspond one-to-one with the multiple sub-regions in the second region image, and the multiple sub-regions in the second region image are respectively positioned the same as the multiple sub-regions in the first region image, means that a sub-region in the first region image is positioned the same as a corresponding sub-region in the second region image.

[0123] For example, the area corresponding to a pixel point in the first area image and the second area image is taken as a sub-area to obtain multiple sub-areas, and the multiple sub-areas in the first area image and the multiple sub-areas in the second area image are registered using the transformation field registration mode; or, the area corresponding to multiple adjacent pixel points is taken as a sub-area to obtain multiple sub-areas, and the multiple sub-areas in the first area image and the multiple sub-areas in the second area image are registered using the BSpline (non-rigid registration using B-spline) registration mode.

[0124] Optionally, a region segmentation model is called to perform region segmentation on the first pathological image and the third pathological image, respectively, to obtain a first region image and a second region image. The region segmentation model is used to segment an image of the region where the part in the target state is located from the image. The region segmentation model is used to identify a target part in the target state of a target object in the image, and to segment the region where the target part is located from the image. For example, the region segmentation model is a lesion region segmentation model, which is used to segment the lesion region in the image. The region segmentation model is trained by a computer device, or is sent to the computer device by other computer devices, and this embodiment of the present application does not impose any restrictions on this.

[0125] Optionally, in order to be able to perform region segmentation on the image more accurately, the image input to the region segmentation model is preprocessed by random flipping, random cropping, color perturbation, etc. to achieve image enhancement, and then the preprocessed image is region segmented to obtain the corresponding region image.

[0126] In the training process of the region segmentation image, the computer device obtains the second sample image, and performs region annotation on the second sample image by manual annotation to obtain a sample region image corresponding to the second sample image, for example, see Figure 8 The second sample image 801 and the sample region image 802 are shown. A region segmentation model is trained based on the second sample image and the sample region image. Specifically, the second sample image is input into the region segmentation model, which outputs a predicted region image corresponding to the second sample image. The sample region image and the predicted region image are compared, and the parameters of the region segmentation model are adjusted based on the difference between the sample region image and the predicted region image. For example, the learning rate during the region segmentation model training process is 0.0001, and the maximum number of iterations for each sample image is 200.

[0127] Optionally, the size of the image that the region segmentation model can process is fixed, for example, the size of the input image is 512*512.

[0128] It should be noted that the embodiment of the present application is only described by first acquiring a third pathological image and then performing target part registration on the first and third pathological images. In another embodiment, the computer device does not acquire a third pathological image, but directly performs target part registration on the first and second pathological images based on the target part in the target object to obtain the second registration parameters. The difference is that when the target part registration is performed on the first and second pathological images directly based on the target part in the target object, the area where the target part is located is the area before the contour adjustment is performed. Therefore, it is necessary to subsequently adjust the second pathological image based on both the first and second registration parameters so that the contour of the target object and the area where the target part is located are adjusted at the same time, avoiding first adjusting the contour of the target object based on the first registration parameters, resulting in changes in the area where the target part is located, and then failing to accurately adjust based on the second registration parameters.

[0129] 206. Adjust the second pathological image based on the first registration parameter and the second registration parameter to obtain a registered image of the second pathological image.

[0130] The registered image and the first pathological image have the same outline of the target object, and the position of the target part is consistent.

[0131] In one possible implementation, a computer device combines the first and second registration parameters to obtain third registration parameters. Based on the third registration parameters, the second pathology image is adjusted to obtain a registered image of the second pathology image. Specifically, the computer device simultaneously adjusts the position of the target object and the target region in the second pathology image, thereby reducing the number of intermediate steps required to obtain the registered image from the second pathology image. The third registration parameters are obtained by concatenating the first and second registration parameters, facilitating adjustment of the second pathology image based on the third registration parameters.

[0132] In another possible implementation, the computer device adjusts the second pathology image based on the first registration parameters to obtain a fourth pathology image. The fourth pathology image is consistent with the outline of the target object in the first pathology image. The computer device then adjusts the fourth pathology image based on the second registration parameters to obtain a registered image of the second pathology image. Specifically, the computer device first adjusts the position of the target object and then adjusts the position of the region where the target site is located. The fourth pathology image is identical to the third pathology image.

[0133] For example, see Figure 9 As shown in the registration result, the area framed in the first pathological image 901 and the registration image 902 is the lesion area, and the regional image 903 is obtained by overlapping the area framed in the first pathological image 901 with the area framed in the registration image 902. It can be seen from the regional image 903 that the positions of the lesion areas are almost completely overlapped.

[0134] For example, see Figure 10 , for the first pathological image 1001 and the second pathological image 1002, after registering the first pathological image 1001 and the second pathological image 1002, the first pathological image 1001 and the registered image 1003 are obtained. That is, the first pathological image 1001 remains unchanged, and the second pathological image 1002 is adjusted to obtain the corresponding registered image 1003.

[0135] For example, see Figure 11 , the registration result of another lesion tissue. Taking the first pathological image 1101 as a reference, the second pathological image 1102 is adjusted to obtain a registered image 1103.

[0136] It should be noted that the pathological image is a color image, and the drawings in the embodiments of the present application are only used as examples for illustration.

[0137] An embodiment of the present application provides an image registration method, which first performs overall registration on the first pathological image and the second pathological image based on the contours of the target objects in the first pathological image and the second pathological image, and then performs local precise registration on the first pathological image and the second pathological image based on the target parts of the target objects in the first pathological image and the second pathological image. This method eliminates the need to extract every feature point in the first pathological image and the second pathological image, reduces the difficulty of registration, improves the efficiency of registration, and avoids inaccurate registered images resulting from inaccurate feature points, thereby improving the accuracy of image registration.

[0138] In addition, the pathological image is binarized or grayscaled to obtain a corresponding contour image. The contour image can clearly display the contour of the target object without any other interference. Therefore, the first contour image and the second contour image are aligned to avoid interference from other factors in the first pathological image and the second pathological image except the contour of the target object, thereby improving the accuracy of the alignment and making the obtained first alignment parameter more accurate.

[0139] Moreover, in the process of local registration, the regional image is further divided into more sub-regions, and the registered image is obtained by adjusting the position of each sub-region. Compared with adjusting the entire regional image, the accuracy of registration is improved. Compared with the feature point-based registration in related technologies, dividing the sub-regions is simpler and reduces the difficulty of registration. Therefore, the local registration in this application reduces the difficulty of registration while ensuring the accuracy of registration.

[0140] Moreover, for pathological images, since pathological images are obtained by scanning stained pathological sections, and the staining styles of different pathological sections vary greatly, the color pathological images are quite different. The method of extracting feature points in related technologies cannot extract effective feature points. However, the registration method of first overall registration and then local precise registration in this application does not require feature point extraction, reduces the difficulty of registration, improves the registration efficiency, and avoids the problem of inaccurate registered images caused by inaccurate feature points, thereby improving the accuracy of image registration.

[0141] In addition, in a possible implementation, during the image registration process, the contour segmentation model and the region segmentation model are called to segment the image. Figure 12 . Figure 12 This is a flow chart of another image registration method provided by an embodiment of the present application. The execution subject of the embodiment of the present application is a computer device, and the method includes the following steps:

[0142] 1201. Obtain a first digital slice and a second digital slice.

[0143] 1202. Obtain a thumbnail of the first digital slice and a thumbnail of the second digital slice.

[0144] 1203. Call a contour segmentation model to perform contour segmentation on the thumbnail of the first digital slice and the thumbnail of the second digital slice to obtain a first contour image corresponding to the first digital slice and a second contour image corresponding to the second digital slice.

[0145] Before this, it is necessary to train a contour segmentation model, that is, obtain digital slices and annotated contour images, train the contour segmentation model based on the digital slices and annotated contour images, and obtain a trained contour segmentation model.

[0146] 1204. Perform global registration on the first contour image and the second contour image to obtain a first registration parameter and a third digital slice corresponding to the second digital slice.

[0147] 1205 : Call a region segmentation model to perform region segmentation on the thumbnail of the first digital slice and the third digital slice to obtain a first region image corresponding to the first digital slice and a second region image corresponding to the third digital slice.

[0148] Previously, it was necessary to train a regional segmentation model, that is, obtain digital slices and regional images corresponding to the annotated lesion areas, and train the regional segmentation model based on the digital slices and the annotated regional images to obtain a trained regional segmentation model.

[0149] 1206. Perform local registration on the first region image and the second region image to obtain second registration parameters.

[0150] 1207. Adjust the second digital slice based on the first registration parameter and the second registration parameter to obtain a registered image of the second digital slice.

[0151] The implementation of steps 1201 to 1207 is similar to the implementation of steps 201 to 206 above, and will not be repeated here.

[0152] In addition, in one possible implementation, the pathological image is registered using the method provided in the embodiment of the present application. After obtaining the registered image, the registered image can be used as a basis to assist doctors in determining the patient's physical condition, enabling doctors to determine the patient's diagnosis results by combining the registered image, other relevant information or the patient's physical condition.

[0153] above Figure 2 and Figure 12 While this description uses a medical setting as an example, the image registration method provided in the embodiments of this application can also be applied to image preprocessing scenarios in machine learning. The method provided in the embodiments of this application is used to register the original image, and the registered image is used for subsequent image recognition, image segmentation, and other processing. It can also be applied in other scenarios, and the embodiments of this application do not limit this.

[0154] Figure 13 This is a schematic diagram of the structure of an image registration device provided in an embodiment of the present application. Figure 13 , the device comprises:

[0155] An image acquisition module 1301 is configured to acquire a first registered image and a second registered image, where the first registered image and the second registered image are pathological images of the same target object;

[0156] A first registration module 1302 is configured to perform contour registration on the first pathological image and the second pathological image to obtain a first registration parameter, where the first registration parameter is a parameter used to adjust the contour of the target object in the second pathological image;

[0157] A second registration module 1303 is configured to register a target portion of the first pathological image and the second pathological image to obtain a second registration parameter, where the target portion is a portion of the target object in a target state, and the second registration parameter is a parameter used to adjust the position of the target portion in the second pathological image;

[0158] The registration image acquisition module 1304 is used to adjust the second registration image based on the first registration parameter and the second registration parameter to obtain a registration image of the second registration image, where the registration image is consistent with the contour of the target object in the first pathological image and the position of the target part is consistent.

[0159] An embodiment of the present application provides an image registration device, which first performs overall registration on the first registration image and the second registration image based on the contours of the target objects in the first registration image and the second registration image, and then performs local precise registration on the first registration image and the second registration image based on the target parts of the target objects in the first registration image and the second registration image. There is no need to extract each feature point in the first registration image and the second registration image, which reduces the difficulty of registration, improves the registration efficiency, and avoids inaccurate registration images caused by inaccurate feature points, thereby improving the accuracy of image registration.

[0160] In one possible implementation, see Figure 14 The first registration module 1302 includes:

[0161] A contour image acquisition unit 1312 is configured to acquire a first contour image of the first registered image and a second contour image of the second registered image;

[0162] The first registration unit 1322 is configured to perform contour registration on the first contour image and the second contour image to obtain a first registration parameter.

[0163] In another possible implementation, see Figure 14 , the contour image acquisition unit 1312 is used to:

[0164] Performing binarization processing on the first registered image and the second registered image respectively to obtain a first contour image and a second contour image; or,

[0165] Grayscale processing is performed on the first registered image and the second registered image respectively to obtain a first contour image and a second contour image.

[0166] In another possible implementation, the second registration module 1303 is configured to:

[0167] Adjusting the second registered image based on the first registration parameter to obtain a third pathological image, wherein the third pathological image is consistent with the contour of the target object in the first registered image;

[0168] The first registration image and the third pathological image are registered at the target site to obtain second registration parameters.

[0169] In another possible implementation, see Figure 14 The second registration module 1303 includes:

[0170] A region segmentation unit 1313 is configured to perform region segmentation on the first registered image and the third pathological image, respectively, to obtain a first region image and a second region image, wherein the first region image is an image of the region where the target part is located in the first registered image, and the second region image is an image of the region where the target part is located in the third pathological image;

[0171] The second registration unit 1323 is configured to perform registration of the target part on the first region image and the second region image to obtain a second registration parameter.

[0172] In another possible implementation, see Figure 14 The region segmentation unit 1313 is used to call the region segmentation model to perform region segmentation on the first registration image and the third pathological image respectively to obtain a first region image and a second region image. The region segmentation model is used to segment the image of the region where the part in the target state is located from the image.

[0173] In another possible implementation, the first region image includes multiple sub-regions, and the second region image includes multiple sub-regions, see Figure 14 The second registration unit 1323 is configured to:

[0174] Using the positions of the plurality of sub-regions in the first region image as a reference, moving the positions of the corresponding sub-regions in the second region image so that the positions of the plurality of sub-regions in the second region image are respectively the same as the positions of the plurality of sub-regions in the first region image;

[0175] A second registration parameter is determined based on the movement distances of the plurality of sub-regions in the second region image.

[0176] In another possible implementation, the registration image acquisition module 1304 is configured to:

[0177] Combining the first registration parameter and the second registration parameter to obtain a third registration parameter;

[0178] The second registered image is adjusted based on the third registration parameter to obtain a registered image of the second registered image.

[0179] In another possible implementation, the registration image acquisition module 1304 is configured to:

[0180] Adjusting the second registered image based on the first registration parameter to obtain a fourth pathological image, wherein the fourth pathological image is consistent with the contour of the target object in the first registered image;

[0181] Based on the second registration parameter, the fourth pathological image is adjusted to obtain a registered image of the second registered image.

[0182] In another possible implementation, the first registration parameter includes at least one of a translation parameter, a rotation parameter, or a scaling parameter. The first registration module 1302 is configured to:

[0183] Using the outline of the target object in the first registered image as a reference, translate the second registered image so that the target object in the second registered image is at the same height as the target object in the first registered image, and determine the translation distance of the second registered image as the translation parameter; or

[0184] Using the outline of the target object in the first registered image as a reference, the second registered image is rotated so that the target object in the second registered image is at the same angle as that in the first registered image, and the angle of rotation of the second registered image is determined as a rotation parameter; or

[0185] The second registered image is scaled with reference to the outline of the target object in the first registered image so that the target object in the second registered image has the same size as that in the first registered image, and the scaled size of the second registered image is determined as a scaling parameter.

[0186] In another possible implementation, the target object is lesion tissue, and the image acquisition module 1301 is configured to downsample the fifth pathological image and the sixth pathological image of the lesion tissue respectively to obtain a first registered image and a second registered image.

[0187] All of the above optional technical solutions can be combined in any way to form optional embodiments of the present application, and will not be described in detail here.

[0188] It should be noted that the image registration device provided in the above embodiment only uses the division of the above functional modules as an example to illustrate image registration. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the image registration device provided in the above embodiment and the image registration method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0189] An embodiment of the present application further provides a computer device comprising a processor and a memory, wherein the memory stores at least one computer program, which is loaded and executed by the processor to implement the operations performed by the image registration method of the above embodiment.

[0190] Optionally, the computer device is provided as a terminal. Figure 15 FIG1 is a schematic diagram of the structure of a terminal 1500 provided in an embodiment of the present application. The terminal 1500 includes: a processor 1501 and a memory 1502.

[0191] The processor 1501 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1501 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 1501 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1501 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1501 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0192] The memory 1502 may include one or more computer-readable storage media, which may be non-transitory. The memory 1502 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1502 is used to store at least one computer program, which is executed by the processor 1501 to implement the image registration method provided in the method embodiment of the present application.

[0193] In some embodiments, terminal 1500 may optionally include a peripheral device interface 1503 and at least one peripheral device. The processor 1501, memory 1502, and peripheral device interface 1503 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 1503 via a bus, signal lines, or circuit boards. Specifically, the peripheral device may include at least one of a display screen 1504, a camera assembly 1505, and a power supply 1506.

[0194] The peripheral device interface 1503 can be used to connect at least one I / O (Input / Output)-related peripheral device to the processor 1501 and the memory 1502. In some embodiments, the processor 1501, the memory 1502, and the peripheral device interface 1503 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1501, the memory 1502, and the peripheral device interface 1503 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0195] The display screen 1504 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 1504 is a touch screen display, the display screen 1504 also has the ability to collect touch signals on the surface or above the surface of the display screen 1504. The touch signal can be input as a control signal to the processor 1501 for processing. At this time, the display screen 1504 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, there can be one display screen 1504, which is set on the front panel of the terminal 1500; in other embodiments, there can be at least two display screens 1504, which are respectively set on different surfaces of the terminal 1500 or in a folding design; in other embodiments, the display screen 1504 can be a flexible display screen, which is set on the curved surface or folding surface of the terminal 1500. Even more, the display screen 1504 can be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 1504 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0196] The camera assembly 1505 is used to capture images or videos. Optionally, the camera assembly 1505 includes a front camera and a rear camera. The front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 1505 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. The dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0197] Power supply 1506 is used to power various components in terminal 1500. Power supply 1506 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1506 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is charged via a wired line, while a wireless rechargeable battery is charged via a wireless coil. The rechargeable battery can also support fast charging technology.

[0198] Those skilled in the art will understand that Figure 15 The structure shown in the figure does not constitute a limitation on the terminal 1500, and the terminal 1500 may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0199] Optionally, the computer device is provided as a server. Figure 16 1 is a schematic diagram of the structure of a server provided in an embodiment of the present application. The server 1600 may vary significantly due to different configurations or performances, and may include one or more processors (Central Processing Units, CPUs) 1601 and one or more memories 1602. The memories 1602 store at least one computer program, which is loaded and executed by the processor 1601 to implement the methods provided in the above-mentioned various method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and input / output interfaces for input and output. The server may also include other components for implementing device functions, which will not be described in detail here.

[0200] An embodiment of the present application further provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement the operations performed by the image registration method of the above embodiment.

[0201] An embodiment of the present application further provides a computer program product, including a computer program, which, when executed by a processor, implements the operations performed by the image registration method of the above embodiment.

[0202] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0203] The above are only optional embodiments of the embodiments of the present application and are not intended to limit the embodiments of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of this application.

Claims

1. An image registration method, characterized in that: The method comprises: Acquiring a first pathological image and a second pathological image, where the first pathological image and the second pathological image are pathological images of the same target object; Performing contour registration on the first pathological image and the second pathological image to obtain a first registration parameter, where the first registration parameter is a parameter used to adjust the contour of the target object in the second pathological image; adjusting the second pathological image based on the first registration parameter to obtain a third pathological image, wherein the third pathological image is consistent with the contour of the target object in the first pathological image; performing regional segmentation on the first pathological image and the third pathological image respectively to obtain a first regional image and a second regional image, where the first regional image is an image of a region where a target part is located in the first pathological image, and the second regional image is an image of a region where the target part is located in the third pathological image; performing registration of the target part on the first regional image and the second regional image to obtain a second registration parameter, where the target part is a part of the target object in a target state, and the second registration parameter is a parameter used to adjust the position of the target part in the second pathological image; Adjusting the second pathological image based on the first registration parameter and the second registration parameter to obtain a registered image of the second pathological image, wherein the registered image is consistent with the contour of the target object in the first pathological image and the position of the target part is consistent; The registering the first region image and the second region image for the target part to obtain the second registration parameter includes: using the positions of the multiple subregions in the first region image as a reference, moving the positions of the corresponding subregions in the second region image so that the positions of the multiple subregions in the second region image are respectively the same as the positions of the multiple subregions in the first region image; and determining the second registration parameter based on the movement distance of the multiple subregions in the second region image.

2. The method according to claim 1, characterized in that The performing contour registration on the first pathological image and the second pathological image to obtain a first registration parameter includes: acquiring a first contour image of the first pathological image and a second contour image of the second pathological image; Perform contour registration on the first contour image and the second contour image to obtain the first registration parameter.

3. The method according to claim 2, characterized in that The acquiring a first contour image of the first pathological image and a second contour image of the second pathological image includes: performing binarization processing on the first pathological image and the second pathological image respectively to obtain the first contour image and the second contour image; or, Grayscale processing is performed on the first pathological image and the second pathological image respectively to obtain the first contour image and the second contour image.

4. The method according to claim 1, wherein The performing region segmentation on the first pathological image and the third pathological image to obtain a first region image and a second region image respectively includes: A region segmentation model is called to perform region segmentation on the first pathological image and the third pathological image respectively to obtain the first region image and the second region image. The region segmentation model is used to segment the image of the region where the part in the target state is located from the image.

5. The method according to claim 1, wherein The adjusting the second pathological image based on the first registration parameter and the second registration parameter to obtain a registered image of the second pathological image includes: combining the first registration parameter and the second registration parameter to obtain a third registration parameter; Based on the third registration parameter, the second pathological image is adjusted to obtain a registered image of the second pathological image.

6. The method according to claim 1, characterized in that The adjusting the second pathological image based on the first registration parameter and the second registration parameter to obtain a registered image of the second pathological image includes: adjusting the second pathological image based on the first registration parameter to obtain a fourth pathological image, wherein the fourth pathological image is consistent with the contour of the target object in the first pathological image; Based on the second registration parameter, the fourth pathological image is adjusted to obtain a registered image of the second pathological image.

7. The method according to any one of claims 1 to 6, characterized in that The first registration parameter includes at least one of a translation parameter, a rotation parameter, or a scaling parameter. The contour registration of the first pathological image and the second pathological image to obtain the first registration parameter includes: Using the outline of the target object in the first pathological image as a reference, translating the second pathological image so that the target object in the second pathological image is at the same height as the target object in the first pathological image, and determining the translation distance of the second pathological image as the translation parameter; or Using the outline of the target object in the first pathological image as a reference, rotating the second pathological image so that the target object in the second pathological image is at the same angle as the target object in the first pathological image, and determining the angle of rotation of the second pathological image as the rotation parameter; or The second pathological image is scaled with reference to the outline of the target object in the first pathological image so that the target object in the second pathological image is the same size as the target object in the first pathological image, and the scaled size of the second pathological image is determined as the scaling parameter.

8. The method according to any one of claims 1 to 6, characterized in that The target object is a lesion tissue, and acquiring the first pathological image and the second pathological image includes: The fifth pathological image and the sixth pathological image of the lesion tissue are downsampled respectively to obtain the first pathological image and the second pathological image.

9. An image registration device, characterized in that: The device comprises: An image acquisition module, configured to acquire a first pathological image and a second pathological image, wherein the first pathological image and the second pathological image are pathological images of the same target object; a first registration module, configured to perform contour registration on the first pathological image and the second pathological image to obtain a first registration parameter, wherein the first registration parameter is a parameter for adjusting the contour of the target object in the second pathological image; a second registration module, configured to adjust the second pathological image based on the first registration parameter to obtain a third pathological image, wherein the third pathological image is consistent with the contour of the target object in the first pathological image; and perform target part registration on the first pathological image and the third pathological image to obtain a second registration parameter, wherein the target part is a part of the target object in a target state, and the second registration parameter is a parameter used to adjust the position of the target part in the second pathological image; a registration image acquisition module, configured to adjust the second pathological image based on the first registration parameter and the second registration parameter to obtain a registered image of the second pathological image, wherein the registered image is consistent with the contour of the target object in the first pathological image and the position of the target part is consistent; The second registration module includes: a region segmentation unit, configured to perform region segmentation on the first pathological image and the third pathological image, respectively, to obtain a first region image and a second region image, wherein the first region image is an image of the region where the target part is located in the first pathological image, and the second region image is an image of the region where the target part is located in the third pathological image; a second registration unit, configured to perform registration of the target part on the first region image and the second region image to obtain the second registration parameter; The second registration unit is configured to: move the positions of the corresponding sub-regions in the second region image with reference to the positions of the multiple sub-regions in the first region image, so that the positions of the multiple sub-regions in the second region image are respectively the same as the positions of the multiple sub-regions in the first region image; and determine the second registration parameters based on the movement distances of the multiple sub-regions in the second region image.

10. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the operations performed by the image registration method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to implement the operations performed by the image registration method according to any one of claims 1 to 8.

12. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the operations performed by the image registration method according to any one of claims 1 to 8 are implemented.

Images