Control point determination, image registration method, device and computer equipment

Abstract

The application relates to a control point determination method and device and a computer device. A plurality of first image blocks are obtained by segmenting a target gradient image, the number of control points to be selected in each image block is determined according to a first preset control point number and pixel values of the image blocks, and then the target control points are determined from the target gradient image according to the image block information corresponding to each image block. The target gradient image is a gradient image of a target image of a subject, and the target image includes any one of a mask image, a gain image and a subtraction image. The image block information includes the number of control points to be selected in the corresponding image block and / or the size of the image block. The target control points are selected on the target gradient image, the feature information of the image is fully utilized, the first preset control point number is distributed according to the pixel values of the image blocks, the target control points are distributed in the area with rich edge information and large artifacts, and the accuracy of the control point selection is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a control point determination, image registration method, apparatus, and computer equipment. Background Technology

[0002] Digital subtraction angiography (DSA) has become an irreplaceable vascular visualization tool in clinical cardiovascular diagnosis and treatment due to its high resolution and contrast. Typically, X-rays are first taken sequentially in the area of ​​interest to the patient. Then, a contrast agent is injected. A frame before injection is used as a mask. A real-time filling image after injection is obtained, called the filling image. Finally, the mask is subtracted from the filling image to obtain the subtracted image, which contains only the blood vessels.

[0003] However, since the mask and the embolized image are acquired at different times, unavoidable factors such as patient movement, breathing, heartbeat, and visceral peristalsis occur during this period. This makes it impossible for the corresponding pixels of the mask and the embolized image to be precisely aligned, resulting in a large number of motion artifacts in the subtraction angiography image, which reduces the diagnostic value of DSA. Therefore, how to accurately obtain the control point and improve the image registration of DSA based on the control point has become an urgent problem to be solved. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, and computer equipment for determining control points and registering images that can improve the accuracy of control point determination and image registration, in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides a method for determining control points, the method comprising:

[0006] The target gradient image is segmented to obtain multiple first image blocks; the target gradient image is the gradient image of the target image of the inspected object, and the target image includes any one of the following: mask image, full image, and subtraction image;

[0007] Based on the first preset number of control points and the pixel value of each image block, determine the number of control points to be selected in each image block;

[0008] Based on the image block information corresponding to each image block, target control points are determined from the target gradient image; the image block information includes the number of control points to be selected in the corresponding image block and / or the size of the image block.

[0009] In one embodiment, determining the target control points from the target gradient image based on the image block information corresponding to each of the image blocks includes:

[0010] The image block corresponding to the image block information that meets the preset conditions is taken as the first target image block;

[0011] Perform the first iteration operation: segment the first target image block to obtain multiple new image blocks for the current time, and determine the number of control points to be selected in each new image block according to the number of control points corresponding to the first target image block and the pixel value of each new image block. Then, take the new image block corresponding to the image block information that meets the preset condition as the first target image block, and return to perform the first iteration operation until the image block information corresponding to each new image block obtained in the current time does not meet the preset condition.

[0012] The target control point is determined from each of the second target image blocks based on the pixel value of each pixel in the second target image block; the second target image block includes the image block corresponding to the image block information that does not meet the preset conditions in each determination.

[0013] In one embodiment, determining the target control points from the target gradient image based on the image block information corresponding to each of the image blocks includes:

[0014] Determine whether each image block meets the preset conditions based on the image block information of each image block;

[0015] If the image block information of each image block does not meet the preset conditions, then the pixel corresponding to the largest pixel value in each image block is taken as the target control point.

[0016] In one embodiment, the step of using the image block corresponding to the image block information that satisfies the preset conditions as the first target image block includes:

[0017] If the number of control points corresponding to the image block is greater than the second preset number of control points, and / or the size of the image block is greater than the preset size, then the image block information corresponding to the image block is determined to meet the preset conditions, and the image block corresponding to the image block information that meets the preset conditions is taken as the first target image block.

[0018] In one embodiment, determining the number of control points to be selected in each image block based on the first preset number of control points and the pixel values ​​of each image block includes:

[0019] The weight of each image block in the target gradient image is determined based on the pixel value of each image block;

[0020] Based on the first preset number of control points and each of the weights, the number of control points to be selected in each of the image blocks is determined.

[0021] Secondly, this application also provides an image registration method, the method comprising:

[0022] Determine the first similarity between the target mask image and the chroma image in the current iteration; the target mask image includes any one of the following: a mask image, an image determined by mapping the target control points on the chroma image to the mask image, and an image determined by mapping the target control points on the subtraction image to the mask image; the target control points are target control points determined using the method described in any of the above embodiments.

[0023] Perform a second iteration operation; the second iteration operation includes determining a matching control point in the film image corresponding to the target control point based on the target control point on the target mask image, generating a new target mask image for the current iteration based on the control point pair and the target mask image, and determining the second similarity between the new target mask image and the film image for the current iteration; the control point pair includes the target control point and the matching control point;

[0024] Based on the first similarity and the second similarity of the current time, a target subtraction image is determined, wherein the target subtraction image is the image after registration of the film image and the target mask image.

[0025] In one embodiment, determining the target subtraction image based on the first similarity and the second similarity of the current iteration includes:

[0026] If the second similarity is greater than or equal to the first similarity, then the new target mask image is used as the target mask image, and the second similarity is used as the first similarity for the next iteration. The second iteration operation is then performed to determine the second similarity for the next iteration.

[0027] If the second similarity in the next step is less than the first similarity in the next step, then the target subtraction image is determined based on the previous new target mask image and the filled image.

[0028] Thirdly, this application also provides a control point determination device, the device comprising:

[0029] The segmentation module is used to segment the target gradient image to obtain multiple first image blocks; the target gradient image is the gradient image of the target image of the inspected object, and the target image includes any one of the following: mask image, full image, and subtraction image;

[0030] The first determining module is used to determine the number of control points to be selected in each of the image blocks based on the first preset number of control points and the pixel value of each image block;

[0031] The second determining module is used to determine target control points from the target gradient image based on the image block information corresponding to each image block; the image block information includes the number of control points to be selected in the corresponding image block and / or the size of the image block.

[0032] Fourthly, this application also provides an image registration apparatus, the apparatus comprising:

[0033] The third determining module is used to determine the first similarity between the target mask image and the stencil image in the current iteration; the target mask image includes any one of the following: the mask image, the image determined by mapping the target control points on the stencil image to the mask image, and the image determined by mapping the target control points on the subtraction image to the mask image; the target control points are target control points determined using the method described in any of the above embodiments.

[0034] An execution module is used to perform a second iteration operation; the second iteration operation includes determining a matching control point in the film image corresponding to the target control point based on the target control point on the target mask image, generating a new target mask image for the current iteration based on the control point pair and the target mask image, and determining a second similarity between the new target mask image and the film image for the current iteration; the control point pair includes the target control point and the matching control point in the film image.

[0035] The fourth determining module is used to determine the target subtraction image based on the first similarity and the second similarity of the current time, wherein the target subtraction image is the image after registration of the smear image and the target mask image.

[0036] Fifthly, this application also provides a computer device, the computer device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the methods provided in the first and second aspects above.

[0037] Sixthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the methods provided in the first and second aspects described above.

[0038] In a seventh aspect, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the methods provided in the first and second aspects described above.

[0039] The aforementioned control point determination, image registration method, apparatus, and computer equipment segment the target gradient image to obtain multiple first image blocks. Based on the number of preset control points and the pixel values ​​of each image block, the number of control points to be selected in each image block is determined. Thus, target control points are determined from the target gradient image based on the image block information corresponding to each image block. The target gradient image is the gradient image of the target image of the object under inspection, and the target image includes any one of a masked image, a full-screen image, and a subtraction image. The image block information includes the number of control points to be selected in the corresponding image block and / or the size of the image block. This application selects target control points on the target gradient image, fully utilizing the image's feature information. Furthermore, this application allocates the number of preset control points based on the pixel values ​​of each image block, avoiding the uniform distribution in existing technologies. This ensures that the selected target control points are reasonably distributed in areas of the target gradient image with rich edge information and large artifacts, improving the accuracy of target control point selection. Attached Figure Description

[0040] Figure 1 This is a diagram illustrating the application environment of the control point determination method in one embodiment;

[0041] Figure 2 This is a flowchart illustrating a control point determination method in one embodiment;

[0042] Figure 3 This is a flowchart illustrating the process of determining a target control point in one embodiment;

[0043] Figure 4A This is a schematic diagram of the target control points in one embodiment;

[0044] Figure 4B This is a schematic diagram of the target control point in another embodiment;

[0045] Figure 5 This is a flowchart illustrating the process of determining the number of control points to be selected in each image block in one embodiment;

[0046] Figure 6 This is a flowchart illustrating an image registration method in one embodiment;

[0047] Figure 7 This is a flowchart illustrating the image registration method in another embodiment;

[0048] Figure 8 This is a schematic diagram of a first image of a target subtraction image in one embodiment;

[0049] Figure 9A This is a schematic diagram of a second image of the target subtraction image in one embodiment;

[0050] Figure 9BThis is a schematic diagram of a third image of the target subtraction image in one embodiment;

[0051] Figure 9C This is a schematic diagram of the fourth image of the target subtraction image in one embodiment;

[0052] Figure 9D This is a schematic diagram of the fifth image of the target subtraction image in one embodiment;

[0053] Figure 10 This is a structural block diagram of a control point determination device in one embodiment. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] The control point determination method provided in this application embodiment can be applied to, for example, Figure 1 The application environment shown includes a computer device, which may be a server, and its internal structure diagram may be as follows. Figure 1 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores image registration-related data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a control point determination method. The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0056] Those skilled in the art will understand that Figure 1 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0057] In one embodiment, such as Figure 2 As shown, a control point determination method is provided, which can be applied to... Figure 1 Taking a computer device as an example, the explanation includes the following steps:

[0058] S201, the target gradient image is segmented to obtain multiple first image blocks; the target gradient image is the gradient image of the target image of the inspected object, and the target image includes any one of the following: mask image, full image, and subtraction image.

[0059] The subtraction image is obtained by subtracting the mask image from the overlay image. The gradient image is obtained by differentiating the target image using a gradient filter; for example, the Sobel edge detection algorithm, Scharr, and Lapacian algorithms can be used. The gradient image contains a large amount of edge feature information from the target image. For edge regions of the image, the grayscale value changes significantly, and the gradient value is also large; for smoother parts of the image, the grayscale value changes less, and the corresponding gradient value is also small.

[0060] In this embodiment, the target gradient image can be segmented using quadtree decomposition, or by using binary search, ternary search, or other methods to obtain multiple first image blocks.

[0061] S202, based on the first preset number of control points and the pixel value of each image block, determine the number of control points to be selected in each image block.

[0062] Image control points are reference points selected on an image to establish geometric transformation functions; in this case, they are used for image matching. Generally, each control point can contain two sets of coordinate data—the coordinates from the two images being matched—and is therefore also called a control point pair.

[0063] The first preset number of control points is the number of control points preset in the target gradient image.

[0064] In this embodiment, for each image block, the pixel values ​​in the image block can be accumulated to obtain the first pixel sum of the image block. Then, the pixel values ​​of all image blocks are accumulated to obtain the second pixel sum of the target gradient image. The first pixel sum and the second pixel sum are normalized to obtain the pixel weight of each image block. Based on the pixel weight and the first preset number of control points, the number of control points to be selected in each image block is determined.

[0065] In some possible implementations, the number of pixels with significantly different pixel values ​​among adjacent pixels in each image block can be counted. That is, if the pixel value of an adjacent pixel is greater than a preset threshold, the count is incremented by 1 to obtain the number of pixels that meet the criteria in each image block. Based on the final count of pixels that meet the criteria in each image block and the first preset number of control points, the number of control points to be selected in each image block is determined.

[0066] S203, determine the target control points from the target gradient image based on the image block information corresponding to each image block; the image block information includes the number of control points to be selected in the corresponding image block and / or the size of the image block.

[0067] In this embodiment, the target control points are determined from the target gradient image based on the image block information corresponding to each image block. This can be done after the number of control points to be selected in each image block meets certain conditions, or after the size information of each image block meets certain conditions.

[0068] When determining target control points from the target gradient image, the point with the largest pixel value in the image patch can be used as the target control point, or the center point of the image patch can be used as the target control point. Alternatively, feature points extracted from each image patch using algorithms such as Harris corner algorithm, Scale-invariant feature transform (SIFT), Speeded Up Robust Features (SURF), Oriented FAST and Rotated BRIEF (ORB), Features from Accelerated Segment Test (FAST), and Small univalue segment assisting nucleus (SUSAN) can be used as target control points.

[0069] In the aforementioned control point determination method, multiple first image blocks are obtained by segmenting the target gradient image. Based on the first preset number of control points and the pixel values ​​of each image block, the number of control points to be selected in each image block is determined. Thus, target control points are determined from the target gradient image based on the image block information corresponding to each image block. The target gradient image is the gradient image of the target image of the object under inspection, and the target image includes any one of a masked image, a full-coverage image, and a subtraction image. The image block information includes the number of control points to be selected in the corresponding image block and / or the size of the image block. This application selects target control points on the target gradient image, fully utilizing the image's feature information. Furthermore, this application allocates the first preset number of control points based on the pixel values ​​of each image block, avoiding the uniform distribution in the prior art. This ensures that the selected target control points are reasonably distributed in areas of the target gradient image with rich edge information and large artifacts, improving the accuracy of target control point selection.

[0070] Figure 3 This is a flowchart illustrating the process of determining a target control point in one embodiment, such as... Figure 3 As shown, this application embodiment relates to a possible implementation of how to determine target control points from a target gradient image based on the image block information corresponding to each image block, including the following steps:

[0071] S301, the image block corresponding to the image block information that meets the preset conditions is taken as the first target image block.

[0072] Specifically, if the number of control points corresponding to an image block is greater than the second preset number of control points, and / or the size of the image block is greater than a preset size, then the image block information corresponding to the image block is determined to meet the preset conditions, and the image block corresponding to the image block information that meets the preset conditions is taken as the first target image block. For example, if the second preset number of control points is 1 and the preset size is 10*10, the image block information that meets any one of these conditions can be taken as the first target image block.

[0073] S302, perform the first iteration operation: segment the first target image block to obtain multiple new image blocks in the current iteration, and determine the number of control points to be selected in each new image block according to the number of control points corresponding to the first target image block and the pixel value of each new image block, and take the new image block corresponding to the image block information that meets the preset conditions as the first target image block, and return to perform the first iteration operation until the image block information corresponding to each new image block obtained in the current iteration does not meet the preset conditions.

[0074] In this embodiment, the first target image block is segmented using the above segmentation algorithm to obtain multiple new image blocks for the current time. Based on the number of control points corresponding to the first target image block and the pixel value of each new image block, the number of control points to be selected in each new image block is determined. It is then determined whether the image block information of each new image block meets the preset conditions. If the preset conditions are met, the image blocks that meet the preset conditions are segmented until the new image blocks do not meet the preset conditions.

[0075] S303, based on the pixel values ​​of each pixel in the second target image block, determine the target control point from each second target image block; the second target image block includes the image block corresponding to the image block information that does not meet the preset conditions determined in each instance.

[0076] In this embodiment, target control points are determined from image blocks that do not meet preset conditions each time. That is, when a new image block cannot be further segmented, the target control points in that image block are determined based on the pixel values ​​in the corresponding image block when segmentation cannot continue.

[0077] It should be noted that, in each iteration, when determining the target control points in image patches that cannot be further segmented, the same feature point selection method can be used, or different feature point selection algorithms can be used to determine the target control points.

[0078] In this embodiment, the target control points in the target gradient image are determined by multiple iterations based on pixel values. By making full use of the feature information of the target gradient image, the selected target control points are more accurate, thereby enabling the pixels corresponding to the mask image and the film image to be more precisely aligned.

[0079] In one embodiment, the control point determination method further includes: determining whether each image block meets a preset condition based on the image block information of each image block; if the image block information of each image block does not meet the preset condition, then the pixel corresponding to the largest pixel value in each image block is taken as the target control point.

[0080] In this embodiment, if the image block information of each image block does not meet the preset conditions, i.e., each image block cannot be further segmented, the pixel corresponding to the largest pixel value in each image block is taken as the target control point, that is, the local gradient maxima of the target gradient image within that image block is taken as the control point. Since the local gradient maxima reflect the location of the greatest pixel change rate within the image block, such as... Figure 4A and Figure 4B As shown, the control points selected based on the target gradient image are mainly concentrated at the edges, with fewer control points at non-edges. In other words, the control points are reasonably distributed in areas with rich edge information and large artifacts in the target gradient image.

[0081] Figure 5 This is a flowchart illustrating the process of determining the number of control points to be selected in each image block in one embodiment, as shown below. Figure 5 As shown, this application embodiment relates to a possible implementation of how to determine the number of control points to be selected in each image block based on a first preset number of control points and the pixel values ​​of each image block, including the following steps:

[0082] S501, determine the weight of each image block in the target gradient image based on the pixel values ​​of each image block.

[0083] In this embodiment, the first pixel sum of each image block and the second pixel sum of the target gradient image can be obtained based on the pixel values ​​of each image block. The quotient of the first pixel sum and the second pixel sum is obtained to normalize the first pixel sum and obtain the weight of each image block in the target gradient image.

[0084] S502, determine the number of control points to be selected in each image block based on the first preset number of control points and each weight.

[0085] In this embodiment, the number of control points to be selected in each image block is obtained by multiplying the first preset number of control points by each weight.

[0086] This can also be understood as, after obtaining the sum of each first pixel, directly determining the number of control points to be selected in each image block based on the ratio between the sums of each first pixel and the number of the first preset control points. For example, if the sums of the first pixels in each image block are 20000, 30000, 30000, and 20000, then the ratio between the sums of the first pixels is 2:3:3:2. If the number of the first preset control points is 100, then the number of control points to be selected in each image block is 20, 30, 30, and 20.

[0087] In this embodiment, the number of control points to be selected in each image block is determined based on the pixel value of each image block and the number of first preset control points. The method is simple and can be implemented quickly.

[0088] In one embodiment, motion artifact removal is essentially an image registration process, also known as pixel shift. This involves aligning the same anatomical tissues in the masked image and the film image in spatial location, followed by digital subtraction to remove bone and soft tissue, resulting in an image containing only blood vessels—the subtracted image. This application provides an image registration method applicable to… Figure 1 Taking the computer device shown as an example, such as Figure 6 As shown, it includes the following steps:

[0089] S601, determine the first similarity between the target mask image and the film image in the current iteration; the target mask image includes any one of the following: a mask image, an image determined by mapping the target control points on the film image to the mask image, and an image determined by mapping the target control points on the subtraction image to the mask image; the target control points are target control points determined using the method provided in any of the above embodiments.

[0090] In this embodiment, the first similarity between the target mask image and the filled image in the current iteration can be determined using various similarity measurement algorithms, such as mutual information, normalized mutual information, correlation coefficient, energy of the histogram of differences, and sum of squared differences.

[0091] S602, perform the second iteration operation; the second iteration operation includes determining the matching control point in the film image corresponding to the target control point based on the target control point on the target mask image, generating the new target mask image for the current time based on the control point pair and the target mask image, and determining the second similarity between the new target mask image and the film image for the current time; the control point pair includes the target control point and the matching control point.

[0092] In this embodiment, as Figure 7 As shown, control point pairs can be determined using block matching algorithms in bidirectional matching. For example, exhaustive search, three-step search, new three-step search, four-step search, simple and efficient search, diamond search, hexagon-based search, and Powell fast search algorithms can be used to determine control point pairs between the target mask image and the filled image.

[0093] In one possible implementation, a block matching search algorithm is used to search for the first matching block in the film image that is most similar to the block containing the target control point CP1 in the target mask. The center point of the first matching block is taken as the matching point CP2 corresponding to the target control point CP1 in the film image, thus forming a pair of control points (CP1, CP2).

[0094] Using the matching point CP2 as a control point, search for the second matching block in the target mask image that is most similar to the block containing the control point CP2 in the mask image. The center point of the second matching block is taken as the matching point CP3 corresponding to the control point CP2. If the distance between CP3 and CP1 is less than the preset distance, the control point pair (CP1, CP2) is considered a valid control point pair; otherwise, the matching search is performed again.

[0095] After determining the control point pair (CP1, CP2), the blocks containing both control point CP1 and CP2 are enlarged by the same factor (e.g., 10 times) through interpolation. The block matching search algorithm is then used to search for matching blocks for control point CP1 on the image. The resulting motion vector is divided by the enlargement factor to obtain the sub-pixel level motion vector. The coordinates of the control point pair (CP1, CP2) are then updated. At this point, the control point pair (CP1, CP2) has the most accurate match. For example, if the control point pair (CP1, CP2) determined using the above method is (5, 10), after enlarging by the same factor through interpolation and searching again, the determined control point pair (CP1, CP2) becomes (5.9, 10.8), making the position of the determined control point pair more accurate.

[0096] It should be noted that since the target mask image needs to be registered with multiple frames of film images, the position of the target control points remains unchanged. When the target mask image and the first frame of film image determine a control point pair, a control point pair (CP1, CP2) is obtained. When the target mask image and subsequent frames of film images determine a control point pair, the corresponding matching point can be determined near the matching point CP2 without re-searching. That is, the displacement of the previous frame of film image is transferred to the next frame of film image as the initial value for the matching of the next frame of film image blocks, ensuring the continuity between frames. This effectively reduces the amount of computation and avoids inter-frame flickering.

[0097] In some possible implementations, a new masking image is generated based on the control point pair and the target masking image, and the second similarity between the new masking image and the target masking image is recalculated. Optionally, the generation of the new masking image from the control point pair and the target masking image can be achieved using a cubic B-spline surface elastic transformation algorithm, or by using registration algorithms such as affine transformation or thin-plate spline elastic transformation.

[0098] S603, based on the first similarity and the second similarity of the current time, determine the target subtraction image, which is the image after registering the film image and the target mask image.

[0099] In this embodiment, if the first similarity score is greater than the second similarity score, the target subtraction image is determined based on the target mask image and the filled image corresponding to the first similarity score; if the first similarity score is not greater than the second similarity score, the target subtraction image is determined using the new mask image and the filled image corresponding to the second similarity score. Specifically, this can include: directly determining the target subtraction image using the new mask image and the filled image corresponding to the second similarity score; or further optimizing the new mask image and determining the target subtraction image based on the optimized mask image and the filled image.

[0100] Optionally, "determining the target subtraction image based on the first similarity and the second similarity of the current iteration" includes the following steps: if the second similarity is greater than or equal to the first similarity, then the new target mask image is used as the target mask image, and the second similarity is used as the first similarity for the next iteration, and the process returns to perform the second iteration operation to determine the second similarity for the next iteration; if the second similarity for the next iteration is less than the first similarity for the next iteration, then the target subtraction image is determined based on the new target mask image and the mask image from the previous iteration.

[0101] In this embodiment, when the second similarity is greater than or equal to the first similarity, it proves that the new target mask image is more similar to the film image, and the elastic transformation registration algorithm is effective. Therefore, the new target mask image is used as the target mask image, and the second similarity is used as the first similarity for the next iteration. The second iteration is then performed to continue generating new target mask images until the second similarity is less than the first similarity for the next iteration. The similarity between the new target mask image and the film image no longer increases but instead decreases, and the iteration ends. At this point, the previous new target mask image and the film image are the most similar. Finally, a subtraction operation is performed based on the previous new target mask image and the film image to generate a target subtraction image, resulting in the target subtraction image with the fewest motion artifacts.

[0102] The image registration method proposed in this application uses a single-frame iterative transformation strategy. When the iteration condition is met, the new target mask generated by the previous elastic transformation is used as the input of the next elastic transformation. This preserves and continues to pass on the registration trend between the new target mask image and the stencil image until the iteration cutoff condition is reached. At this point, the previous new target mask image will be the mask image most similar to the stencil image, and the final target subtraction image will also have the fewest artifacts.

[0103] In one embodiment, such as Figure 8 As shown, Figure 8 The image on the left is a subtraction image determined directly using the plenum image and the mask image. Figure 8 The right side is a subtraction image obtained by the method proposed in this application. It can be seen that the ribs on the left, the diaphragm on the upper side, and the intestinal gas on the right and lower sides are significantly improved. The blood vessels are clearly distinguished from the background. It can be found that this method is very effective for both small and large motion artifacts. The subtraction image obtained removes most of the artifacts and significantly improves the quality of the subtraction image.

[0104] In one embodiment, Figure 9A and Figure 9B The above Figure 4A The corresponding subtraction image, Figure 9C and Figure 9D For the above Figure 4B The corresponding subtraction image. For example... Figure 9A , Figure 9B , Figure 9C and Figure 9D As shown, Figure 9A and Figure 9C It is a subtraction image determined directly using the full-size image and the mask image. Figure 9B and Figure 9D The image obtained by the method proposed in this application shows that the blood vessel contrast is significantly improved and most motion artifacts are eliminated, thus enhancing the quality of the subtraction image and significantly improving its diagnostic value.

[0105] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0106] Based on the same inventive concept, this application also provides a control point determination apparatus for implementing the control point determination method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more control point determination apparatus embodiments provided below can be found in the limitations of the control point determination method described above, and will not be repeated here.

[0107] In one embodiment, such as Figure 10 As shown, a control point determination device is provided, comprising: a segmentation module 11, a first determination module 12, and a second determination module 13, wherein:

[0108] The segmentation module 11 is used to segment the target gradient image to obtain multiple first image blocks; the target gradient image is the gradient image of the target image of the inspected object, and the target image includes any one of the following: mask image, full image, and subtraction image;

[0109] The first determining module 12 is used to determine the number of control points to be selected in each image block based on the first preset number of control points and the pixel value of each image block;

[0110] The second determining module 13 is used to determine target control points from the target gradient image based on the image block information corresponding to each image block; the image block information includes the number of control points to be selected in the corresponding image block and / or the size of the image block.

[0111] In one embodiment, the second determining module includes:

[0112] The first determining unit is used to take the image block corresponding to the image block information that meets the preset conditions as the first target image block;

[0113] The execution unit is used to perform the first iteration operation: segment the first target image block to obtain multiple new image blocks in the current iteration, and determine the number of control points to be selected in each new image block according to the number of control points corresponding to the first target image block and the pixel value of each new image block, and take the new image block corresponding to the image block information that meets the preset conditions as the first target image block, and return to execute the first iteration operation until the image block information corresponding to each new image block obtained in the current iteration does not meet the preset conditions.

[0114] The second determining unit is used to determine target control points from each second target image block based on the pixel values ​​of each pixel in the second target image block; the second target image block includes the image block corresponding to the image block information that does not meet the preset conditions in each determination.

[0115] In one embodiment, the second determining module further includes:

[0116] The third determining unit is used to determine whether each image block meets the preset conditions based on the image block information of each image block.

[0117] The fourth determining unit is used to take the pixel corresponding to the largest pixel value in each image block as the target control point if the image block information of each image block does not meet the preset conditions.

[0118] In one embodiment, the first determining unit is further configured to determine that the image block information corresponding to the image block satisfies a preset condition if the number of control points corresponding to the image block is greater than the number of second preset control points, and / or the size of the image block is greater than a preset size, and to use the image block corresponding to the image block information that satisfies the preset condition as the first target image block.

[0119] In one embodiment, the first determining module includes:

[0120] The fifth determining unit is used to determine the weight of each image block in the target gradient image based on the pixel values ​​of each image block;

[0121] The sixth determining unit is used to determine the number of control points to be selected in each image block based on the first preset number of control points and each weight.

[0122] In one embodiment, an image registration apparatus is provided, the apparatus comprising:

[0123] The third determining module is used to determine the first similarity between the target mask image and the stencil image in the current iteration; the target mask image includes any one of the following: the mask image, the image determined by mapping the target control points on the stencil image to the mask image, and the image determined by mapping the target control points on the subtraction image to the mask image; the target control points are target control points determined using the method of any of the above embodiments.

[0124] An execution module is used to perform a second iteration operation; the second iteration operation includes determining matching control points in the film image corresponding to the target control points based on the target control points on the target mask image, generating a new target mask image for the current iteration based on the control point pair and the target mask image, and determining the second similarity between the new target mask image and the film image for the current iteration; the control point pair includes the target control point and the matching control point;

[0125] The fourth determining module is used to determine the target subtraction image based on the first similarity and the second similarity of the current time. The target subtraction image is the image after registering the film image and the target mask image.

[0126] In one embodiment, the fourth determining module includes:

[0127] The seventh determining unit is used to, if the second similarity is greater than or equal to the first similarity, take the new target mask image as the target mask image, take the second similarity as the first similarity for the next iteration, and return to perform the second iteration operation to determine the second similarity for the next iteration.

[0128] The eighth determining unit is used to determine the target subtraction image based on the previous new target mask image and the filled image if the second similarity in the next step is less than the first similarity in the next step.

[0129] Each module in the aforementioned control point determination device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0130] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0131] The target gradient image is segmented to obtain multiple first image blocks; the target gradient image is the gradient image of the target image of the inspected object, and the target image includes any one of the following: mask image, full image, and subtraction image;

[0132] Based on the first preset number of control points and the pixel values ​​of each image block, determine the number of control points to be selected in each image block;

[0133] Based on the image patch information corresponding to each image patch, target control points are determined from the target gradient image; the image patch information includes the number of control points to be selected in the corresponding image patch and / or the size of the image patch.

[0134] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0135] The image block corresponding to the image block information that meets the preset conditions is taken as the first target image block;

[0136] Perform the first iteration operation: segment the first target image block to obtain multiple new image blocks in the current iteration, and determine the number of control points to be selected in each new image block based on the number of control points corresponding to the first target image block and the pixel values ​​of each new image block. Then, take the new image block corresponding to the image block information that meets the preset conditions as the first target image block, and return to perform the first iteration operation until the image block information corresponding to each new image block obtained in the current iteration does not meet the preset conditions.

[0137] Based on the pixel values ​​of each pixel in the second target image block, target control points are determined from each second target image block; the second target image block includes the image block corresponding to the image block information that does not meet the preset conditions in each determination.

[0138] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0139] Determine whether each image block meets the preset conditions based on the image block information of each image block;

[0140] If the image block information of each image block does not meet the preset conditions, then the pixel corresponding to the largest pixel value in each image block will be used as the target control point.

[0141] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0142] If the number of control points corresponding to the image block is greater than the second preset number of control points, and / or the size of the image block is greater than the preset size, then the image block information corresponding to the image block is determined to meet the preset conditions, and the image block corresponding to the image block information that meets the preset conditions is taken as the first target image block.

[0143] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0144] The weight of each image patch in the target gradient image is determined based on the pixel values ​​of each image patch.

[0145] Based on the first preset number of control points and their respective weights, determine the number of control points to be selected in each image block.

[0146] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0147] Determine the first similarity between the target mask image and the film image in the current iteration; the target mask image includes any one of the following: a mask image, an image determined by mapping the target control points on the film image to the mask image, and an image determined by mapping the target control points on the subtraction image to the mask image; the target control points are the target control points determined using the method of any of the above embodiments.

[0148] Perform a second iteration operation; the second iteration operation includes determining matching control points in the film image corresponding to the target control points based on the target control points on the target mask image, generating a new target mask image for the current iteration based on the control point pair and the target mask image, and determining the second similarity between the new target mask image and the film image for the current iteration; the control point pair includes the target control point and the matching control point;

[0149] Based on the first and second similarities of the current iteration, the target subtraction image is determined. The target subtraction image is the image obtained by registering the film image and the target mask image.

[0150] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0151] If the second similarity is greater than or equal to the first similarity, then the new target mask image is used as the target mask image, and the second similarity is used as the first similarity for the next iteration. Then, the second iteration operation is performed to determine the second similarity for the next iteration.

[0152] If the second similarity score in the next iteration is less than the first similarity score in the next iteration, then the target subtraction image is determined based on the previous new target mask image and the filled image.

[0153] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0154] The target gradient image is segmented to obtain multiple first image blocks; the target gradient image is the gradient image of the target image of the inspected object, and the target image includes any one of the following: mask image, full image, and subtraction image;

[0155] Based on the first preset number of control points and the pixel values ​​of each image block, determine the number of control points to be selected in each image block;

[0156] Based on the image patch information corresponding to each image patch, target control points are determined from the target gradient image; the image patch information includes the number of control points to be selected in the corresponding image patch and / or the size of the image patch.

[0157] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0158] The image block corresponding to the image block information that meets the preset conditions is taken as the first target image block;

[0159] Perform the first iteration operation: segment the first target image block to obtain multiple new image blocks in the current iteration, and determine the number of control points to be selected in each new image block based on the number of control points corresponding to the first target image block and the pixel values ​​of each new image block. Then, take the new image block corresponding to the image block information that meets the preset conditions as the first target image block, and return to perform the first iteration operation until the image block information corresponding to each new image block obtained in the current iteration does not meet the preset conditions.

[0160] Based on the pixel values ​​of each pixel in the second target image block, target control points are determined from each second target image block; the second target image block includes the image block corresponding to the image block information that does not meet the preset conditions in each determination.

[0161] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0162] Determine whether each image block meets the preset conditions based on the image block information of each image block;

[0163] If the image block information of each image block does not meet the preset conditions, then the pixel corresponding to the largest pixel value in each image block will be used as the target control point.

[0164] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0165] If the number of control points corresponding to the image block is greater than the second preset number of control points, and / or the size of the image block is greater than the preset size, then the image block information corresponding to the image block is determined to meet the preset conditions, and the image block corresponding to the image block information that meets the preset conditions is taken as the first target image block.

[0166] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0167] The weight of each image patch in the target gradient image is determined based on the pixel values ​​of each image patch.

[0168] Based on the first preset number of control points and their respective weights, determine the number of control points to be selected in each image block.

[0169] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0170] Determine the first similarity between the target mask image and the film image in the current iteration; the target mask image includes any one of the following: a mask image, an image determined by mapping the target control points on the film image to the mask image, and an image determined by mapping the target control points on the subtraction image to the mask image; the target control points are target control points determined using the method of any of the above embodiments.

[0171] Perform a second iteration operation; the second iteration operation includes determining matching control points in the film image corresponding to the target control points based on the target control points on the target mask image, generating a new target mask image for the current iteration based on the control point pair and the target mask image, and determining the second similarity between the new target mask image and the film image for the current iteration; the control point pair includes the target control point and the matching control point;

[0172] Based on the first and second similarities of the current iteration, the target subtraction image is determined. The target subtraction image is the image obtained by registering the film image and the target mask image.

[0173] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0174] If the second similarity is greater than or equal to the first similarity, then the new target mask image is used as the target mask image, and the second similarity is used as the first similarity for the next iteration. Then, the second iteration operation is performed to determine the second similarity for the next iteration.

[0175] If the second similarity score in the next iteration is less than the first similarity score in the next iteration, then the target subtraction image is determined based on the previous new target mask image and the filled image.

[0176] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0177] The target gradient image is segmented to obtain multiple first image blocks; the target gradient image is the gradient image of the target image of the inspected object, and the target image includes any one of the following: mask image, full image, and subtraction image;

[0178] Based on the first preset number of control points and the pixel values ​​of each image block, determine the number of control points to be selected in each image block;

[0179] Based on the image patch information corresponding to each image patch, target control points are determined from the target gradient image; the image patch information includes the number of control points to be selected in the corresponding image patch and / or the size of the image patch.

[0180] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0181] The image block corresponding to the image block information that meets the preset conditions is taken as the first target image block;

[0182] Perform the first iteration operation: segment the first target image block to obtain multiple new image blocks in the current iteration, and determine the number of control points to be selected in each new image block based on the number of control points corresponding to the first target image block and the pixel values ​​of each new image block. Then, take the new image block corresponding to the image block information that meets the preset conditions as the first target image block, and return to perform the first iteration operation until the image block information corresponding to each new image block obtained in the current iteration does not meet the preset conditions.

[0183] Based on the pixel values ​​of each pixel in the second target image block, target control points are determined from each second target image block; the second target image block includes the image block corresponding to the image block information that does not meet the preset conditions in each determination.

[0184] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0185] Determine whether each image block meets the preset conditions based on the image block information of each image block;

[0186] If the image block information of each image block does not meet the preset conditions, then the pixel corresponding to the largest pixel value in each image block will be used as the target control point.

[0187] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0188] If the number of control points corresponding to the image block is greater than the second preset number of control points, and / or the size of the image block is greater than the preset size, then the image block information corresponding to the image block is determined to meet the preset conditions, and the image block corresponding to the image block information that meets the preset conditions is taken as the first target image block.

[0189] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0190] The weight of each image patch in the target gradient image is determined based on the pixel values ​​of each image patch.

[0191] Based on the first preset number of control points and their respective weights, determine the number of control points to be selected in each image block.

[0192] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0193] Determine the first similarity between the target mask image and the film image in the current iteration; the target mask image includes any one of the following: a mask image, an image determined by mapping the target control points on the film image to the mask image, and an image determined by mapping the target control points on the subtraction image to the mask image; the target control points are target control points determined using the method of any of the above embodiments.

[0194] Perform a second iteration operation; the second iteration operation includes determining matching control points in the film image corresponding to the target control points based on the target control points on the target mask image, generating a new target mask image for the current iteration based on the control point pair and the target mask image, and determining the second similarity between the new target mask image and the film image for the current iteration; the control point pair includes the target control point and the matching control point;

[0195] Based on the first and second similarities of the current iteration, the target subtraction image is determined. The target subtraction image is the image obtained by registering the film image and the target mask image.

[0196] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0197] If the second similarity is greater than or equal to the first similarity, then the new target mask image is used as the target mask image, and the second similarity is used as the first similarity for the next iteration. Then, the second iteration operation is performed to determine the second similarity for the next iteration.

[0198] If the second similarity score in the next iteration is less than the first similarity score in the next iteration, then the target subtraction image is determined based on the previous new target mask image and the filled image.

[0199] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0200] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0201] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0202] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for determining control points, characterized in that, The method includes: The target gradient image is segmented to obtain multiple image blocks; the target gradient image is the gradient image of the target image of the inspected object, and the target image includes any one of the following: masked image, full image, and subtraction image; Based on the first preset number of control points and the pixel value of each image block, determine the number of control points to be selected in each image block; If the number of control points corresponding to the image block is greater than the number of second preset control points, and / or the size of the image block is greater than the preset size, then it is determined that the image block information corresponding to the image block meets the preset conditions. The image block corresponding to the image block information that meets the preset conditions is taken as the first target image block, and the first target image block is iteratively processed to obtain multiple new image blocks in the current step, until the image block information corresponding to each new image block obtained in the current step does not meet the preset conditions; according to the pixel value of each pixel in the second target image block, target control points are determined from each second target image block; the second target image block includes the image block corresponding to the image block information that does not meet the preset conditions determined in each step; the image block information includes the number of control points to be selected in the corresponding image block and / or the size of the image block.

2. The method according to claim 1, characterized in that, The iterative processing of the first target image block to segment it into multiple new image blocks for the current iteration, until the image block information corresponding to each new image block obtained in the current iteration no longer meets the preset condition, includes: Perform the first iteration operation: segment the first target image block to obtain multiple new image blocks for the current iteration, and determine the number of control points to be selected in each new image block based on the number of control points corresponding to the first target image block and the pixel values ​​of each new image block. Then, take the new image block corresponding to the image block information that meets the preset conditions as the first target image block, and return to perform the first iteration operation until the image block information corresponding to each new image block obtained in the current iteration does not meet the preset conditions.

3. The method according to claim 1, characterized in that, The method further includes: If the image block information of each image block does not meet the preset conditions, then the pixel corresponding to the largest pixel value in each image block is taken as the target control point.

4. The method according to claim 1, characterized in that, The step of determining the number of control points to be selected in each image block based on the first preset number of control points and the pixel values ​​of each image block includes: The weight of each image block in the target gradient image is determined based on the pixel value of each image block; Based on the first preset number of control points and each of the weights, the number of control points to be selected in each of the image blocks is determined.

5. An image registration method, characterized in that, The method includes: Determine the first similarity between the target mask image and the film image in the current iteration; the target mask image includes any one of the following: a mask image, an image determined by mapping the target control points on the film image to the mask image, and an image determined by mapping the target control points on the subtraction image to the mask image, wherein the target control points are target control points determined using the method described in any one of claims 1-4; Perform a second iteration operation; the second iteration operation includes determining a matching control point in the film image corresponding to the target control point based on the target control point on the target mask image, generating a new target mask image for the current iteration based on the control point pair and the target mask image, and determining the second similarity between the new target mask image and the film image for the current iteration; the control point pair includes the target control point and the matching control point; Based on the first similarity and the second similarity of the current time, a target subtraction image is determined, wherein the target subtraction image is the image after registration of the film image and the target mask image.

6. The method according to claim 5, characterized in that, Determining the target subtraction image based on the first similarity and the second similarity of the current iteration includes: If the second similarity is greater than or equal to the first similarity, then the new target mask image is used as the target mask image, and the second similarity is used as the first similarity for the next iteration. The second iteration operation is then performed to determine the second similarity for the next iteration. If the second similarity in the next step is less than the first similarity in the next step, then the target subtraction image is determined based on the previous new target mask image and the filled image.

7. A control point determination device, characterized in that, The device includes: The segmentation module is used to segment the target gradient image to obtain multiple first image blocks; the target gradient image is the gradient image of the target image of the inspected object, and the target image includes any one of the following: mask image, full image, and subtraction image; The first determining module is used to determine the number of control points to be selected in each of the image blocks based on the first preset number of control points and the pixel value of each image block; The second determining module is configured to: determine that the image block information corresponding to the image block satisfies a preset condition if the number of control points corresponding to the image block is greater than a second preset number of control points, and / or the size of the image block is greater than a preset size; take the image block corresponding to the image block information that satisfies the preset condition as a first target image block, and perform iterative processing on the first target image block to segment it into multiple new image blocks for the current time, until the image block information corresponding to each new image block obtained in the current time does not satisfy the preset condition; determine target control points from each second target image block according to the pixel values ​​of each pixel in the second target image block; the second target image block includes the image block corresponding to the image block information that does not satisfy the preset condition determined in each time; the image block information includes the number of control points to be selected in the corresponding image block and / or the size of the image block.

8. An image registration device, characterized in that, The device includes: The third determining module is used to determine the first similarity between the target mask image and the stencil image in the current iteration; the target mask image includes any one of the following: a mask image, an image determined by mapping the target control points on the stencil image to the mask image, and an image determined by mapping the target control points on the subtraction image to the mask image; the target control points are target control points determined using the method described in any one of claims 1-4. An execution module is used to perform a second iteration operation; the second iteration operation includes determining a pair of control points in the film image based on the target control points on the target mask image, generating a new target mask image for the current iteration based on the control point pairs and the target mask image, and determining a second similarity between the new target mask image and the film image for the current iteration; the control point pair includes the target control point and the control points in the film image corresponding to the target control point; The fourth determining module is used to determine the target subtraction image based on the first similarity and the second similarity of the current time, wherein the target subtraction image is the image after registration of the smear image and the target mask image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

Images