Slice adjusting quantity determining method, slice scanning area determining method, slice adjusting quantity determining device, slice scanning area determining device, equipment and medium

Abstract

The invention discloses a slice adjusting quantity determining method, a slice scanning area determining method, a slice adjusting quantity determining device, slice scanning area determining equipment and a medium. The method comprises the following steps: on the basis of obtaining feature region images of a plurality of continuous slice images, determining a first adjustment amount of a feature region of a current number compared with a feature region of a slice image with an initial number based on a difference between the feature region images of the slice image with the current number and the feature region image of the slice image with the initial number, the current serial number is a serial number between the start serial number and the end serial number; predicting a second adjustment amount of the current number compared with the feature region of the initial number slice image based on the difference between the initial number slice image and the termination number slice image; and according to the first adjustment amount and the second adjustment amount, determining a target adjustment amount of the region of interest of the slice image with the current number compared with the initial number. According to the scheme, the position of each layer of slice can be automatically adjusted to realize position registration, and the registration efficiency is improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of image processing technology, and in particular to a method for determining slice adjustment amount, a method for determining slice scanning area, a device, a device and a medium. Background Art

[0002] The slice three-dimensional reconstruction technology can be understood as making a sample into continuous slices, batch scanning these slices to obtain a high-resolution two-dimensional image sequence, and then using computer algorithms to stack and reconstruct these two-dimensional images into a three-dimensional model to reveal the spatial structure characteristics of the sample. This technology has important application value in the fields of biomedical research and so on.

[0003] In order to obtain high-quality three-dimensional reconstruction results, the scanned images of different layers of slices need to maintain a consistent positional relationship in space as much as possible. However, in the prior art, when batch scanning slices, operators need to manually adjust the position of each layer of slices to achieve image registration. This method is inefficient, requires a high level of technical skills from operators, has a high labor intensity, and is prone to introducing human errors. Summary of the Invention

[0004] The present invention provides a method for determining slice adjustment amount, a method for determining slice scanning area, a device, a device and a medium, which can automatically adjust the position of each layer of slices to achieve position registration of different layer slice images, not only helps to improve the spatial accuracy of three-dimensional reconstruction, but also can greatly improve work efficiency and reduce the labor intensity of operators.

[0005] In a first aspect, an embodiment of the present invention provides a method for determining slice adjustment amount, including:

[0006] Based on the feature region images of continuously obtained multiple slice images, determine a first adjustment amount of the feature region of the slice image with the current number compared to the slice image with the starting number based on the difference between the feature region images of the slice images with the current number and the starting number, where the current number is a number between the starting number and the ending number;

[0007] Predict a second adjustment amount of the feature region of the slice image with the current number compared to the slice image with the starting number based on the difference between the feature region images of the slice images with the starting number and the ending number;

[0008] Determine a target adjustment amount of the region of interest of the slice image with the current number compared to the slice image with the starting number according to the first adjustment amount and the second adjustment amount.

[0009] In a second aspect, an embodiment of the present invention provides a method for determining slice scanning area, including:

[0010] Based on obtaining a continuous plurality of slice images, in response to a region selection operation, determine the region of interest and the feature region in the starting numbered slice image, and determine the region of interest and the feature region in the ending numbered slice image;

[0011] Through an image feature automatic matching algorithm, based on the region of interest and the feature region in the starting numbered slice image, determine the region of interest and the feature region in the currently numbered slice image, where the currently numbered is the number between the starting number and the ending number;

[0012] Based on the target adjustment amount of the region of interest in the currently numbered slice image compared to the starting numbered slice image, adjust the region of interest in the currently numbered slice image to obtain a target scanning region, and the target adjustment amount is determined by the slice adjustment amount determination method described in the first aspect.

[0013] In a third aspect, an embodiment of the present invention provides a slice adjustment amount determination device, including:

[0014] A first adjustment amount determination module, configured to, based on obtaining the feature region images of a continuous plurality of slice images, determine the first adjustment amount of the feature region in the currently numbered slice image compared to the starting numbered slice image based on the difference between the feature region images of the currently numbered and the starting numbered slice images, where the currently numbered is the number between the starting number and the ending number;

[0015] A second adjustment amount determination module, configured to predict the second adjustment amount of the feature region in the currently numbered slice image compared to the starting numbered slice image based on the difference between the feature region images of the starting numbered and the ending numbered slice images;

[0016] A target adjustment amount determination module, configured to determine the target adjustment amount of the region of interest in the currently numbered slice image compared to the starting numbered slice image according to the first adjustment amount and the second adjustment amount.

[0017] In a fourth aspect, an embodiment of the present invention provides a slice scanning region determination device, including:

[0018] A region selection module, configured to, based on obtaining a continuous plurality of slice images, in response to a region selection operation, determine the region of interest and the feature region in the starting numbered slice image, and determine the region of interest and the feature region in the ending numbered slice image;

[0019] A region matching module, configured to, through an image feature automatic matching algorithm, based on the region of interest and the feature region in the starting numbered slice image, determine the region of interest and the feature region in the currently numbered slice image, where the currently numbered is the number between the starting number and the ending number;

[0020] An area adjustment module, configured to adjust an area of interest of a slice image with the current number relative to a starting number based on a target adjustment amount, so as to obtain a target scanning area, and the target adjustment amount is determined by a slice adjustment amount determining device described in the third aspect.

[0021] In a fifth aspect, an embodiment of the present invention provides an electronic device, including:

[0022] At least one processor; and

[0023] A memory communicatively connected to the at least one processor; wherein,

[0024] The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the at least one processor is enabled to execute the method described in the first aspect or execute the method described in the second aspect.

[0025] In a sixth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the method described in the first aspect or implements the method described in the second aspect.

[0026] Based on the feature region images of a continuous plurality of slice images obtained, the technical solution provided by the embodiment of the present invention determines a first adjustment amount of the feature region of the slice image with the current number relative to the slice image with the starting number based on the difference between the feature region images of the slice images with the current number and the starting number, where the current number is a number between the starting number and the ending number; predicts a second adjustment amount of the feature region of the slice image with the current number relative to the slice image with the starting number based on the difference between the feature region images of the slice images with the starting number and the ending number; and determines a target adjustment amount of the area of interest of the slice image with the current number relative to the slice image with the starting number according to the first adjustment amount and the second adjustment amount. This solution can automatically determine the target adjustment amount of the area of interest of each slice relative to the slice with the starting number, and this target adjustment amount is used to automatically adjust the slice position to achieve position registration of different layer slice images, which not only helps to improve the spatial accuracy of three-dimensional reconstruction, but also can greatly improve work efficiency and reduce the labor intensity of operators.

[0027] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0029] Figure 1 is a flowchart of a method for determining the slice adjustment amount according to Embodiment 1 of the present invention;

[0030] Figure 2 is a flowchart of a method for determining the slice scanning area according to Embodiment 2 of the present invention;

[0031] Figure 3 is a schematic diagram of the comparison of slice images according to Embodiment 3 of the present invention;

[0032] Figure 4 is a schematic diagram of the area selection operation in the slice image according to Embodiment 3 of the present invention;

[0033] Figure 5 is a schematic diagram of the region of interest image and the feature region image according to Embodiment 3 of the present invention;

[0034] Figure 6 is a schematic structural diagram of a device for determining the slice adjustment amount according to Embodiment 4 of the present invention;

[0035] Figure 7 is a schematic structural diagram of a device for determining the slice scanning area according to Embodiment 5 of the present invention;

[0036] Figure 8 is a schematic structural diagram of an electronic device for implementing the embodiments of the present invention. Detailed implementation manners

[0037] In order to enable those skilled in the art of the present technology to better understand the solutions of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present invention.

[0038] It should be noted that the terms "first", "second", etc. in the present invention are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0039] Embodiment 1

[0040] Figure 1 It is a flowchart of a method for determining the slice adjustment amount provided according to Embodiment 1 of the present invention. This embodiment is applicable to the situation of determining the slice adjustment amount. This method can be executed by a slice adjustment amount determination device, and this device can be implemented in the form of software and / or hardware and integrated in an electronic device. Further, the electronic device includes but is not limited to: a computer, a laptop computer, etc.

[0041] As Figure 1 shown, the method includes:

[0042] S110. Based on obtaining the feature region images of multiple consecutive slice images, determine the first adjustment amount of the feature region of the slice image with the current number compared to the slice image with the starting number based on the difference between the feature region images of the slice images with the current number and the starting number. The current number is a number between the starting number and the ending number.

[0043] In the embodiments of the present invention, the slice can be a biological slice or a slice in other slice scanning scenarios, which is not limited here.

[0044] In practical applications, slices of different layers belonging to the same sample are usually arranged in sequence on the sample stage of a scanning electron microscope. A panoramic image of the sample stage (usually an optical image) can be obtained, and the slices to be scanned are selected based on the panoramic image, such as selecting 500 slices to be scanned. When selecting slices, since there is a large color contrast between the slices and the background color of the sample stage, the edges of the selected slices can be automatically recognized through this contrast (see Patent CN119599867A in detail), so as to number each slice (for subsequent scanning operations in the order of the numbers) and perform overall positioning (so that the sample stage can automatically move the slice corresponding to a certain number to under the objective lens).

[0045] Under normal circumstances, the slice numbering order is from top to bottom and from left to right. Therefore, the slices can be arranged in several columns from top to bottom and from left to right on the sample stage layer by layer, and numbered according to the arrangement order. Then, scanning operations can be performed in the order of the slice numbers to obtain a series of consecutive slice images.

[0046] Among a series of consecutive slice images, the first obtained slice image corresponds to the starting number, and the last obtained slice image corresponds to the ending number. The current number is the number of the slice image for which the slice adjustment amount needs to be determined currently, and the current number is any number between the starting number and the ending number.

[0047] Each slice image can contain two types of regions. One is the feature region, and the other is the region of interest. The positions of these two types of regions in each slice image can be predetermined. Details on how to determine the positions of these two types of regions in each slice image can be found in the subsequent embodiments.

[0048] The feature region can be a region in the slice image with certain obvious features or characteristics. Scanning the feature region can obtain a feature region image. The region of interest (ROI) can be a region in the slice image that needs to be concerned, such as the region to be three-dimensionally reconstructed during the scanning operation. Scanning the region of interest can obtain a region of interest image. Each slice image can contain one feature region, and in this case, the features of the feature regions in different slice images are the same; each slice image can also contain multiple feature regions, and in this case, the number of feature regions in different slice images is the same and the features correspond.

[0049] Based on obtaining the feature region images of a series of consecutive slice images, if each slice image contains one feature region, then compare the differences between the feature region images of the slice image with the current number and the slice image with the starting number, determine the offset and rotation amounts of the feature region of the slice image with the current number compared to the slice image with the starting number, and determine the offset and rotation amounts of the feature region as the first adjustment amount of the feature region of the slice image with the current number compared to the slice image with the starting number.

[0050] If each slice image contains multiple feature regions, then respectively take each feature region as the target feature region, compare the differences between the feature region images corresponding to the target feature region of the slice image with the current number and the slice image with the starting number, determine the offset and rotation amounts of the target feature region of the slice image with the current number compared to the slice image with the starting number; determine the offset and rotation amounts corresponding to each of the multiple feature regions as the first adjustment amount of the feature region of the slice image with the current number compared to the slice image with the starting number.

[0051] Among them, to determine the offset and rotation amount of the corresponding feature region by comparing two feature region images, one can refer to Patent CN114663686B or other publicly disclosed registration techniques based on feature point detection. The registration technique based on feature point detection extracts key feature points (such as corner points, edges, etc.) in the image, calculates the geometric transformation matrix between the two images, and realizes the alignment of the images. The implementation effect of the registration technique based on feature point detection has a strong correlation with the image contrast. Therefore, the scanning contrast of the electron microscope can be adjusted, and the scanning mode with the largest contrast can be selected to obtain the feature region image.

[0052] S120. Predict the second adjustment amount of the feature region of the current numbered slice image compared with the feature region of the starting numbered slice image based on the difference between the feature region images of the slice images with the starting number and the ending number.

[0053] In this step, the difference between the feature region images of the slice images with the ending number and the starting number can be compared to determine the region adjustment amount of the feature region of the slice image with the ending number compared with the feature region of the slice image with the starting number. The determination method of the region adjustment amount is basically the same as the determination method of the first adjustment amount of the feature region of the current numbered slice image compared with the feature region of the starting numbered slice image in S110, and will not be elaborated here.

[0054] Based on the determined region adjustment amount, predict the second adjustment amount of the feature region of the current numbered slice image compared with the feature region of the starting numbered slice image. The second adjustment amount can be determined by ((M - 1) / (N - 1)) × region adjustment amount, where M is the current number and N is the total number of slice images. Among them, if each slice image contains one feature region, the region adjustment amount involved in the calculation is the offset or rotation amount of this feature region, and the second adjustment amount includes the offset and rotation amount of the current numbered slice image compared with the feature region of the starting numbered slice image for this feature region; if each slice image contains multiple feature regions, the region adjustment amount involved in the calculation is the offset or rotation amount of any one of the multiple feature regions, and the second adjustment amount includes the offset and rotation amount of the current numbered slice image compared with the feature regions of the starting numbered slice image corresponding to each of the multiple feature regions.

[0055] Exemplarily, when N is 500, M is 2, and the slice image with the ending number is offset by 0.1 μm and rotated 1° clockwise compared with the feature region of the slice image with the starting number for a certain feature region, it can be predicted that the slice image with the current number is offset by 0.1 μm / 499 and rotated 1° / 499 clockwise compared with the feature region of the slice image with the starting number for this feature region.

[0056] S130. Determine the target adjustment amount of the region of interest of the current numbered slice image compared with the region of interest of the starting numbered slice image according to the first adjustment amount and the second adjustment amount.

[0057] In this step, the offset difference between two offsets for the same feature region in the first adjustment amount and the second adjustment amount can be determined, and the rotation amount difference between two rotation amounts for the same feature region in the first adjustment amount and the second adjustment amount can be determined.

[0058] If each slice image contains one feature region, the offset difference of this feature region is determined as the offset of the region of interest in the slice image with the current number compared to the slice image with the starting number, and the rotation amount difference of this feature region is determined as the rotation amount of the region of interest in the slice image with the current number compared to the slice image with the starting number.

[0059] If each slice image contains multiple feature regions, the offset of the region of interest in the slice image with the current number compared to the slice image with the starting number is determined according to the offset differences corresponding to each of the multiple feature regions, and the rotation amount of the region of interest in the slice image with the current number compared to the slice image with the starting number is determined according to the rotation amount differences corresponding to each of the multiple feature regions.

[0060] The offset and rotation amount of the region of interest in the slice image with the current number compared to the slice image with the starting number are determined as the target adjustment amount of the region of interest in the slice image with the current number compared to the slice image with the starting number.

[0061] The technical solution provided by the embodiment of the present invention, based on obtaining the feature region images of a continuous plurality of slice images, determines the first adjustment amount of the feature region of the slice image with the current number compared to the slice image with the starting number based on the difference between the feature region images of the slice image with the current number and the slice image with the starting number, where the current number is a number between the starting number and the ending number; predicts the second adjustment amount of the feature region of the slice image with the current number compared to the slice image with the starting number based on the difference between the feature region images of the slice image with the starting number and the slice image with the ending number; and determines the target adjustment amount of the region of interest in the slice image with the current number compared to the slice image with the starting number according to the first adjustment amount and the second adjustment amount. This solution can automatically determine the target adjustment amount of the region of interest in each layer of slices compared to the slice with the starting number, and this target adjustment amount is used to automatically adjust the slice position to achieve position registration of slice images in different layers, which not only helps to improve the spatial accuracy of three-dimensional reconstruction, but also can greatly improve work efficiency and reduce the labor intensity of operators.

[0062] In one embodiment, each slice image contains multiple feature regions, the number of feature regions in different slice images is the same and the features correspond, and determining the target adjustment amount of the region of interest in the slice image with the current number compared to the slice image with the starting number according to the first adjustment amount and the second adjustment amount includes:

[0063] Taking each feature region as the target feature region respectively, determine the first offset and the first rotation amount in the first adjustment amount for the target feature region, and the second offset and the second rotation amount in the second adjustment amount for the target feature region;

[0064] Based on the first offset and the second offset, determine the offset difference corresponding to the target feature region, and based on the first rotation amount and the second rotation amount, determine the rotation amount difference corresponding to the target feature region;

[0065] According to the offset differences and rotation amount differences corresponding to each feature region, determine the target adjustment amount of the region of interest of the current numbered slice image compared to the starting numbered slice image.

[0066] That is, taking each feature region as the target feature region respectively, determine the offset and rotation amount corresponding to the target feature region in the first adjustment amount as the first offset and the first rotation amount respectively; determine the offset and rotation amount corresponding to the target feature region in the second adjustment amount as the second offset and the second rotation amount respectively.

[0067] Determine the difference between the first offset and the second offset as the offset difference corresponding to the target feature region, and determine the difference between the first rotation amount and the second rotation amount as the rotation amount difference corresponding to the target feature region.

[0068] According to the offset differences corresponding to each feature region, determine the offset of the region of interest of the current numbered slice image compared to the starting numbered slice image; according to the rotation amount differences corresponding to each feature region, determine the rotation amount of the region of interest of the current numbered slice image compared to the starting numbered slice image; determine the offset and rotation amount of the region of interest of the current numbered slice image compared to the starting numbered slice image as the target adjustment amount.

[0069] In one embodiment, determining the target adjustment amount of the region of interest of the current numbered slice image compared to the starting numbered slice image according to the offset differences and rotation amount differences corresponding to each feature region includes:

[0070] Combining the result of averaging the offset differences corresponding to each feature region and the result of averaging the rotation amount differences corresponding to each feature region to determine the target adjustment amount; or,

[0071] Combining the result of weighted averaging the offset differences corresponding to each feature region and the result of weighted averaging the rotation amount differences corresponding to each feature region to determine the target adjustment amount.

[0072] That is, there are two calculation methods for the target adjustment amount. The first is a calculation method with a lower calculation amount, and the second is a calculation method with a higher calculation amount.

[0073] A calculation method with lower computational complexity, that is, the result of averaging the offset differences corresponding to each feature region is determined as the offset of the region of interest of the slice image with the current number compared to the slice image with the starting number; the result of averaging the rotation differences corresponding to each feature region is determined as the rotation of the region of interest of the slice image with the current number compared to the slice image with the starting number; the offset and rotation of the region of interest are the target adjustment amounts.

[0074] A calculation method with higher computational complexity, that is, the result of weighted averaging the offset differences corresponding to each feature region is determined as the offset of the region of interest of the slice image with the current number compared to the slice image with the starting number; the result of weighted averaging the rotation differences corresponding to each feature region is determined as the rotation of the region of interest of the slice image with the current number compared to the slice image with the starting number; the offset and rotation of the region of interest are the target adjustment amounts.

[0075] In one embodiment, the weights for weighted averaging the offset differences corresponding to each feature region are determined based on the distance between the center of the region of interest in the slice image with the current number and the centers of each feature region; the weights for weighted averaging the rotation differences corresponding to each feature region are determined based on the rotation amounts for each feature region in the second adjustment amount.

[0076] In practical applications, when weighted averaging the offset differences corresponding to each feature region, the distance between the center of the region of interest in the slice image with the current number and the centers of each feature region can be determined. It can be set that the offset weight corresponding to the feature region with a far distance is small, and the offset weight corresponding to the feature region with a near distance is large. There is no limitation here.

[0077] When weighted averaging the rotation differences corresponding to each feature region, the rotation amounts for each feature region in the second adjustment amount can be determined. It can be set that the rotation weight corresponding to the feature region with a large rotation amount is small, and the rotation weight corresponding to the feature region with a small rotation amount is large. There is no limitation here.

[0078] In one embodiment, the slice adjustment amount determination method further includes:

[0079] Based on the adjustment amount parts corresponding to different feature regions in the first adjustment amount, respectively determine multiple third adjustment amounts of the region of interest of the slice image with the current number compared to the slice image with the starting number;

[0080] Through the multiple third adjustment amounts, determine the target adjustment amount of the region of interest of the slice image with the current number compared to the slice image with the starting number.

[0081] That is, for each feature region, the offset and rotation amount corresponding to the feature region are determined in the first adjustment amount, and the determined offset and rotation amount are used as the adjustment amount part corresponding to the feature region; through the adjustment amount part of each feature region, a set of third adjustment amounts of the region of interest of the current numbered slice image compared to the starting numbered slice image are respectively calculated, and thus multiple third adjustment amounts can be obtained. Optionally, each adjustment amount part can be directly determined as a third adjustment amount.

[0082] In the case of determining multiple third adjustment amounts, the region of interest images of the current numbered slice image are respectively adjusted according to the multiple third adjustment amounts; then, by comparing the adjusted region of interest images with the region of interest images of the starting numbered slice image, the credibility of each third adjustment amount is determined; based on the credibility of each third adjustment amount, one or more third adjustment amounts are selected to determine the target adjustment amount. This calculation method for determining the target adjustment amount can be understood as a calculation method with a higher calculation amount.

[0083] In one embodiment, determining the target adjustment amount of the region of interest of the current numbered slice image compared to the starting numbered slice image through multiple third adjustment amounts includes:

[0084] Adjusting the region of interest images of the current numbered slice image respectively according to the multiple third adjustment amounts to obtain multiple verification graphs;

[0085] Respectively taking each verification graph as the target verification graph, and performing similarity verification based on the overlapping part between the target verification graph and the region of interest image of the starting numbered slice image to determine the similarity verification result;

[0086] Based on the similarity verification results corresponding to the multiple verification graphs, select one from the multiple third adjustment amounts as the target adjustment amount, or determine the weights of each third adjustment amount and obtain the target adjustment amount by weighted averaging the multiple third adjustment amounts based on the determined weights.

[0087] That is, the region of interest images of the current numbered slice image are respectively offset and rotated according to each third adjustment amount to obtain multiple verification graphs; respectively taking each verification graph as the target verification graph, comparing the overlapping part between the target verification graph and the region of interest image of the starting numbered slice image, and determining the similarity of the overlapping part as the similarity verification result; sorting the similarities corresponding to the multiple verification graphs, determining the third adjustment amount of the verification graph with the highest similarity as the target adjustment amount, or, performing weight division according to the similarity magnitude, the third adjustment amount corresponding to the verification graph with a larger similarity has a larger weight, and then weighted averaging each third adjustment amount to obtain the target adjustment amount.

[0088] Embodiment 2

[0089] Figure 2It is a flowchart of a method for determining a slice scanning area according to Embodiment 2 of the present invention. This embodiment is applicable to the situation of determining a slice scanning area. This method can be executed by a slice scanning area determination device, which can be implemented in the form of software and / or hardware and integrated in an electronic device. Further, the electronic device includes, but is not limited to: a computer, a laptop computer, etc.

[0090] It should be noted that the target adjustment amount involved in this embodiment is the target adjustment amount of the region of interest of the slice image with the current number compared to the starting number slice image determined by the slice adjustment amount determination method of Embodiment 1. Contents not elaborated in this embodiment can be referred to Embodiment 1.

[0091] As Figure 2 shown, the method includes:

[0092] S210. Based on obtaining a continuous plurality of slice images, in response to a region selection operation, determine the region of interest and the feature region in the starting number slice image, and determine the region of interest and the feature region in the ending number slice image.

[0093] The region selection operation can be an operation for selecting the region of interest and the feature region in the starting number and ending number slice images. For example, it can be a box selection operation for a certain region in the slice image; or it can be a click operation for a certain point in the slice image, and a rectangular region with a set size centered on this point is determined as the region to be selected.

[0094] In this step, in response to the region selection operation, the region of interest and the feature region can be selected in the starting number slice image; the selection of the region of interest in the starting number slice image can be based on actual application needs and is not limited here; for the selection of the feature region in the starting number slice image, by comparing the starting number and ending number slice images, a feature region with obvious common features in the two slice images can be selected.

[0095] In one embodiment, the selection of the feature region in the starting number slice image at least satisfies the following conditions: the contrast between the selected feature region and the background in the starting number slice image exceeds the contrast threshold, the edge recognition degree exceeds the recognition threshold, and it has uniqueness; the selected feature region is a common feature region of the starting number slice image and the ending number slice image, and the structural similarity between the starting number slice image and the ending number slice image exceeds the similarity threshold, and the adjustment amount is lower than the adjustment amount threshold; when there are multiple selected feature regions, the multiple feature regions show a non-aggregated distribution state in the starting number slice image.

[0096] In the starting-number sliced image, the selected feature region should have a pattern structure with obvious features. The requirements for obvious features are that the contrast between the pattern structure and the background is large (i.e., the contrast between the selected feature region and the background exceeds the contrast threshold), the edges are easy to identify (i.e., the edge recognition degree exceeds the recognition threshold), and it is significantly different from other structures in the nearby or entire sliced region (i.e., this pattern structure is unique and there is no repetition). The selected feature region is the feature region common to the starting-number and ending-number sliced images, that is, the basically same structure in the two sliced images, such as a special structure with small structural changes in different layers (i.e., the structural similarity exceeds the similarity threshold). The position offset or rotation amplitude of the selected feature region in the starting-number and ending-number sliced images should be as small as possible (i.e., the adjustment amount is lower than the adjustment amount threshold). The selected feature regions are multiple and should be distributed as much as possible to avoid clustering together (i.e., non-clustered distribution).

[0097] In the case of determining the region of interest and the feature region in the starting-number sliced image, in response to the region selection operation, in the ending-number sliced image, a region with basically the same position and features as the region of interest in the starting-number sliced image can be selected as the region of interest in the ending-number sliced image; a region with basically the same position and features as the feature region in the starting-number sliced image can also be selected as the feature region in the ending-number sliced image.

[0098] S220. Based on the region of interest and the feature region in the starting-number sliced image, determine the region of interest and the feature region in the current-number sliced image through an image feature automatic matching algorithm, where the current number is the number between the starting number and the ending number.

[0099] Since each slice is numbered and globally positioned in the first embodiment, the position of the center point of the region of interest in the starting-number sliced image relative to the whole slice can be determined. Through the image feature automatic matching algorithm, based on the position of the center point of the region of interest in the starting-number sliced image relative to the whole slice and combined with the position mapping relationship, the position of the center point of the region of interest in the current-number sliced image can be determined. In the current-number sliced image, search outward centered on the center point of the region of interest in the current-number sliced image to determine the region similar to the region of interest in the starting-number sliced image in the current-number sliced image as the region of interest in the current-number sliced image. The above process of position mapping can be understood as aligning the edges of the starting-number sliced image and the current-number sliced image and copying the border of the region of interest in the starting-number sliced image to the current-number sliced image to obtain the region of interest in the current-number sliced image.

[0100] The determination of the feature region in the current numbered slice image is basically the same as the determination method of the region of interest in the current numbered slice image, which will not be elaborated here.

[0101] It should be noted that through the above S210 and S220, the regions of interest and feature regions of each slice image in a continuous plurality of slice images can be determined, so as to scan the region of interest to obtain a region-of-interest image and scan the feature region to obtain a feature-region image, that is, the region-of-interest images and feature-region images of each slice image can be obtained.

[0102] S230. Based on the target adjustment amount of the region of interest of the current numbered slice image compared with the starting numbered slice image, adjust the region of interest of the current numbered slice image to obtain a target scanning region, and the target adjustment amount is determined by a slice adjustment amount determination method.

[0103] In this step, in the case of determining the region of interest of the current numbered slice image, based on the target adjustment amount of the region of interest of the current numbered slice image compared with the starting numbered slice image determined in Embodiment 1, adjust the region of interest of the current numbered slice image, and determine the adjusted region of interest as the target scanning region, so as to scan through the target scanning region to obtain the region-of-interest image corresponding to the current numbered slice image for three-dimensional reconstruction. Among them, the adjustment of the region of interest of the current numbered slice image can be understood as the adjustment of the rotation and offset of the slice of the current number.

[0104] The technical solution of the embodiment of the present invention facilitates the acquisition of the region-of-interest images and feature-region images of each slice image by determining the regions of interest and feature regions of each slice image in a continuous plurality of slice images, so as to determine the target adjustment amount; based on the target adjustment amount, the automatic adjustment of the region of interest of the current numbered slice image is realized, and the current numbered slice image is registered with the starting numbered slice image in position, which not only helps to improve the spatial accuracy of three-dimensional reconstruction, but also can greatly improve work efficiency and reduce the labor intensity of operators.

[0105] In one embodiment, before determining the region of interest and feature region in the terminating numbered slice image in response to a region selection operation, it further includes: generating a recommended region of interest and feature region in the terminating numbered slice image based on the region of interest and feature region in the starting numbered slice image through an image feature automatic matching algorithm.

[0106] Through the image feature automatic matching algorithm, based on the regions of interest and feature regions in the starting numbered slice image, the regions of interest and feature regions in the recommended ending numbered slice image are generated, which is basically the same as the method for determining the regions of interest and feature regions in the current numbered slice image described above, and will not be elaborated here. In the case of automatically generating the regions of interest and feature regions in the recommended ending numbered slice image, through region selection operations, the recommended regions of interest and feature regions can be manually confirmed or fine-tuned to determine the regions of interest and feature regions in the ending numbered slice image.

[0107] After generating the regions of interest and feature regions in the recommended ending numbered slice image and then performing manual confirmation or fine-tuning through the operator's region selection operations, it can make the determination of the regions of interest and feature regions in the ending numbered slice image more accurate, and can also verify whether the feature regions selected by the operator in the starting numbered slice image are appropriate.

[0108] If in the automatically recommended ending numbered slice image, the pattern structure of the feature region is not easily recognizable or changes too much compared to the pattern structure of the starting numbered slice image, then the feature regions selected by the operator in the starting numbered slice image may be incorrect; if there are feature regions with repeated structures in the automatically recommended ending numbered slice image, then the feature regions selected by the operator in the starting numbered slice image may be incorrect.

[0109] As for the obvious feature pattern structure with a large position offset or rotation amplitude, or multiple feature regions being too concentrated, it usually does not affect the determination of the regions of interest and feature regions in the ending numbered slice image; however, a large position offset or rotation amplitude of the obvious feature pattern structure will increase the computational amount when determining the position and rotation adjustment amount of the current numbered slice; when multiple feature regions are too concentrated, it is possible that the feature regions have rotation or offset, while the region of interest has no rotation or offset, but the rotation or offset of the feature regions will misjudge that the region of interest also has rotation or offset.

[0110] Embodiment III

[0111] The embodiments of the present invention are exemplary descriptions of the above embodiments. The embodiments of the present invention provide a precise positioning method for the regions of interest in slice scanning. Taking biological slices as an example, it specifically includes the following content:

[0112] First, it is necessary to obtain the slice images of multiple consecutive biological slices.

[0113] In practical applications, biological slices of different layers belonging to the same sample are usually arranged in sequence on the sample stage of a scanning electron microscope. For example, the biological slices are arranged in several columns from top to bottom and from left to right on the sample stage in order from the upper layer to the lower layer. A panoramic image of the sample stage (usually an optical image) is obtained at a first magnification ratio (the lowest or a relatively low magnification ratio). Based on the panoramic image, the biological slices to be scanned are selected. For example, 500 biological slices to be scanned are selected.

[0114] When selecting slices, due to the large color contrast between the slices and the background color of the sample stage, the edges of the selected slices can be automatically recognized through this contrast (see Patent CN119599867A in detail), so as to number each slice (for subsequent scanning operations in the order of the numbers) and perform overall positioning (so that the sample stage can automatically move the slice corresponding to a certain number under the objective lens).

[0115] Suppose an operator selects 500 biological slices to be scanned. Then, slice images of the 1st to the 500th biological slices need to be obtained by scanning at a second magnification ratio, that is, slice images of multiple consecutive biological slices are obtained. The scanning order can be in the order of the slice numbers or not in the order of the slice numbers, which is not limited here.

[0116] Secondly, determine the region of interest and the feature region of the 1st slice image.

[0117] In response to the region selection operation, determine the region of interest and the feature region of the 1st slice image. The region of interest of the 1st slice image is the region to be three-dimensionally reconstructed in the scanning operation. The selection of the feature region of the 1st slice image needs to be determined by comparing the 1st and the 500th slice images.

[0118] Figure 3 It is a schematic diagram of the comparison of slice images provided in Embodiment 3 of the present invention. As Figure 3 shown, display the 1st and the 500th slice images on the same page so that the operator can compare the two slice images, and then select the feature region.

[0119] The selection of the feature region of the 1st slice image needs to meet the following requirements:

[0120] There are distinct pattern structures in the feature regions. The requirement for distinctness is that the pattern structures have a large contrast with the background and their edges are easy to identify, being significantly different from other structures in the nearby or entire slice region (i.e., the pattern structures are unique and there are no repeated identical cases); in the 1st and 500th images, the distinct pattern structures have basically the same structure (i.e., a special biological structure with small structural changes in different layers of the biological section), and the higher the similarity of the structures, the better; in the 1st and 500th images, the position offset or rotation amplitude of the distinct pattern structures is as small as possible; when there are multiple feature regions, they should be distributed as widely as possible to avoid clustering together.

[0121] In the case of obtaining biological section images by magnifying a million times with a scanning electron microscope, the biological structures (i.e., feature regions) suitable as image registration feature sites need to meet the following conditions: high stability, small morphological changes, easy to identify, and widely distributed. These structures are usually located inside cells or at the subcellular level and are relatively stable organs or tissues within cells. The following are several common intracellular organs or tissues of fine microorganisms suitable as feature sites:

[0122] 1. Mitochondria

[0123] Mitochondria are the energy factories within cells, having a double-membrane structure, and their morphology is usually oval or rod-shaped. In regions where mitochondria are dense (such as muscle cells or neurons), their distribution and morphology are relatively stable. Mitochondria are numerous and widely distributed, and are easily captured by a scanning electron microscope. In sections of different layers, the shape and position of mitochondria change little, making them suitable as stable feature points. It is applicable to the study of metabolically active cell types.

[0124] 2. Ribosomes

[0125] Ribosomes are small granular structures responsible for protein synthesis within cells, with a diameter of about 20 - 30 nanometers. They can float freely in the cytoplasm or attach to the endoplasmic reticulum. The size and morphology of ribosomes hardly change in sections of different layers. Their density of existence is high, and they are easy to form an obvious contrast in scanning electron microscope images. Since ribosomes are small and vulnerable to the environment, they may need to be used in combination with other feature points.

[0126] 3. Centrosome

[0127] The centrosome is a microtubule organizing center that plays a key role in the process of cell division, consisting of two mutually perpendicular centrioles surrounded by a dense protein matrix. The structure of the centrosome is unique and stable, and it is easy to identify in consecutive sections. Its geometric shape is regular and suitable as a reference point for precise registration. It is applicable to the study of cell division-related processes.

[0128] 4. Golgi apparatus

[0129] The Golgi apparatus is a complex structure composed of multiple flat vesicles that are primarily involved in the processing and secretion of proteins and lipids. The vesicles of the Golgi apparatus are clearly visible and morphologically stable. Local structural variations are minimal across different slices, providing reliable registration. This makes it suitable for studying cells with active secretory functions.

[0130] 5. Endoplasmic reticulum

[0131] The endoplasmic reticulum (ER) is divided into rough ER and smooth ER. The former has numerous ribosomes attached to its surface, while the latter is smooth and free of particles. The network-like structure of the ER is highly coherent in serial sections. The distribution of ribosomes in the rough ER, in particular, provides additional markers. This makes it suitable for studying cells involved in protein synthesis and transport.

[0132] 6. Lysosomes

[0133] Lysosomes are the digestive "workshops" within cells, enclosed by a single membrane and containing a variety of hydrolytic enzymes. Lysosomes are relatively uniform in size and morphology, and are evenly distributed. Their position varies minimally across slices, making them suitable as stable features. They are well-suited for studying cellular autophagy or degradation processes.

[0134] 7. Microtubules and microfilaments

[0135] Microtubules and microfilaments are essential components of the cytoskeleton, composed of polymers of α- / β-tubulin and actin, respectively, forming the intracellular support and transport network. The fibrous structures of microtubules and microfilaments exhibit a high degree of continuity and regularity in serial sections. Their arrangement serves as a natural grid for precise positioning, making them suitable for studying the mechanisms that maintain cellular morphology and transport.

[0136] 8. Chromatin (structure within the nucleus)

[0137] Chromatin is a complex composed of deoxyribonucleic acid (DNA) and histone proteins, presenting as filamentous or blocky structures distributed within the cell nucleus. The distribution pattern of chromatin maintains a certain degree of consistency across different slices. Its complex morphology provides rich texture information, which can be used for feature matching. It is suitable for studying gene expression regulation within the cell nucleus.

[0138] 9. Cell membrane and its accessory structures

[0139] The cell membrane is composed of a phospholipid bilayer, which may have microvilli, corrugations, or other accessory structures. The cell membrane's edges are clearly defined and easily identified under a scanning electron microscope. The presence of specific accessory structures on the cell membrane, such as synaptic vesicles or microvilli, can further enhance registration accuracy. This technique is suitable for studying intercellular communication or substance exchange processes.

[0140] Figure 4 This is a schematic diagram of the region selection operation in a slice image provided according to Embodiment 3 of the present invention. As Figure 4 shown, in the first slice image, the region of interest P1 and three feature regions A, B, and C can be determined through the region selection operation.

[0141] Then, determine the region of interest and feature regions of the 500th slice image.

[0142] After the ROI and 3 feature regions are determined in the first slice image, the ROI can be automatically searched for and determined on the 500th slice image through the image feature automatic matching algorithm. Since each biological slice is numbered and globally positioned above, the position of the center point of the ROI in the first slice image relative to the overall slice is known. According to the position mapping relationship, the position of the center point of the ROI in the 500th slice image can be known. Search outward from the center point of the ROI for the image region similar to the ROI in the 500th slice image and the first slice image, and the region of interest of the automatically recommended 500th slice image can be found. The determination of the 3 feature regions of the 500th slice image is the same as the automatic search method of the ROI of the 500th slice image.

[0143] According to the automatic search results: on the one hand, manual confirmation or fine-tuning is still required. The more accurate the ROI and feature regions determined for the 500th slice image are, the more accurate the automatic selection of the ROI and feature regions of subsequent other slice images will be; on the other hand, it can be verified whether the feature regions selected by the operator in the first slice image are appropriate. If the selection is inappropriate, for example:

[0144] If the pattern structure is not easily recognizable or the pattern structure changes too much, the feature regions automatically selected in the 500th slice image may be incorrect; if there are repetitive structures, more than 3 feature regions may be determined in the 500th slice image.

[0145] As for the obvious pattern structure with a large position offset or rotation amplitude, or the excessive aggregation of multiple feature regions, it generally does not affect the automatic search for and determination of the ROI and 3 feature regions on the 500th slice image; however, if the obvious pattern structure has a large position offset or rotation amplitude, it will increase the calculation amount when judging the position and rotation adjustment amount of the intermediate slices (the 2nd to the 499th slice); if multiple feature regions are overly aggregated, there may be a rotation or offset of the feature regions, while the ROI has no rotation or offset, but the rotation or offset of the feature regions will misjudge that the ROI also has a rotation or offset.

[0146] Then, determine the regions of interest and feature regions of the 2nd to the 499th slice images.

[0147] Through the automatic image feature matching algorithm, based on the position of the center point of the region of interest in the first slice image relative to the whole slice, combined with the position mapping relationship, the position of the center point of the region of interest in the current numbered slice image can be determined; in the current numbered slice image, search outward centered on the center point of the region of interest in the current numbered slice image to determine the region similar to the region of interest in the first slice image in the current numbered slice image as the region of interest in the current numbered slice image. The above process of position mapping can be understood as aligning the edges of the first slice image and the current numbered slice image, and copying the border of the region of interest in the first slice image to the current numbered slice image to obtain the region of interest in the current numbered slice image. The determination of the feature region in the current numbered slice image is basically the same as the determination method of the region of interest in the current numbered slice image, which will not be elaborated here. The current number is any one of 2 to 499.

[0148] Then, obtain the region of interest images and feature region images of the first to the 500th slice images.

[0149] In the above-described alignment of the edges of the first slice image and the current numbered slice image to determine the region of interest and feature region in the current numbered slice image, the positioning adjustment of the ROI has been involved, that is, it is necessary to determine how to correctly place the current numbered slice under the objective lens so that the objective lens can accurately observe the ROI or its nearby position, which involves the moving direction and distance of the sample stage (moving the current numbered slice to the position where the first slice is located - under the objective lens), and also involves the rotation direction and amplitude of the sample stage (because there are inevitable differences in the placement postures of the current numbered slice and the first slice). The specific implementation of the above process can be seen in Patent CN119599867A.

[0150] However, the above positioning adjustment is a rough adjustment, mainly based on the images obtained at the first magnification (the lowest or relatively low magnification) for positioning adjustment. At the third magnification (the third magnification > the second magnification > the first magnification) during the actual scanning operation, there are still large offsets and rotations in the determined ROI region, resulting in the region of interest finally scanned being not suitable for three-dimensional reconstruction; the purpose of this application is to provide a semi-automatic method for precise positioning of the slice ROI based on the rough positioning of the slice.

[0151] Based on the first magnification, after obtaining the positions and directions of the ROI and three feature regions in the first to the 500th slice images relative to the whole slice, pre-scanning images for positioning of the ROI and three feature regions in the first to the 500th slice images (i.e., region of interest images and feature region images) are obtained by scanning at the second magnification.

[0152] Figure 5 It is a schematic diagram of a region of interest image and a feature region image provided according to Embodiment 3 of the present invention, Figure 5 for partial exemplary illustration. As Figure 5 shown, each row can correspond to a slice image, the P1 column can correspond to the region of interest image of the slice image, and the A, B, and C columns can correspond to the feature region images of the slice image for different feature regions.

[0153] Then, determine the target adjustment amount of the region of interest of the 2nd to 499th slice images relative to the region of interest of the 1st slice image.

[0154] Based on the registration technology of feature point detection, determine that in the currently numbered slice image, relative to the 1st slice image, the offsets of the three feature regions A, B, and C are D1, D2, and D3 respectively, and the rotation amounts are R1, R2, and R3 respectively, that is, determine the first adjustment amount of the feature region of the currently numbered slice image compared to the starting numbered slice image.

[0155] Based on the differences between the feature regions of the 1st and 500th slice images, predict that in the currently numbered slice image, relative to the 1st slice image, the offsets of the three feature regions A, B, and C are Δ1, Δ2, and Δ3 respectively, and the rotation amounts are θ1, θ2, and θ3 respectively, that is, predict the second adjustment amount of the feature region of the currently numbered slice image compared to the starting numbered slice image.

[0156] According to the first adjustment amount and the second adjustment amount, determine the target adjustment amount of the region of interest of the currently numbered slice image compared to the 1st slice image, including the following three methods, where the target adjustment amount includes the offset x and the rotation amount y of the region of interest.

[0157] (1) Calculation method with lower computational complexity:

[0158]

[0159]

[0160] (2) Calculation method with higher computational complexity:

[0161]

[0162]

[0163] Specifically, the weights a, b, and c can be obtained from the distances between the center of the ROI and the centers of the feature regions A, B, and C. That is, the offset weight of the feature region far from the center of the ROI is small, and the offset weight of the feature region close to the center of the ROI is large. The specific functional relationship can be designed by oneself; the weights e, f, and g can be obtained from the predicted rotation amounts of the feature regions A, B, and C. That is, the weight of the feature region with a large predicted rotation amount is small, and the weight of the feature region with a small predicted rotation amount is large. The specific functional relationship can be designed by oneself.

[0164] (3) Calculation method with higher computational complexity:

[0165] Based on the offset and rotation amounts of the feature region A of the current numbered slice image compared to the first slice image, the first set of offset and rotation amounts of the ROI are calculated; based on the offset and rotation amounts of the feature region B, the second set of offset and rotation amounts of the ROI are calculated; based on the offset and rotation amounts of the feature region C, the third set of offset and rotation amounts of the ROI are calculated; that is, multiple third adjustment amounts of the region of interest of the current numbered slice image compared to the starting numbered slice image are determined.

[0166] Offset and rotate the ROI in the current numbered slice image according to the above first to third sets of offset and rotation amounts to obtain the first to third verification images. Compare each verification image with the ROI in the first slice image respectively. It can be a comparison of partial overlapping regions. The partial overlapping regions can be Figure 4 illustrated by the overlapping region of the P1 green square and the blue square shown. The green square region is the ROI in the first slice image, and the blue square region is the ROI in the current numbered slice image adjusted according to the above first to third sets of offset and rotation amounts; because only this part of the overlapping region has similarity and comparability, the similarity between the first to third verification images and the ROI in the first slice image is determined after comparison.

[0167] Based on the determined similarity, determine which set of offset and rotation amounts to use to adjust the ROI in the current numbered slice, and determine the selected set of offset and rotation amounts as the target adjustment amount; or determine the weights of the first to third sets of offset and rotation amounts therefrom, and use the offset and rotation amounts obtained by weighting based on the determined weights as the target adjustment amount.

[0168] Finally, adjust the region of interest of the current numbered slice image based on the target adjustment amount to obtain the target scanning region, so as to scan through the target scanning region to obtain the region of interest image corresponding to the current numbered slice image for three-dimensional reconstruction.

[0169] The above description is for easy understanding. In actual applications, there is a large span between the first and 500th slice images. The ROIs and feature regions of the 1st, 100th, 200th, 300th, 400th, and 500th images can be obtained sequentially in an artificial-assisted manner; then the ROIs and feature regions of the 2nd to 99th, 101st to 199th, 201st to 299th, 301st to 399th, and 401st to 499th slice images are automatically obtained, and the offset and rotation adjustment amounts are calculated. In this way, the registration effect of each slice ROI image obtained is better.

[0170] This application provides a real-time correction mechanism for slice scanning positions, which can dynamically adjust the slice positions during scanning to reduce errors such as offset and rotation, has the ability to accurately locate the region of interest, and has the following advantages:

[0171] 1. High degree of automation, realizing semi-automatic image registration scanning operations, improving the scanning operation efficiency, and reducing the workload of operators.

[0172] 2. Existing registration methods often focus on the alignment of the overall image while ignoring the fine adjustment of local ROIs, which may lead to distortion of key structures in 3D reconstruction; while the solution provided by this application focuses on ROIs and has a strong ability to accurately locate ROIs.

[0173] 3. The solution of this application is not limited to 3D reconstruction of biological slices and can also be applied to batch scanning scenarios of other slices with similar requirements. However, this application makes full use of the characteristics of biological slice images (such as thin layer structure, minute features, etc.), so it is more targeted in the application of 3D reconstruction of biological slices.

[0174] Embodiment 4

[0175] Figure 6 It is a schematic structural diagram of a slice adjustment amount determination device provided according to Embodiment 4 of the present invention. This embodiment is applicable to the situation of determining the slice adjustment amount, such as Figure 6 shown. The specific structure of this device includes:

[0176] The first adjustment amount determination module 61 is used to determine the first adjustment amount of the feature region of the current numbered slice image compared to the starting numbered slice image based on the difference between the feature region images of consecutive multiple slice images, where the current number is a number between the starting number and the ending number;

[0177] The second adjustment amount determination module 62 is used to predict the second adjustment amount of the feature region of the current numbered slice image compared to the starting numbered slice image based on the difference between the feature region images of the starting numbered and ending numbered slice images;

[0178] A target adjustment amount determination module 63, configured to determine a target adjustment amount of a region of interest of a sliced image of the current number relative to the sliced image of the starting number according to the first adjustment amount and the second adjustment amount.

[0179] The sliced image adjustment amount determination device provided in this embodiment, based on the feature region images of multiple consecutive sliced images obtained by the first adjustment amount determination module, determines a first adjustment amount of the feature region of the sliced image of the current number relative to the feature region of the sliced image of the starting number based on the difference between the feature region images of the sliced image of the current number and the sliced image of the starting number, where the current number is a number between the starting number and the ending number; the second adjustment amount determination module predicts a second adjustment amount of the feature region of the sliced image of the current number relative to the feature region of the sliced image of the starting number based on the difference between the feature region images of the sliced image of the starting number and the sliced image of the ending number; the target adjustment amount determination module determines a target adjustment amount of the region of interest of the sliced image of the current number relative to the sliced image of the starting number according to the first adjustment amount and the second adjustment amount. This solution can automatically determine the target adjustment amount of the region of interest of each sliced image relative to the sliced image of the starting number, and this target adjustment amount is used to automatically adjust the sliced position to achieve position registration of sliced images of different layers, which not only helps to improve the spatial accuracy of three-dimensional reconstruction, but also can greatly improve work efficiency and reduce the labor intensity of operators.

[0180] Further, each sliced image includes multiple feature regions, the number of feature regions of different sliced images is the same and the features correspond, and the target adjustment amount determination module 63 is specifically configured to:

[0181] Respectively take each feature region as a target feature region, and determine a first offset amount and a first rotation amount of the first adjustment amount for the target feature region, and a second offset amount and a second rotation amount of the second adjustment amount for the target feature region;

[0182] Determine an offset amount difference corresponding to the target feature region based on the first offset amount and the second offset amount, and determine a rotation amount difference corresponding to the target feature region based on the first rotation amount and the second rotation amount;

[0183] Determine a target adjustment amount of the region of interest of the sliced image of the current number relative to the sliced image of the starting number according to the offset amount differences and rotation amount differences corresponding to each feature region.

[0184] Further, the target adjustment amount determination module 63 is specifically configured to:

[0185] Combine the result of averaging the offset amount differences corresponding to each feature region and the result of averaging the rotation amount differences corresponding to each feature region to determine as the target adjustment amount; or,

[0186] The result of taking the weighted average of the offset differences corresponding to each feature region and the result of taking the weighted average of the rotation differences corresponding to each feature region are combined and determined as the target adjustment amount.

[0187] Furthermore, the weights for taking the weighted average of the offset differences corresponding to each feature region are determined based on the distances between the center of the region of interest in the current numbered slice image and the centers of each feature region;

[0188] The weights for taking the weighted average of the rotation differences corresponding to each feature region are determined based on the rotation amounts for each feature region in the second adjustment amount.

[0189] Furthermore, the apparatus further includes:

[0190] A third adjustment amount determination module, configured to respectively determine a plurality of third adjustment amounts of the region of interest in the current numbered slice image compared to the slice image of the starting number based on the adjustment amount parts corresponding to different feature regions in the first adjustment amount;

[0191] A target adjustment amount determination module for the region of interest, configured to determine the target adjustment amount of the region of interest in the current numbered slice image compared to the slice image of the starting number through the plurality of third adjustment amounts.

[0192] Furthermore, the target adjustment amount determination module for the region of interest is specifically configured to:

[0193] Adjust the region of interest image of the current numbered slice image according to the plurality of third adjustment amounts respectively to obtain a plurality of verification images;

[0194] Respectively use each verification image as a target verification image, perform similarity verification based on the overlapping part between the target verification image and the region of interest image of the slice image of the starting number, and determine the similarity verification result;

[0195] Based on the similarity verification results corresponding to the plurality of verification images, select one of the plurality of third adjustment amounts as the target adjustment amount, or determine the weights of the third adjustment amounts and obtain the target adjustment amount by weighted averaging the third adjustment amounts based on the determined weights.

[0196] The slice adjustment amount determination apparatus provided by the embodiments of the present invention can execute the slice adjustment amount determination method provided by the embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the method.

[0197] Embodiment Five

[0198] Figure 7 It is a structural schematic diagram of a slice scanning region determination apparatus provided according to Embodiment Five of the present invention. This embodiment is applicable to the situation of determining the slice scanning region, such as Figure 7As shown, the specific structure of the device includes:

[0199] A region selection module 71, configured to, based on obtaining a plurality of consecutive slice images, in response to a region selection operation, determine the region of interest and the feature region in the slice image with the starting number, and determine the region of interest and the feature region in the slice image with the ending number;

[0200] A region matching module 72, configured to, through an image feature automatic matching algorithm, based on the region of interest and the feature region in the slice image with the starting number, determine the region of interest and the feature region in the slice image with the current number, where the current number is the number between the starting number and the ending number;

[0201] A region adjustment module 73, configured to, based on the target adjustment amount of the region of interest in the slice image with the current number compared to the slice image with the starting number, adjust the region of interest in the slice image with the current number to obtain a target scanning region, and the target adjustment amount is determined by a slice adjustment amount determining device.

[0202] The slice scanning region determination device provided in this embodiment, through the region selection module, based on obtaining a plurality of consecutive slice images, in response to a region selection operation, determines the region of interest and the feature region in the slice image with the starting number, and determines the region of interest and the feature region in the slice image with the ending number; through the region matching module, through an image feature automatic matching algorithm, based on the region of interest and the feature region in the slice image with the starting number, determines the region of interest and the feature region in the slice image with the current number, where the current number is the number between the starting number and the ending number; through the region adjustment module, based on the target adjustment amount of the region of interest in the slice image with the current number compared to the slice image with the starting number, adjusts the region of interest in the slice image with the current number to obtain a target scanning region, and the target adjustment amount is determined by a slice adjustment amount determining device. This solution, through the determination of the region of interest and the feature region of each slice image in a plurality of consecutive slice images, facilitates the acquisition of the region of interest image and the feature region image of each slice image, so as to determine the target adjustment amount; based on the target adjustment amount, the automatic adjustment of the region of interest in the slice image with the current number is realized, enabling the slice image with the current number to be registered with the slice image with the starting number in terms of position, which not only helps to improve the spatial accuracy of three-dimensional reconstruction, but also can greatly improve work efficiency and reduce the labor intensity of operators.

[0203] Further, the selection of the feature region in the slice image with the starting number satisfies at least the following conditions:

[0204] The selected feature region has a contrast with the background in the slice image with the starting number that exceeds the contrast threshold, an edge recognition degree that exceeds the recognition threshold, and is unique;

[0205] The selected feature region is the common feature region in the starting numbered slice image and the ending numbered slice image, and the structural similarity in the starting numbered slice image and the ending numbered slice image exceeds the similarity threshold, and the adjustment amount is lower than the adjustment amount threshold;

[0206] When there are multiple selected feature regions, the multiple feature regions show a non-aggregated distribution state in the starting numbered slice image.

[0207] Further, the device further includes a recommendation module for:

[0208] Before determining the region of interest and the feature region in the ending numbered slice image in response to the region selection operation, based on the region of interest and the feature region in the starting numbered slice image, generate the recommended region of interest and feature region in the ending numbered slice image through an image feature automatic matching algorithm.

[0209] The slice scanning region determination device provided by the embodiments of the present invention can execute the slice scanning region determination method provided by the embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the method.

[0210] Embodiment Six

[0211] Figure 8 It is a schematic structural diagram of an electronic device implementing the embodiments of the present invention. The electronic device aims to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0212] As Figure 8 shown, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. Among them, the memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0213] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disc, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0214] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the slice adjustment amount determination method or the slice scan area determination method.

[0215] In some embodiments, the slice adjustment amount determination method or the slice scan area determination method can be implemented as a computer program, which is tangibly included in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the slice adjustment amount determination method or the slice scan area determination method described above can be executed. Alternatively, in other embodiments, the processor 11 can be configured to execute the slice adjustment amount determination method or the slice scan area determination method by any other suitable means (e.g., by means of firmware).

[0216] The various embodiments of the systems and technologies described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special or general programmable processor, can receive data and instructions from a storage system, at least one input device, and at least one output device, and can transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0217] A computer program for implementing the method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, a special purpose computer, or other programmable data processing device, such that when the computer programs are executed by the processor, the functions / operations specified in the flowchart and / or block diagram are implemented. The computer programs can be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0218] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0219] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and the input received from the user can be in any form (including acoustic input, voice input, or tactile input).

[0220] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected with each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.

[0221] A computing system can include a client and a server. The client and the server are generally far from each other and typically interact through a communication network. The client-server relationship is created by computer programs that run on the respective computers and have a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0222] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is made herein.

[0223] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for determining a slice adjustment amount, characterized in that: include: Determining a first adjustment amount for the characteristic region of the slice image with the current number compared to the slice image with the starting number based on a difference between the characteristic region images of the slice image with the current number and the slice image with the starting number, on the basis of obtaining characteristic region images of a plurality of consecutive slice images, wherein the current number is a number between the starting number and the ending number; predicting a second adjustment amount of the characteristic region of the current numbered slice image compared to the starting numbered slice image based on a difference between the characteristic region images of the starting numbered slice image and the ending numbered slice image; A target adjustment amount of the region of interest of the slice image with the current number compared to the slice image with the starting number is determined according to the first adjustment amount and the second adjustment amount.

2. The method according to claim 1, characterized in that Each slice image includes a plurality of feature regions, and different slice images have the same number of feature regions and consistent feature correspondence. Determining a target adjustment amount for a region of interest of the slice image with the current number compared to the slice image with the starting number based on the first adjustment amount and the second adjustment amount includes: Taking each characteristic region as a target characteristic region, determining a first offset and a first rotation amount for the target characteristic region in the first adjustment amount, and a second offset and a second rotation amount for the target characteristic region in the second adjustment amount; Determine an offset difference corresponding to the target feature area based on the first offset and the second offset, and determine a rotation difference corresponding to the target feature area based on the first rotation and the second rotation; According to the offset difference and the rotation difference corresponding to each characteristic region, a target adjustment amount of the region of interest of the slice image with the current number compared with the slice image with the starting number is determined.

3. The method according to claim 2, characterized in that Determining a target adjustment amount of the region of interest of the slice image with the current number compared to the slice image with the starting number according to the offset difference and the rotation difference corresponding to each characteristic region includes: The target adjustment amount is determined by combining the average of the offset differences corresponding to each feature area and the average of the rotation differences corresponding to each feature area; or The target adjustment amount is determined by combining a weighted average of the offset differences corresponding to each feature area and a weighted average of the rotation differences corresponding to each feature area.

4. The method according to claim 3, characterized in that Taking a weighted average of the offset differences corresponding to the characteristic regions, and determining the weights based on the distances between the center of the region of interest in the current numbered slice image and the center of each characteristic region; The weight of the weighted average of the rotation amount differences corresponding to the characteristic regions is determined based on the rotation amount for each characteristic region in the second adjustment amount.

5. The method according to claim 1, wherein Also includes: determining, based on adjustment amount portions corresponding to different characteristic regions in the first adjustment amount, a plurality of third adjustment amounts for regions of interest of the slice image with the current number compared to the starting number; A target adjustment amount of the region of interest of the slice image with the current number compared with the slice image with the starting number is determined through a plurality of third adjustment amounts.

6. The method according to claim 5, characterized in that Determining a target adjustment amount of the region of interest of the slice image with the current number compared to the slice image with the starting number by using a plurality of third adjustment amounts includes: Adjusting the region of interest image of the current numbered slice image according to a plurality of third adjustment amounts to obtain a plurality of calibration images; Taking each verification image as a target verification image, performing a similarity verification based on an overlapping portion of the target verification image and the image of the region of interest of the starting numbered slice image, and determining a similarity verification result; Based on the similarity verification results corresponding to the multiple verification images, one is selected from the multiple third adjustment amounts as the target adjustment amount, or the weights of the third adjustment amounts are determined and the target adjustment amount is obtained by weighted averaging the multiple third adjustment amounts based on the determined weights.

7. A method for determining a slice scanning area, characterized in that: include: Based on the acquisition of a plurality of consecutive slice images, in response to a region selection operation, determining a region of interest and a feature region in a slice image with a starting number, and determining a region of interest and a feature region in a slice image with an ending number; Determine the region of interest and the feature region in the slice image with the current number based on the region of interest and the feature region in the slice image with the starting number by an automatic image feature matching algorithm, wherein the current number is a number between the starting number and the ending number; Based on the target adjustment amount of the region of interest of the current numbered slice image compared to the starting numbered slice image, the region of interest of the current numbered slice image is adjusted to obtain a target scanning area, and the target adjustment amount is determined by the slice adjustment amount determination method according to any one of claims 1-6.

8. The method according to claim 7, characterized in that The selection of feature areas in the starting number slice image must meet at least the following conditions: The contrast between the selected feature area and the background in the starting number slice image exceeds the contrast threshold, the edge recognition exceeds the recognition threshold, and the feature area is unique; The selected feature region is a feature region common to the start number slice image and the end number slice image, and the structural similarity between the start number slice image and the end number slice image exceeds a similarity threshold, and the adjustment amount is lower than an adjustment amount threshold; When multiple characteristic regions are selected, the multiple characteristic regions are in a non-clustered distribution state in the starting numbered slice image.

9. The method according to claim 7, characterized in that Before determining the region of interest and the feature region in the slice image with the termination number in response to the region selection operation, the method further includes: The image feature automatic matching algorithm is used to generate the recommended region of interest and feature region in the ending slice image based on the region of interest and feature region in the starting slice image.

10. A device for determining a slice adjustment amount, characterized in that: include: a first adjustment amount determination module configured to determine, based on the obtained characteristic area images of a plurality of consecutive slice images, a first adjustment amount of the characteristic area of the slice image with the current number compared to the characteristic area images of the slice image with the starting number, based on a difference between the characteristic area images of the slice image with the current number and the slice image with the starting number, wherein the current number is a number between the starting number and the ending number; a second adjustment amount determination module, configured to predict a second adjustment amount of the characteristic area of the current numbered slice image compared to the starting numbered slice image based on a difference between the characteristic area images of the starting numbered slice image and the ending numbered slice image; The target adjustment amount determination module is configured to determine a target adjustment amount of the region of interest of the slice image having the current number compared to the starting number according to the first adjustment amount and the second adjustment amount.

11. A device for determining a slice scanning area, characterized in that: include: A region selection module is used to determine the region of interest and the feature region in the slice image with the starting number, and to determine the region of interest and the feature region in the slice image with the ending number in response to a region selection operation based on obtaining a plurality of consecutive slice images; A region matching module is configured to determine the region of interest and the feature region in the slice image with the current number based on the region of interest and the feature region in the slice image with the starting number by using an automatic image feature matching algorithm, wherein the current number is a number between the starting number and the ending number; A region adjustment module is used to adjust the region of interest of the current numbered slice image to obtain a target scanning region based on a target adjustment amount of the region of interest of the current numbered slice image compared to the starting numbered slice image, wherein the target adjustment amount is determined by the slice adjustment amount determination device according to claim 10.

12. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 1 to 6, or to execute the method according to any one of claims 7 to 9.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images