Image processing method and device, equipment and medium

Abstract

The invention relates to an image processing method and device, equipment and a medium, and the method comprises the steps: obtaining a first image and a second image which comprise a plurality of trajectory lines and a plurality of trajectory points; and then selecting a first trajectory in the first image, and rotating the first image based on an included angle between the first trajectory and the horizontal direction, namely rotating the first image to the horizontal direction. The scaling ratios of the first image and the second image which are obtained initially may be different, so that the first image needs to be zoomed in or out to obtain the third image, and the third image and the second image have the same proportion. A target offset is then determined based on the second image and the third image, and the third image is moved based on the target offset. According to the image processing method provided by the embodiment of the invention, the first image and the second image can be registered at pixel-level precision, and the accuracy of image processing is improved.

Description

Image processing method, device, equipment and medium Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, apparatus, device and medium. Background Art

[0002] Gene expression refers to the process by which genetic information from genes is synthesized into functional gene products. The products of gene expression are typically proteins, and all known organisms utilize gene expression to synthesize macromolecules. In spatial gene expression analysis, image registration is required to align gene expression images with the actual spatial location of biological samples to obtain accurate gene expression information.

[0003] Existing registration technologies can roughly align the positions of microscope images and gene expression visualization images, which are generated using a gene expression matrix containing spatial position information. By leveraging the image's inherent characteristics and supplementing with manual visual adjustments, the spatial alignment between the microscope image and the gene expression visualization image can be achieved. However, existing registration technologies can only achieve positional alignment within the relatively macroscopic dimension of biological samples, resulting in low registration accuracy and inability to support more refined gene expression analysis.

[0004] Summary of the Invention

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the present application provides an image processing method, apparatus, device and medium to improve the accuracy of image processing.

[0006] In a first aspect, an embodiment of the present application provides an image processing method, the method comprising:

[0007] Acquire a first image and a second image, wherein the first image and the second image include a plurality of trajectory lines and / or a plurality of trajectory points, wherein the trajectory point is formed by the intersection of two trajectory lines, and each of the plurality of trajectory lines has a corresponding index number;

[0008] Acquire a first trajectory line in the first image, and rotate the first image based on an angle between the first trajectory line and a horizontal direction;

[0009] Enlarging or reducing the first image to obtain a third image, wherein the third image has the same proportion as the second image;

[0010] determining a target offset based on the second image and the third image;

[0011] The third image is moved based on the target offset.

[0012] In a possible implementation, the step of enlarging or reducing the first image to obtain the third image includes:

[0013] Acquire a second trajectory line in the first image, and determine a first distance based on the first trajectory line and the second trajectory line;

[0014] Acquire a third trajectory line and a fourth trajectory line in the second image, and determine a second spacing based on the third trajectory line and the fourth trajectory line, wherein the index number of the third trajectory line is the same as the index number of the first trajectory line, and the index number of the fourth trajectory line is the same as the index number of the second trajectory line;

[0015] A ratio value is determined based on the first spacing and the second spacing, and the first image is enlarged or reduced according to the ratio value to obtain a third image.

[0016] In a possible implementation, the step of enlarging or reducing the first image to obtain the third image includes:

[0017] enlarging or reducing the first image to obtain a fourth image, wherein the fourth image has the same proportion as the second image;

[0018] The fourth image is moved according to a preset period and a preset rotation angle to obtain the third image.

[0019] In a possible implementation, the step of enlarging or reducing the first image to obtain the fourth image includes:

[0020] enlarging or reducing the first image to obtain a fifth image, wherein the fifth image has the same ratio as the second image;

[0021] respectively calculating the first barycentric coordinates of the fifth image and the second barycentric coordinates of the second image;

[0022] determining a center of gravity offset based on the first center of gravity coordinate and the second center of gravity coordinate;

[0023] The fifth image is moved based on the center of gravity offset to obtain the fourth image.

[0024] In a possible implementation, the step of moving the fourth image according to a preset period and a preset rotation angle to obtain the third image includes:

[0025] Obtaining the preset period and multiple preset rotation angles;

[0026] For any preset rotation angle, determining a first offset based on the fourth image and the second image;

[0027] Determine a plurality of second offsets corresponding to the first offset within the preset period, wherein each second offset in the plurality of second offsets corresponds one-to-one to each period within the preset period;

[0028] For any second offset, moving the fourth image based on the second offset to obtain a sixth image;

[0029] determining a first similarity between the sixth image and the second image;

[0030] Determining a first maximum similarity based on the multiple preset rotation angles and the first similarity corresponding to each period;

[0031] Determine that a sixth image corresponding to the first maximum similarity is the third image.

[0032] In a possible implementation, determining the target offset based on the second image and the third image includes:

[0033] determining a second offset corresponding to the third image as the target offset;

[0034] The moving the third image based on the target offset comprises:

[0035] The third image is moved based on the preset rotation angle corresponding to the third image and the target offset.

[0036] In a possible implementation, the step of moving the fourth image according to a preset period and a preset rotation angle to obtain the third image includes:

[0037] Acquiring the preset period and the plurality of preset rotation angles;

[0038] For any preset rotation angle, the fourth image is moved according to each period within the preset period to obtain a plurality of seventh images, each seventh image in the plurality of seventh images corresponding one-to-one to each period within the preset period;

[0039] For any seventh image, determining a second similarity between the seventh image and the second image;

[0040] Determining a second maximum similarity based on the multiple preset rotation angles and the second similarity corresponding to each period;

[0041] Determine that the seventh image corresponding to the second maximum similarity is the third image.

[0042] In a possible implementation, determining the target offset based on the second image and the third image includes:

[0043] determining a plurality of third offsets based on the second image and the third image;

[0044] determining the target offset based on the plurality of third offsets;

[0045] The moving the third image based on the target offset comprises:

[0046] The third image is moved based on the preset rotation angle corresponding to the third image and the target offset.

[0047] In a possible implementation, determining the first offset based on the fourth image and the second image includes:

[0048] Acquire multiple groups of trajectory point sets, each group of the multiple trajectory point sets includes a first trajectory point and a second trajectory point, the first trajectory point is located in the fourth image, the second trajectory point is located in the second image, and the first trajectory point and the second trajectory point have the same index identifier;

[0049] For any set of trajectory point sets, determining a fourth offset based on the coordinates of the first trajectory point and the coordinates of the second trajectory point in the trajectory point set;

[0050] The first offset is determined based on fourth offsets corresponding to the multiple groups of trajectory point sets.

[0051] In a possible implementation, when the second image has multiple sub-images, the trajectory line of each sub-image has the same index number, and determining the fourth offset based on the coordinates of the first trajectory point and the coordinates of the second trajectory point in the trajectory point set includes:

[0052] determining the second track point in each sub-image based on the index identifier of the first track point, wherein the first track point and the second track point have the same index identifier;

[0053] determining a Euclidean distance between the first trajectory point and any second trajectory point of the plurality of second trajectory points;

[0054] Determining a second trajectory point corresponding to a minimum Euclidean distance among the plurality of Euclidean distances;

[0055] A fourth offset is determined based on the coordinates of the first trajectory point and the coordinates of the second trajectory point corresponding to the minimum Euclidean distance.

[0056] In a possible implementation manner, determining the first similarity between the sixth image and the second image includes:

[0057] Performing frequency domain transformation on the sixth image to obtain a first frequency domain image;

[0058] performing a frequency domain transform on the second image to obtain a second frequency domain image;

[0059] Calculating a Hamming distance between the first frequency domain image and the second frequency domain image as a first similarity;

[0060] The determining a first maximum similarity based on the plurality of preset rotation angles and the first similarity corresponding to each period includes:

[0061] Based on the multiple preset rotation angles and the Hamming distance corresponding to each period, a minimum Hamming distance is determined as the first maximum similarity.

[0062] In a possible implementation, moving the fourth image according to each period within the preset period to obtain the seventh image includes:

[0063] Calculating the third centroid coordinates of the fourth image;

[0064] The fourth image is moved according to each of the periods, with the period where the third center of gravity coordinates are located as the center, to obtain the seventh image.

[0065] In a possible implementation, determining a plurality of third offsets based on the second image and the third image includes:

[0066] Acquire multiple groups of trajectory point sets, each group of the multiple trajectory point sets includes a third trajectory point and a fourth trajectory point, the third trajectory point is located in the third image, the fourth trajectory point is located in the second image, and the third trajectory point and the fourth trajectory point have the same index identifier;

[0067] For any set of trajectory point sets, a third offset is determined based on the coordinates of the third trajectory point and the coordinates of the fourth trajectory point in the trajectory point set.

[0068] In a possible implementation, after moving the third image based on the target offset, the method further includes:

[0069] acquiring, based on the moved third image and the second image, a plurality of sets of trajectory point sets, each set of the plurality of trajectory point sets comprising a fifth trajectory point and a sixth trajectory point, the fifth trajectory point being located in the moved third image, the sixth trajectory point being located in the second image, and the fifth trajectory point having the same index identifier as the sixth trajectory point;

[0070] For any set of trajectory point sets, determining a fifth offset based on the coordinates of the fifth trajectory point and the coordinates of the sixth trajectory point in the trajectory point set;

[0071] determining a final offset based on fifth offsets corresponding to the plurality of sets of trajectory points;

[0072] The shifted third image is shifted based on the final offset.

[0073] In a possible implementation, acquiring the second image includes:

[0074] An eighth image is acquired, and convolution enhancement processing is performed on the eighth image to obtain the second image.

[0075] In a second aspect, an embodiment of the present application provides an image processing device, the device comprising:

[0076] a first acquisition unit, configured to acquire a first image and a second image, wherein the first image and the second image include a plurality of trajectory lines and / or a plurality of trajectory points, wherein the trajectory point is formed by the intersection of two trajectory lines, and each of the plurality of trajectory lines has a corresponding index number;

[0077] a second acquiring unit, configured to acquire a first trajectory line in the first image, and rotate the first image based on an angle between the first trajectory line and a horizontal direction;

[0078] a first processing unit, configured to enlarge or reduce the first image to obtain a third image, wherein the third image has the same ratio as the second image;

[0079] a second processing unit, configured to determine a target offset based on the second image and the third image;

[0080] A third processing unit is configured to move the third image based on the target offset.

[0081] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising: a memory and a processor;

[0082] The memory is used to store instructions or computer programs;

[0083] The processor is used to execute the instructions or computer programs in the memory so that the electronic device performs the image processing method described in any one of the implementations of the first aspect above.

[0084] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the image processing method described in any one of the implementations of the first aspect above.

[0085] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a program. When the program runs on a processor, it enables a computer or a network device to execute the image processing method described in any one of the implementation methods of the first aspect above.

[0086] The technical solution provided by the embodiments of the present application has the following advantages compared with the prior art:

[0087] When processing an image, this embodiment first acquires a first image and a second image, wherein the first image and the second image include a plurality of trajectory lines and a plurality of trajectory points, the trajectory points are formed by the intersection of two trajectory lines, and each trajectory line has a corresponding index number. Then, the first trajectory line in the first image is selected, and the first image is rotated based on the angle between the first trajectory line and the horizontal direction, that is, the first image is rotated to the horizontal direction. The scaling ratios of the initially obtained first image and the second image may be different, so it is necessary to enlarge or reduce the first image to obtain a third image so that the third image has the same ratio as the second image. Then, the target offset is determined based on the second image and the third image, and the third image is moved based on the target offset. Through the image processing method provided in the embodiment of the present application, the first image and the second image can be aligned with pixel-level accuracy, thereby improving the accuracy of image processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments provided in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0089] FIG1 is a schematic diagram of a flow chart of an image processing method provided in an embodiment of the present application;

[0090] FIG2a is a schematic diagram of an image provided in an embodiment of the present application;

[0091] FIG2 b is another image schematic diagram provided in an embodiment of the present application;

[0092] FIG3 is a schematic diagram of a first image provided in an embodiment of the present application;

[0093] FIG4 is a schematic diagram of an image rotation provided in an embodiment of the present application;

[0094] FIG5a is a schematic diagram of a third image with multiple periods provided in an embodiment of the present application;

[0095] FIG5 b is a schematic diagram of a second image with multiple periods provided in an embodiment of the present application;

[0096] FIG6 is a schematic diagram of a preset cycle provided in an embodiment of the present application;

[0097] FIG7 is a schematic flow chart of another image processing method provided in an embodiment of the present application;

[0098] FIG8 is a schematic structural diagram of an image processing device provided in an embodiment of the present application;

[0099] FIG9 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0100] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. The described embodiments are only exemplary implementation methods of the present application and are not all implementation methods. Those skilled in the art can combine the embodiments of the present application to obtain other embodiments without creative work, and these embodiments are also within the scope of protection of the present application.

[0101] In order to facilitate understanding of the technical solution provided by this application, the technical background involved in this application will be introduced below.

[0102] In the analysis of gene expression in the spatial dimension, in order to match the gene expression image with the actual spatial position of the biological sample, an image registration step is required to obtain accurate gene expression information. Existing registration technologies are mostly based on specific graphic marks on the spatiotemporal chip to obtain a microscope image of the biological sample, and then perform a rough position correspondence between the microscope image and the gene expression visualization image based on the position of the mark on the microscope image. Combined with the characteristics of the image itself and supplemented by manual visual adjustment, the spatial position correspondence between the microscope image and the gene expression visualization image is achieved. However, existing registration technologies can only achieve position matching in the relatively macroscopic dimension of biological samples, resulting in low registration accuracy and inability to support more refined gene expression analysis.

[0103] Based on this, an embodiment of the present application provides an image processing method to improve the accuracy of image processing. In specific implementation, a first image and a second image are first acquired, wherein the first image and the second image include a plurality of trajectory lines and a plurality of trajectory points, the trajectory points are formed by the intersection of two trajectory lines, and each trajectory line has a corresponding index number. Then the first trajectory line in the first image is selected, and the first image is rotated based on the angle between the first trajectory line and the horizontal direction, that is, the first image is rotated to the horizontal direction. The scaling ratios of the initially obtained first image and the second image may be different, so it is necessary to enlarge or reduce the first image to obtain a third image so that the third image has the same ratio as the second image. Then, the target offset is determined based on the second image and the third image, and the third image is moved based on the target offset. Through the image processing method provided in the embodiment of the present application, the first image and the second image can be aligned with pixel-level accuracy to improve the accuracy of image processing.

[0104] The image processing method provided in the embodiments of the present application will be introduced below with reference to the accompanying drawings.

[0105] Refer to Figure 1, which is a flowchart of an image processing method provided in an embodiment of the present application.

[0106] The method specifically comprises the following steps:

[0107] S101: Acquire a first image and a second image;

[0108] The first image and the second image include a plurality of trajectory lines and / or a plurality of trajectory points. A trajectory point is formed by the intersection of two trajectory lines, and each trajectory line has a corresponding index number.

[0109] Before processing the image, first obtain the first image and the second image. There are multiple trajectory lines and trajectory points in the first image. Each trajectory point is formed by the intersection of two trajectory lines, and each trajectory line has a corresponding index number. There are also multiple trajectory lines and trajectory points in the second image, and the index numbers of the trajectory lines correspond one-to-one with the index numbers of the trajectory lines in the first image. As shown in Figures 2a and 2b, Figure 2a is a schematic diagram of a portion of the first image, and Figure 2b is a schematic diagram of a portion of the second image. The numerical labels in Figure 2a represent the index numbers of the trajectory lines in the first image, and the black dots represent the trajectory points formed by the intersection of the two trajectory lines. Similarly, the numerical labels in Figure 2b represent the index numbers of the trajectory lines in the second image, which correspond one-to-one with the trajectory lines in the first image, and the gray dots represent the trajectory points formed by the intersection of the two trajectory lines.

[0110] It should be noted that the schematic diagram of the first image and the second image provided in the above embodiment is only a possible implementation method and does not limit the specific form of the image.

[0111] In one possible implementation, the first image can be a microscope image, and the second image can be a gene expression visualization image, a gene expression image, or a gene image. This allows pixel-level registration of the microscope image and the gene expression visualization image. The microscope image is obtained by photographing a biological sample on a spatiotemporal chip using a microscope, and the gene expression visualization image is generated based on the same biological sample using a gene expression matrix. The spatiotemporal chip has regularly arranged sites for collecting genes from the biological sample. Each site can capture the gene sequence of the biological sample. Gene sequencing can be used to obtain the gene sequence or number of bases at each site. The number of gene sequences at each site can then be used as the value of an element in the gene expression matrix, so that the elements of the gene expression matrix correspond to the sites on the spatiotemporal chip (e.g., one element corresponds to one site, and the element in the first row and first column of the gene expression matrix corresponds to the site in the upper left corner of the spatiotemporal chip). This can be followed by analogy to obtain the gene expression matrix, and the gene expression visualization image can then be drawn based on the gene expression matrix. Pixels in the gene expression visualization image correspond to elements of the gene expression matrix, and the pixel values ​​of the pixels correspond to the values ​​of the elements. Therefore, the gene expression visualization image contains spatial information from the same biological sample. Chips can include spatiotemporal chips, sequencing chips, and the like.

[0112] S102: Acquire a first trajectory line in a first image, and rotate the first image based on an angle between the first trajectory line and a horizontal direction.

[0113] When the first image is a microscope image, since the microscope image is obtained by photography, the trajectory line in the microscope image is not necessarily in the horizontal or vertical direction. Therefore, the first trajectory line in the microscope image can be selected as a reference, the angle between the first trajectory line and the horizontal direction can be calculated, and the microscope image can be rotated by the aforementioned angle to be in the horizontal direction. Referring to Figure 3, a schematic diagram of a first image is shown. As can be seen from Figure 3, the first image is not in the horizontal direction, and there is an angle between the first trajectory line and the horizontal direction. Therefore, the first image can be rotated to the horizontal direction after calculating the aforementioned angle.

[0114] When the second image is a gene expression visualization image, since it is generated from a gene expression matrix and appears as many discrete points, it is less capable of detecting expression clustering features or edge features of biological samples. In a preferred implementation, tissue segmentation can be performed on the gene expression visualization image to obtain the biological sample outline as shown by gene expression. Convolution enhancement is then performed on the approximate area containing the sample outline to enhance the edge features of the gene expression visualization image, facilitating subsequent registration of the microscope image with the gene expression visualization image.

[0115] S103: Enlarging or reducing the first image to obtain a third image, wherein the third image has the same ratio as the second image.

[0116] When the first image is captured by a microscope, the second image is a gene expression visualization image and can serve as a reference image. However, the first and second images may have different scale ratios. To align the first and second images, the first image needs to be enlarged or reduced to obtain a third image of the same size as the second image.

[0117] In one possible implementation, the scaling ratio of the first image can be determined based on the spacing between trajectory lines with corresponding index numbers in the first and second images. Specifically, the second trajectory line in the first image is obtained, and a first spacing is determined based on the first and second trajectory lines of the first image. A third and fourth trajectory lines are obtained in the second image, where the index number of the third trajectory line is the same as the index number of the first trajectory line, and the index number of the fourth trajectory line is the same as the index number of the second trajectory line, and a second spacing is determined based on the third and fourth trajectory lines. Then, based on the first and second spacings, a scaling ratio value between the first and second images is determined, and the first image is enlarged or reduced according to the aforementioned ratio value to obtain a third image with the same scale as the second image. The first and second trajectory lines are parallel, and the third and fourth trajectory lines are parallel.

[0118] When determining the first spacing based on the first and second trajectory lines, the calculation can be performed using trajectory points on the first and second trajectory lines. When two trajectory points have the same horizontal coordinate or vertical coordinate, the difference between the corresponding vertical coordinates or horizontal coordinates can be used as the first spacing. Similarly, the second spacing between the third and fourth trajectory lines can be determined based on the same principle. It should be noted that the method for calculating the scaling ratio of the first and second images provided in the above embodiment is merely an example and is not limited to the above implementation.

[0119] S104: Determine a target offset based on the second image and the third image.

[0120] After the first image is scaled to a third image with the same proportions as the second image, the target offset can be subsequently determined based on the second and third images. As can be seen from the above embodiment, after the original first image is rotated to a horizontal orientation, there may be a 90-degree angle difference between the first and second images. To further improve the accuracy of image processing, after the first image is scaled to a third image with the same proportions as the second image, the third image can be rotated according to a preset rotation angle, and the corresponding target offset can be determined at each rotation angle. This will be explained below with reference to a specific application scenario.

[0121] As shown in Figure 4, S represents the third image, V represents the second image, the black dot represents the center of gravity of the third image, and the gray dot represents the center of gravity of the second image. The four regions in Figure 4 represent the original third image and the images after the third image is rotated 90 degrees, 180 degrees, and 270 degrees clockwise, respectively. After the third image is rotated, the origin coordinates of the third image and the origin coordinates of the second image can be aligned. The target offset can then be determined based on the origin-aligned third and second images.

[0122] For any preset rotation angle, in order to process the image more conveniently and accurately, this embodiment provides a preferred implementation method, that is, first, the third image can be preliminarily positioned to approximately the same position as the second image based on the centroid coordinates of the third image and the second image, and then the target offset is calculated based on the moved third image for precise positioning. In a specific implementation, the first image is enlarged or reduced to obtain a fifth image, wherein the fifth image has the same proportion as the second image. Then, the first centroid coordinates of the fifth image and the second centroid coordinates of the second image are calculated respectively, and the centroid offset is determined based on the first centroid coordinates and the second centroid coordinates, and the fifth image is moved based on the centroid offset to obtain a fourth image, that is, a preliminarily positioned fourth image. Among them, when calculating the centroid coordinates of the fifth image and the second image, a density centroid algorithm based on pixel grayscale values ​​can be used for implementation, and this embodiment does not limit this.

[0123] It should be noted that the image movement in the intermediate process can be implemented on a temporary image that copies the first image, that is, the first image is not moved and aligned. After the calculation is completed to obtain the final target offset, the first image is moved again at one time and aligned with the second image.

[0124] After obtaining the preliminarily determined fourth image, the target offset can be determined based on the fourth and second images. In practical applications, the trajectory lines inherent to the spatiotemporal chip may have multiple periods. Therefore, the microscopic images and gene expression visualization images obtained based on the spatiotemporal chip may also have multiple periods, meaning that the fourth and second images have multiple periods. Each period has a corresponding sub-image, with each sub-image containing the same number of trajectory lines and trajectory points, and the index numbers of the trajectory lines between the sub-images correspond one-to-one. As shown in Figures 5a and 5b, Figure 5a shows the third image with multiple periods, and Figure 5b shows the second image with multiple periods. Figure 5a only shows the sub-images for the first and second periods. It can be seen that the sub-images for each period have the same number of index lines and index points, with each index line corresponding to an index number. Figure 5b also shows the sub-images corresponding to the first and second periods. The schematic diagrams of the third and second images provided in the above embodiment are merely one possible implementation and do not limit the specific form of the images. Therefore, when determining the target offset based on the fourth and second images, the effect of period must be considered.

[0125] In a specific implementation, a first offset can be determined based on the fourth image and the second image. This first offset is the undetermined offset between the fourth image and the second image. Then, with the first offset as the center, multiple second offsets corresponding to the first offset within a preset period are determined, i.e., each period within the preset period corresponds to a second offset. For any second offset, the fourth image is shifted based on the second offset to obtain a sixth image. A first similarity between the sixth image and the second image is then calculated, thereby obtaining the first similarity corresponding to each period within the preset period. Since the similarity determined above is obtained at any one rotation angle, the above steps need to be repeated at other preset rotation angles to obtain multiple first similarities corresponding to each rotation angle. Based on the multiple first similarities determined for all rotation angles and all periods, a first maximum similarity is determined, and the sixth image corresponding to this first maximum similarity is determined as the third image. The second offset corresponding to the third image can then be determined as the target offset.

[0126] When determining the first offset based on the fourth image and the second image, one possible implementation method is to first obtain multiple groups of trajectory point sets, where each group of trajectory point sets includes a first trajectory point and a second trajectory point, the first trajectory point is located in the fourth image, the second trajectory point is located in the second image, and the first trajectory point and the second trajectory point have the same index identifier. Since a trajectory point is formed by two intersecting trajectory lines, each trajectory line has an index number, the index identifier of the trajectory point can be composed of the index numbers of the two intersecting trajectory lines of the trajectory point. In this embodiment, since there are multiple trajectory points in both the fourth image and the second image, it is necessary to obtain trajectory points with a corresponding relationship to form a group of trajectory point sets, that is, trajectory points with the same index number. For any group of trajectory point sets in the multiple groups of trajectory point sets, the fourth offset can be determined based on the coordinates of the first trajectory point and the coordinates of the second trajectory point, and then the first offset is determined based on the multiple fourth offsets.

[0127] To facilitate a clearer understanding of this solution, before introducing the implementation of the third offset, the implementation of the fourth offset will be described first.

[0128] In a specific implementation, since the second image has multiple periods, i.e., multiple sub-images, the first and second trajectory points obtained with the same index identifier may not belong to the corresponding period, and therefore, multiple second trajectory points may be determined. When determining the fourth offset based on the coordinates of the first and second trajectory points, any first trajectory point in the fourth image can be selected and, based on the index identifier of the first trajectory point, multiple second trajectory points with the same identifier can be determined. To obtain the second trajectory point corresponding to the first trajectory point, the Euclidean distance between the first trajectory point and each second trajectory point can be calculated. After calculating multiple Euclidean distances, the second trajectory point corresponding to the minimum Euclidean distance is obtained, which is the second trajectory point corresponding to the first trajectory point. The fourth offset is then determined based on the coordinate difference between the second trajectory point and the first trajectory point corresponding to the minimum Euclidean distance. Since the fourth image contains multiple trajectory points, i.e., multiple sets of trajectory point sets, it is necessary to traverse all trajectory point sets and determine the fourth offset corresponding to each set of trajectory point sets using the method for determining the basic offset described above. In one possible implementation, among the multiple fourth offsets determined, fourth offsets that exceed a preset range may be first excluded, and then a target offset may be determined based on the remaining fourth offsets. For example, the median of the multiple fourth offsets may be selected as the target offset, or the average of all fourth offsets may be calculated as the target offset, to maximize the accuracy of the offset determination. It should be noted that this embodiment does not limit the specific method for determining the target offset based on the fourth offset.

[0129] The image movement within a preset period will be described below with reference to a specific application scenario.

[0130] Referring to FIG. 6 , FIG. 6 is a schematic diagram of a preset period determined with the first offset as the center.

[0131] In one possible implementation, the second image is used as a reference. As shown in FIG6 , the middle gray area represents the period in which the second trajectory point is located. After determining the first offset based on the first and second trajectory points, a 5*5 period is selected with the first offset (the period to which the corresponding second trajectory point belongs) as the center, and the second offset corresponding to each period is determined. The difference between the second offsets corresponding to adjacent periods is the width of the sub-image. Similarly, the second offset corresponding to each period can be determined.

[0132] The fourth image can then be shifted by the second offset corresponding to any period to obtain a sixth image, and the similarity between the sixth image and the second image can be calculated. This embodiment provides a possible implementation for calculating the similarity: performing a frequency domain transform on the sixth image to obtain a first frequency domain image, performing a frequency domain transform on the second image to obtain a second frequency domain image, and calculating the Hamming distance between the first frequency domain image and the second frequency domain image as the similarity. The smaller the Hamming distance, the greater the similarity between the first frequency domain image and the second frequency domain image. After determining the Hamming distances corresponding to all preset rotation angles and all preset periods, the minimum Hamming distance is used as the first maximum similarity.

[0133] In this case, a frequency domain image can be obtained using a discrete cosine transform, that is, a discrete cosine transform is performed on the moved third image to obtain a first frequency domain image, and a discrete cosine transform is performed on the second image to obtain a second frequency domain image. Then, a low-frequency frequency domain image is intercepted from the first frequency domain image. The pixel mean of the low-frequency frequency domain image is calculated based on the low-frequency frequency domain image. In the low-frequency frequency domain image, the pixel values ​​of pixels with pixel values ​​higher than the pixel mean are set to 1, the pixel values ​​of pixels with pixel values ​​lower than the pixel mean are set to 0, and the pixel values ​​of pixels with pixel values ​​equal to the pixel mean are set to 0 or 1. Then, a hash fingerprint is obtained based on the reset low-frequency frequency domain image. Following the same execution steps, a hash fingerprint of the second frequency domain image is obtained. Then, based on the hash fingerprint of the first frequency domain image and the hash fingerprint of the second frequency domain image, the number of different bits in the two hash fingerprints is determined, which is used as the Hamming distance between the first frequency domain image and the second frequency domain image. It should be noted that the method for calculating similarity in the above embodiment is only an exemplary implementation method and is not limited to the above implementation method. Other achievable methods for calculating image similarity are also within the scope of protection of this application.

[0134] In the above-described embodiment of the present application, the fourth image can first be preliminarily moved and positioned according to its center of gravity offset relative to the second image. In another possible implementation, when determining the third image, the fourth image is not preliminarily positioned. Instead, for any preset rotation angle, the fourth image is moved according to each period within a preset cycle, obtaining a seventh image corresponding to each period. For example, the third center of gravity coordinates of the fourth image can be first calculated. With the period corresponding to the third center of gravity coordinates as the center, the fourth image is moved according to each period within the preset cycle, obtaining a seventh image. A second similarity between the seventh image and the second image is then calculated, thereby obtaining a second similarity corresponding to each period within the preset cycle. The above steps are then repeated for other preset rotation angles to obtain the second similarity corresponding to each preset rotation angle and each period. A second maximum similarity is determined based on these multiple second similarities. The seventh image corresponding to the second maximum similarity is used as the third image, and the target offset is then determined based on this third image and the second image.

[0135] Specifically, first, a plurality of third offsets can be determined based on the second image and the third image, and the target offset can be determined based on the plurality of third offsets. The calculation principle of the third offset is the same as the principle of calculating the fourth offset in the above embodiment. That is, a plurality of groups of trajectory point sets are obtained, wherein each group of trajectory point sets includes a third trajectory point and a fourth trajectory point, the third trajectory point is located in the third image, the fourth trajectory point is located in the second image, and the third trajectory point and the fourth trajectory point have the same index identifier. For any group of trajectory point sets, the third offset is determined based on the coordinates of the third trajectory point and the coordinates of the fourth trajectory point. The specific implementation method of determining the third offset based on the coordinates of the third trajectory point and the coordinates of the fourth trajectory point can refer to the specific implementation method of determining the fourth offset based on the coordinates of the first trajectory point and the coordinates of the second trajectory point, which will not be repeated here.

[0136] Similarly, after determining the third offsets corresponding to multiple sets of trajectory point sets, you can first exclude the third offsets that exceed the preset range, and then select the median as the target offset among the multiple third offsets that meet the preset range, or calculate the average of multiple third offsets as the target offset, so as to maximize the accuracy of determining the offset.

[0137] S105: Moving the third image based on the target offset.

[0138] After the target offset is determined based on the method provided in the above embodiment, a preset rotation angle corresponding to the target offset is determined, and the third image is moved based on the preset rotation angle and the target offset.

[0139] It should be noted that the image movement in the intermediate process is a processing process implemented on a temporary image that copies the first image, that is, the first image is not moved and aligned. After the calculation is completed to obtain the final target offset, the first image is moved again at one time and aligned with the second image.

[0140] In the image processing method provided in the embodiment of the present application, the first image and the second image can be registered based on image coordinates with pixel-level precision, thereby improving the accuracy of image processing and facilitating subsequent research and development and analysis using the registered images.

[0141] When the third image and the second image have multiple periods, the offset corresponding to the preset period is derived from the calculated offset using the theoretical period width, which may result in errors in the actual image. To correct for errors in theoretical inference and further improve image processing accuracy, in a preferred implementation, after shifting the third image based on the target offset, the corrected offset can be reused to determine the final offset based on the shifted third image and the second image. The third image is then further shifted based on the determined final offset, and the third image is registered with the second image to improve registration accuracy. For example, multiple sets of trajectory point sets are acquired based on the shifted third image and the second image, each set of trajectory point sets including a fifth trajectory point and a sixth trajectory point, the fifth trajectory point being located in the third image and the sixth trajectory point being located in the second image, and the fifth and sixth trajectory points having the same index identifier. For each set of trajectory point sets, a fifth offset is determined based on the coordinates of the fifth and sixth trajectory points. After determining the fifth offset corresponding to each of the multiple sets of trajectory point sets, a final offset is determined based on the multiple fifth offsets. The implementation of determining the fifth offset and determining the final offset based on multiple fifth offsets can be found in the above embodiment and will not be described in detail here.

[0142] The image processing method provided by the embodiment of the present application will be described below in conjunction with an application scenario. Referring to FIG7 , FIG7 is a flow chart of another image processing method provided by the embodiment of the present application.

[0143] In this application scenario, the first image is a microscope image, and the second image is a gene expression visualization image. After obtaining the microscope image and the gene expression visualization image, the images are pre-processed, for example, the microscope image is rotated to the horizontal direction, and the microscope image is scaled to the same scale as the gene expression visualization image. In addition, the gene expression visualization image can also be subjected to convolution enhancement processing to enhance the edge features of the image. In the present embodiment, the microscope image can be rotated to four directions, namely the original horizontal direction, 90 degrees clockwise rotation, 180 degrees clockwise rotation, and 270 degrees clockwise rotation, and the same image processing steps are performed in any direction. That is, the center of gravity of the microscope image and the gene expression visualization image are calculated respectively, and the microscope image is initially positioned. Then, the trajectory point coordinates of the corresponding index mark are used in the microscope image and the gene expression visualization image to determine the basic offset. Taking the basic offset as the center, traverse the basic offsets corresponding to the preset period, calculate the image similarity under the corresponding period, perform the above steps for each rotation direction, determine the image similarity corresponding to each rotation direction and each period, select the period with the highest similarity as the benchmark, determine the basic offset corresponding to this period as the target offset, and then move the microscope image based on the target offset to achieve image registration.

[0144] The beneficial effects of the image processing method provided by this embodiment are as described in the above method embodiments, and will not be described in detail here.

[0145] Based on the above method embodiment, the present application also provides an image processing device. Referring to FIG8 , the device 800 includes:

[0146] A first acquisition unit 801 is configured to acquire a first image and a second image, wherein the first image and the second image include a plurality of trajectory lines and / or a plurality of trajectory points, each of which is formed by the intersection of two trajectory lines, and each of the plurality of trajectory lines has a corresponding index number;

[0147] A second acquiring unit 802 is configured to acquire a first trajectory line in the first image, and rotate the first image based on an angle between the first trajectory line and a horizontal direction;

[0148] A first processing unit 803 is configured to enlarge or reduce the first image to obtain a third image, where the third image has the same ratio as the second image;

[0149] A second processing unit 804 is configured to determine a target offset based on the second image and the third image;

[0150] The third processing unit 805 is configured to move the third image based on the target offset.

[0151] In one possible implementation, the first processing unit 803 is specifically configured to obtain a second trajectory line in the first image, and determine a first spacing based on the first trajectory line and the second trajectory line; obtain a third trajectory line and a fourth trajectory line in the second image, and determine a second spacing based on the third trajectory line and the fourth trajectory line, wherein the index number of the third trajectory line is the same as the index number of the first trajectory line, and the index number of the fourth trajectory line is the same as the index number of the second trajectory line; determine a scale value based on the first spacing and the second spacing, and enlarge or reduce the first image according to the scale value to obtain a third image.

[0152] In one possible implementation, the first processing unit 803 is specifically configured to enlarge or reduce the first image to obtain a fourth image, wherein the fourth image has the same ratio as the second image; and move the fourth image according to a preset period and a preset rotation angle to obtain the third image.

[0153] In one possible implementation, the first processing unit 803 is specifically used to enlarge or reduce the first image to obtain a fifth image, and the fifth image has the same ratio as the second image; calculate the first center of gravity coordinates of the fifth image and the second center of gravity coordinates of the second image respectively; determine the center of gravity offset based on the first center of gravity coordinates and the second center of gravity coordinates; and move the fifth image based on the center of gravity offset to obtain the fourth image.

[0154] In one possible implementation, the first processing unit 803 is specifically used to obtain the preset period and multiple preset rotation angles; for any preset rotation angle, determine a first offset based on the fourth image and the second image; determine multiple second offsets corresponding to the first offset within the preset period, each second offset of the multiple second offsets corresponds one-to-one to each period within the preset period; for any second offset, move the fourth image based on the second offset to obtain a sixth image; determine a first similarity between the sixth image and the second image; determine a first maximum similarity based on the multiple preset rotation angles and the first similarities corresponding to each period; and determine that the sixth image corresponding to the first maximum similarity is the third image.

[0155] In a possible implementation, the second processing unit 804 is specifically configured to determine the second offset corresponding to the third image as the target offset;

[0156] The third processing unit 805 is specifically configured to move the third image based on the preset rotation angle corresponding to the third image and the target offset.

[0157] In one possible implementation, the first processing unit 803 is specifically used to obtain the preset period and the multiple preset rotation angles; for any preset rotation angle, move the fourth image according to each period within the preset period to obtain multiple seventh images, and each seventh image in the multiple seventh images corresponds one-to-one to each period within the preset period; for any seventh image, determine the second similarity between the seventh image and the second image; determine the second maximum similarity based on the multiple preset rotation angles and the second similarities corresponding to each period; and determine that the seventh image corresponding to the second maximum similarity is the third image.

[0158] In a possible implementation, the second processing unit 804 is specifically configured to determine a plurality of third offsets based on the second image and the third image; and determine the target offset based on the plurality of third offsets;

[0159] The third processing unit 805 is specifically configured to move the third image based on the preset rotation angle corresponding to the third image and the target offset.

[0160] In one possible implementation, the first processing unit 803 is specifically configured to obtain multiple groups of trajectory point sets, each of the multiple groups of trajectory point sets including a first trajectory point and a second trajectory point, the first trajectory point being located in the fourth image, the second trajectory point being located in the second image, and the first trajectory point and the second trajectory point having the same index identifier; for any group of trajectory point sets, determine a fourth offset based on coordinates of the first trajectory point and the coordinates of the second trajectory point in the trajectory point set; and determine the first offset based on the fourth offsets corresponding to the multiple groups of trajectory point sets.

[0161] In one possible implementation, when the second image has multiple sub-images, the trajectory line of each sub-image has the same index number, and the first processing unit 803 is specifically configured to determine the second trajectory point in each sub-image based on the index identifier of the first trajectory point, where the first trajectory point and the second trajectory point have the same index identifier; determine the Euclidean distance between the first trajectory point and any second trajectory point among the multiple second trajectory points; determine the second trajectory point corresponding to the minimum Euclidean distance among the multiple Euclidean distances; and determine a fourth offset based on the coordinates of the first trajectory point and the coordinates of the second trajectory point corresponding to the minimum Euclidean distance.

[0162] In one possible implementation, the first processing unit 803 is specifically used to perform a frequency domain transform on the sixth image to obtain a first frequency domain image; perform a frequency domain transform on the second image to obtain a second frequency domain image; calculate the Hamming distance between the first frequency domain image and the second frequency domain image as the first similarity; the first processing unit 803 is also used to determine the minimum Hamming distance as the first maximum similarity based on the multiple preset rotation angles and the Hamming distance corresponding to each period.

[0163] In a possible implementation, the first processing unit 803 is specifically configured to calculate the third barycentric coordinates of the fourth image; and to move the fourth image according to each of the periods with the third barycentric coordinates as the center to obtain the seventh image.

[0164] In one possible implementation, the first processing unit 803 is specifically configured to obtain multiple groups of trajectory point sets, each group of the multiple trajectory point sets including a third trajectory point and a fourth trajectory point, the third trajectory point being located in the third image, the fourth trajectory point being located in the second image, and the third trajectory point and the fourth trajectory point having the same index identifier; and for any group of trajectory point sets, determining a third offset based on the coordinates of the third trajectory point and the coordinates of the fourth trajectory point in the trajectory point set.

[0165] In one possible implementation, the device 800 is further configured to obtain multiple groups of trajectory point sets based on the moved third image and the second image, each group of the multiple trajectory point sets including a fifth trajectory point and a sixth trajectory point, the fifth trajectory point being located in the moved third image, the sixth trajectory point being located in the second image, and the fifth trajectory point having the same index identifier as the six trajectory points; for any group of trajectory point sets, determining a fifth offset based on the coordinates of the fifth trajectory point and the coordinates of the sixth trajectory point in the trajectory point set; determining a final offset based on the fifth offsets corresponding to the multiple groups of trajectory point sets; and moving the moved third image based on the final offset.

[0166] In a possible implementation, the first acquisition unit 801 is specifically configured to acquire an eighth image, and perform convolution enhancement processing on the eighth image to obtain the second image.

[0167] Based on the above method embodiment and apparatus embodiment, the present application embodiment further provides an electronic device. Referring to FIG9 , the electronic device 900 includes: a memory 901 and a processor 902;

[0168] The memory 901 is used to store instructions or computer programs;

[0169] The processor 902 is configured to execute the instructions or computer programs in the memory, so that the electronic device executes the image processing method described in the above method embodiment.

[0170] In addition, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the image processing method described in the above method embodiment.

[0171] An embodiment of the present application further provides a computer program product, which includes a program. When the program runs on a processor, it enables a computer or a network device to execute the image processing method described in the above method embodiment.

[0172] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. In particular, for system or device embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiments described above are merely illustrative, wherein the units or modules described as separate components may or may not be physically separated, and the components shown as units or modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units or modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Ordinary technicians in this field can understand and implement it without expending creative work.

[0173] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0174] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0175] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0176] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a first image and a second image, wherein the first image and the second image include a plurality of trajectory lines and / or a plurality of trajectory points, wherein the trajectory point is formed by the intersection of two trajectory lines, and each of the plurality of trajectory lines has a corresponding index number; Acquire a first trajectory line in the first image, and rotate the first image based on an angle between the first trajectory line and a horizontal direction; Enlarging or reducing the first image to obtain a third image, wherein the third image has the same proportion as the second image; determining a target offset based on the second image and the third image; The third image is moved based on the target offset.

2. The method according to claim 1, characterized in that The step of enlarging or reducing the first image to obtain a third image includes: Acquire a second trajectory line in the first image, and determine a first distance based on the first trajectory line and the second trajectory line; Acquire a third trajectory line and a fourth trajectory line in the second image, and determine a second spacing based on the third trajectory line and the fourth trajectory line, wherein the index number of the third trajectory line is the same as the index number of the first trajectory line, and the index number of the fourth trajectory line is the same as the index number of the second trajectory line; A ratio value is determined based on the first spacing and the second spacing, and the first image is enlarged or reduced according to the ratio value to obtain a third image.

3. The method according to claim 1, characterized in that The step of enlarging or reducing the first image to obtain a third image includes: enlarging or reducing the first image to obtain a fourth image, wherein the fourth image has the same proportion as the second image; The fourth image is moved according to a preset period and a preset rotation angle to obtain the third image.

4. The method according to claim 3, characterized in that The step of enlarging or reducing the first image to obtain a fourth image includes: enlarging or reducing the first image to obtain a fifth image, wherein the fifth image has the same ratio as the second image; respectively calculating the first barycentric coordinates of the fifth image and the second barycentric coordinates of the second image; determining a center of gravity offset based on the first center of gravity coordinate and the second center of gravity coordinate; The fifth image is moved based on the center of gravity offset to obtain the fourth image.

5. The method according to claim 3 or 4, characterized in that The step of moving the fourth image according to a preset period and a preset rotation angle to obtain the third image includes: Obtaining the preset period and multiple preset rotation angles; For any preset rotation angle, determining a first offset based on the fourth image and the second image; Determine a plurality of second offsets corresponding to the first offset within the preset period, wherein each second offset in the plurality of second offsets corresponds one-to-one to each period within the preset period; For any second offset, moving the fourth image based on the second offset to obtain a sixth image; determining a first similarity between the sixth image and the second image; Determining a first maximum similarity based on the multiple preset rotation angles and the first similarity corresponding to each period; Determine that a sixth image corresponding to the first maximum similarity is the third image.

6. The method according to claim 5, characterized in that The determining of the target offset based on the second image and the third image includes: determining a second offset corresponding to the third image as the target offset; The moving the third image based on the target offset comprises: The third image is moved based on the preset rotation angle corresponding to the third image and the target offset.

7. The method according to claim 3, characterized in that The step of moving the fourth image according to a preset period and a preset rotation angle to obtain the third image includes: Acquiring the preset period and the plurality of preset rotation angles; For any preset rotation angle, the fourth image is moved according to each period within the preset period to obtain a plurality of seventh images, each seventh image in the plurality of seventh images corresponding one-to-one to each period within the preset period; For any seventh image, determining a second similarity between the seventh image and the second image; Determining a second maximum similarity based on the multiple preset rotation angles and the second similarity corresponding to each period; Determine that the seventh image corresponding to the second maximum similarity is the third image.

8. The method according to claim 7, characterized in that The determining of the target offset based on the second image and the third image includes: determining a plurality of third offsets based on the second image and the third image; determining the target offset based on the plurality of third offsets; The moving the third image based on the target offset comprises: The third image is moved based on the preset rotation angle corresponding to the third image and the target offset.

9. The method according to claim 5, characterized in that The determining of the first offset based on the fourth image and the second image includes: Acquire multiple groups of trajectory point sets, each group of the multiple trajectory point sets includes a first trajectory point and a second trajectory point, the first trajectory point is located in the fourth image, the second trajectory point is located in the second image, and the first trajectory point and the second trajectory point have the same index identifier; For any set of trajectory point sets, determining a fourth offset based on the coordinates of the first trajectory point and the coordinates of the second trajectory point in the trajectory point set; The first offset is determined based on fourth offsets corresponding to the multiple groups of trajectory point sets.

10. The method according to claim 9, characterized in that When the second image has multiple sub-images, the trajectory lines of each sub-image have the same index number, and determining the fourth offset based on the coordinates of the first trajectory point and the coordinates of the second trajectory point in the trajectory point set includes: determining the second track point in each sub-image based on the index identifier of the first track point, wherein the first track point and the second track point have the same index identifier; determining a Euclidean distance between the first trajectory point and any second trajectory point of the plurality of second trajectory points; Determining a second trajectory point corresponding to a minimum Euclidean distance among the plurality of Euclidean distances; A fourth offset is determined based on the coordinates of the first trajectory point and the coordinates of the second trajectory point corresponding to the minimum Euclidean distance.

11. The method according to claim 5, characterized in that The determining a first similarity between the sixth image and the second image includes: Performing frequency domain transformation on the sixth image to obtain a first frequency domain image; performing a frequency domain transform on the second image to obtain a second frequency domain image; Calculating a Hamming distance between the first frequency domain image and the second frequency domain image as a first similarity; The determining a first maximum similarity based on the plurality of preset rotation angles and the first similarity corresponding to each period includes: Based on the multiple preset rotation angles and the Hamming distance corresponding to each period, a minimum Hamming distance is determined as the first maximum similarity.

12. The method according to claim 7, characterized in that The step of moving the fourth image according to each period within the preset period to obtain a seventh image includes: Calculating the third centroid coordinates of the fourth image; The fourth image is moved according to each of the periods, with the period where the third center of gravity coordinates are located as the center, to obtain the seventh image.

13. The method according to claim 8, characterized in that The determining a plurality of third offsets based on the second image and the third image comprises: Acquire multiple groups of trajectory point sets, each group of the multiple trajectory point sets includes a third trajectory point and a fourth trajectory point, the third trajectory point is located in the third image, the fourth trajectory point is located in the second image, and the third trajectory point and the fourth trajectory point have the same index identifier; For any set of trajectory point sets, a third offset is determined based on the coordinates of the third trajectory point and the coordinates of the fourth trajectory point in the trajectory point set.

14. The method according to claim 1, wherein After moving the third image based on the target offset, the method further includes: acquiring, based on the moved third image and the second image, a plurality of sets of trajectory point sets, each set of the plurality of trajectory point sets comprising a fifth trajectory point and a sixth trajectory point, the fifth trajectory point being located in the moved third image, the sixth trajectory point being located in the second image, and the fifth trajectory point having the same index identifier as the sixth trajectory point; For any set of trajectory point sets, determining a fifth offset based on the coordinates of the fifth trajectory point and the coordinates of the sixth trajectory point in the trajectory point set; determining a final offset based on fifth offsets corresponding to the plurality of sets of trajectory points; The shifted third image is shifted based on the final offset.

15. The method according to any one of claims 1 to 14, characterized in that Acquiring the second image includes: An eighth image is acquired, and convolution enhancement processing is performed on the eighth image to obtain the second image.

16. An image processing device, characterized in that: The device comprises: a first acquisition unit, configured to acquire a first image and a second image, wherein the first image and the second image include a plurality of trajectory lines and / or a plurality of trajectory points, wherein the trajectory point is formed by the intersection of two trajectory lines, and each of the plurality of trajectory lines has a corresponding index number; a second acquiring unit, configured to acquire a first trajectory line in the first image, and rotate the first image based on an angle between the first trajectory line and a horizontal direction; a first processing unit, configured to enlarge or reduce the first image to obtain a third image, wherein the third image has the same ratio as the second image; a second processing unit, configured to determine a target offset based on the second image and the third image; A third processing unit is configured to move the third image based on the target offset.

17. An electronic device, characterized in that: The electronic device includes: a memory and a processor; The memory is used to store instructions or computer programs; The processor is configured to execute the instructions or computer program in the memory so that the electronic device executes the image processing method according to any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the image processing method according to any one of claims 1 to 15.

19. A computer program product, characterized in that The computer program product includes a program, and when the program is run on a processor, it causes a computer or a network device to execute the image processing method according to any one of claims 1 to 15.

Images