Positioning table data splicing method and device based on sample structure, equipment and medium

Abstract

The invention discloses a positioning table data splicing method and device based on a sample structure, equipment and a medium, and relates to the technical field of image processing, and the method comprises the steps: determining splicing seam positioning data in a to-be-spliced positioning table, and carrying out the image rendering of the splicing seam positioning data through bilinear interpolation, so as to obtain a plurality of rendered images; integrating the pixel value of each pixel in the plurality of rendered images based on linear filtering transformation to obtain a plurality of continuity enhanced images; determining target pixels meeting a preset enhancement condition by using a preset linear functional, and enhancing the target pixels to obtain a plurality of target enhanced images; and determining an image registration offset, splicing the to-be-spliced positioning table according to the image registration offset to obtain a target splicing positioning table, then rendering the target splicing positioning table, and performing splicing effect verification on the obtained target splicing image. Therefore, the problem of splicing dislocation caused by errors in image registration can be reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a positioning table data splicing method, device, equipment and medium based on a sample structure. Background Art

[0002] Super-resolution microscopy breaks through the optical diffraction limit, allowing biologists to analyze the ultrastructure of biomolecules at nanoscale resolution. Since the size of a single microscopic image is often smaller than the size of the biological sample, it is necessary to stitch a large number of small-field super-resolution images to form a super-resolution panoramic image to meet the needs of pathologists to observe the microstructure of the entire sample. However, the existing super-resolution panoramic image stitching algorithm borrows the idea of ​​point cloud registration to perform super-resolution positioning data registration. Due to the large number of positioning points and the randomness of the excitation, when the sample structure at the seam is discontinuous and the data structure is incomplete, the registration offset cannot be accurately calculated, resulting in errors in image registration, resulting in serious stitching misalignment, affecting subsequent analysis, and the stitching performance needs to be improved.

[0003] From the above, it can be seen that how to reduce the stitching misalignment problem caused by errors in image registration is a technical problem that needs to be solved urgently. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a method, device, equipment and medium for stitching positioning table data based on sample structure, which can reduce the stitching misalignment problem caused by errors in image registration. The specific scheme is as follows:

[0005] In a first aspect, the present application provides a positioning table data splicing method based on a sample structure, comprising:

[0006] Determine a plurality of seam location data in the to-be-joined location tables, and use bilinear interpolation to perform image rendering on the seam location data to obtain a plurality of rendered images;

[0007] Integrating the pixel value of each pixel in the plurality of rendered images based on a linear filtering transformation to obtain a plurality of continuity-enhanced images and a target integration direction and a target linear filtering transformation amplitude corresponding to each pixel in the plurality of continuity-enhanced images;

[0008] Determine a target pixel that meets a preset enhancement condition from the plurality of continuous enhanced images by using a preset linear functional, and perform structural enhancement on the target pixel based on the target integral direction corresponding to the target pixel and the target linear filter transformation amplitude according to a directional filter transformation, so as to obtain a plurality of target enhanced images;

[0009] Based on the phase correlation method, the image registration offsets corresponding to the several target enhanced images are determined, and data splicing is performed on the several positioning tables to be spliced ​​according to the image registration offsets to obtain a target splicing positioning table corresponding to the several positioning tables to be spliced, and then the target splicing positioning table is rendered, and the splicing effect of the obtained target spliced ​​image is verified.

[0010] Optionally, the rendering and reconstruction of a plurality of seam images corresponding to the seam positioning data by using bilinear interpolation to obtain a plurality of rendered images includes:

[0011] Determine a plurality of seam images corresponding to the seam positioning data, and determine a plurality of positioning points in the plurality of seam images;

[0012] Determine the fluorescent molecules corresponding to the plurality of positioning points based on a preset pixel unit, and assign a pixel weight to each pixel in each of the fluorescent molecules;

[0013] The plurality of seam images are rendered based on the pixel weights and the fluorescent molecules to obtain a plurality of rendered images.

[0014] Optionally, the step of integrating the pixel value of each pixel in the plurality of rendered images based on the linear filtering transformation to obtain a plurality of continuity enhanced images and a target integration direction and a target linear filtering transformation amplitude corresponding to each pixel in the plurality of continuity enhanced images comprises:

[0015] Integrating each pixel in the plurality of rendered images according to a preset integration range and a preset integration direction to obtain a plurality of continuity-enhanced images corresponding to the plurality of rendered images;

[0016] The maximum integral corresponding to each pixel is recorded, and the angle value corresponding to the maximum integral is used as the target integral direction corresponding to each pixel, and the integral value corresponding to the maximum integral is used as the target linear filter transformation amplitude corresponding to each pixel.

[0017] Optionally, the determining the target pixel satisfying the preset enhancement condition from the plurality of continuity enhanced images by using a preset linear functional includes:

[0018] Determine the polar coordinates corresponding to each pixel based on the target integral direction corresponding to each pixel and the target linear filter transformation amplitude corresponding to each pixel;

[0019] Calculating the dot product between the unit vector corresponding to each pixel in the plurality of continuity enhanced images and the polar coordinates corresponding to each pixel to obtain a plurality of pixel dot products;

[0020] A plurality of target pixel dot products whose dot product values ​​are higher than a preset dot product threshold value are determined among the plurality of pixel dot products, and pixels corresponding to the plurality of target pixel dot products are used as target pixels that meet a preset enhancement condition.

[0021] Optionally, the target pixel is enhanced based on the target integral direction corresponding to the target pixel and the target linear filter transformation amplitude according to the directional filter transformation to obtain a plurality of target enhanced images, including:

[0022] Based on the preset integral range, dot product processing is performed on the unit vector corresponding to the target pixel and the polar coordinates corresponding to the target pixel to obtain a plurality of target enhanced images corresponding to the plurality of continuity enhanced images.

[0023] Optionally, the determining of the image registration offsets corresponding to the plurality of target enhanced images based on the phase correlation method, and performing data splicing on the plurality of positioning tables to be spliced ​​according to the image registration offsets to obtain a target splicing positioning table corresponding to the plurality of positioning tables to be spliced, and then rendering the target splicing positioning table, and verifying the splicing effect of the obtained target spliced ​​image, includes:

[0024] Determine the image pairs to be registered in the plurality of target enhanced images, and determine the image registration offsets respectively corresponding to the image pairs to be registered according to the phase correlation method, so as to perform data splicing on the plurality of positioning tables to be spliced ​​according to the image registration offsets, so as to obtain a target splicing positioning table corresponding to the plurality of positioning tables to be spliced; the image pairs to be registered are two images to be spliced ​​in the plurality of target enhanced images;

[0025] Rendering the target stitching positioning table to obtain a target stitching image, and verifying the target stitching image to determine whether there are stitching traces in the target stitching image;

[0026] If there is no stitching trace, then the target stitching positioning table corresponding to the target stitching image is verified.

[0027] Optionally, determining the image registration offsets respectively corresponding to the image pairs to be registered, and performing data splicing on the plurality of positioning tables to be spliced ​​according to the image registration offsets, so as to obtain a target splicing positioning table corresponding to the plurality of positioning tables to be spliced, comprises:

[0028] Based on Fourier transform, two images to be registered corresponding to the pair of images to be registered are respectively mapped to the frequency domain to obtain a first matrix and a second matrix corresponding to the two images to be registered;

[0029] Calculating a cross power spectrum of the first matrix and the second matrix, and performing an inverse Fourier transform on the cross power spectrum to obtain a target impulse response function;

[0030] Determine a maximum function value corresponding to the target impulse response function, and use a coordinate parameter corresponding to the maximum function value as an image registration offset;

[0031] The data of the to-be-stitched positioning table corresponding to any one of the to-be-registered image pairs are translated based on the image registration offset to complete the registration with the to-be-stitched positioning table corresponding to another to-be-registered image in the to-be-registered image pair to obtain a target stitching positioning table.

[0032] In a second aspect, the present application provides a positioning table data splicing device based on a sample structure, comprising:

[0033] An image rendering module is used to determine the positioning data of the seams in the positioning tables to be spliced, and perform image rendering on the positioning data of the seams by using bilinear interpolation to obtain a plurality of rendered images;

[0034] A pixel integration module, used for integrating the pixel value of each pixel in the plurality of rendered images based on a linear filtering transformation, so as to obtain a plurality of continuity-enhanced images and a target integration direction and a target linear filtering transformation amplitude corresponding to each pixel in the plurality of continuity-enhanced images;

[0035] A pixel enhancement module, used to determine a target pixel satisfying a preset enhancement condition from the plurality of continuous enhanced images by using a preset linear functional, and to perform structural enhancement on the target pixel based on the target integral direction corresponding to the target pixel and the target linear filter transformation amplitude according to a directional filter transformation, so as to obtain a plurality of target enhanced images;

[0036] A positioning table stitching module is used to determine the image registration offsets corresponding to the several target enhanced images based on the phase correlation method, and to perform data stitching on the several positioning tables to be stitched according to the image registration offsets to obtain a target stitching positioning table corresponding to the several positioning tables to be stitched, and then to render the target stitching positioning table and to verify the stitching effect of the obtained target stitching image.

[0037] In a third aspect, the present application provides an electronic device, including:

[0038] Memory, used to store computer programs;

[0039] A processor is used to execute the computer program to implement the aforementioned positioning table data splicing method based on sample structure.

[0040] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned positioning table data splicing method based on a sample structure is implemented.

[0041] In the present application, the positioning data at the seam in several positioning tables to be spliced ​​are determined, and the positioning data at the seam are rendered using bilinear interpolation to obtain several rendered images; the pixel value of each pixel in the several rendered images is integrated based on a linear filtering transformation to obtain several continuity enhanced images and a target integral direction and a target linear filtering transformation amplitude corresponding to each pixel in the several continuity enhanced images; a target pixel that meets a preset enhancement condition is determined from the several continuity enhanced images using a preset linear functional, and the target pixel is structurally enhanced based on the target integral direction and the target linear filtering transformation amplitude corresponding to the target pixel according to a directional filter transformation to obtain several target enhanced images; the image registration offset corresponding to the several target enhanced images is determined based on a phase correlation method, and the data of the several positioning tables to be spliced ​​are spliced ​​according to the image registration offset to obtain a target splicing positioning table corresponding to the several positioning tables to be spliced, and then the target splicing positioning table is rendered, and the splicing effect of the obtained target splicing image is verified. As can be seen from the above, the embodiment of the present application determines the positioning data at the seam from several positioning tables to be spliced, and uses bilinear interpolation to render the positioning data at the seam to obtain several rendered images, enhances the several rendered images based on linear filtering transformation and directional filter transformation to obtain several target enhanced images, and finally analyzes the several target enhanced images to determine the image registration offset, and splices several positioning tables to be spliced ​​based on the image registration offset to obtain a target splicing positioning table, and then renders the target splicing positioning table, and verifies the splicing effect of the obtained target splicing image. In this way, the positioning tables to be spliced ​​are spliced ​​through the above process of the present application, and the sample structure of the image is enhanced by using linear filtering transformation and directional filter transformation to reduce the problems of incomplete structure and specific artifacts in the positioning data to be spliced, thereby reducing the splicing misalignment problem caused by errors in image registration. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0043] Figure 1A flow chart of a positioning table data splicing method based on a sample structure disclosed in this application;

[0044] Figure 2 It is a timing diagram of the overall structure of a positioning table data splicing method based on a sample structure disclosed in this application;

[0045] Figure 3 A schematic diagram of weight distribution of three positioning points disclosed in this application;

[0046] Figure 4a and Figure 4b Schematic diagram of two specific artifacts disclosed in this application;

[0047] Figure 5 A schematic diagram of a preset integral range of a linear filtering transformation disclosed in the present application;

[0048] Figure 6a An original image obtained by single-molecule localization microscopy disclosed in the present application;

[0049] Figure 6b The intensity image obtained by performing linear filtering transformation on the original image disclosed in the present application;

[0050] Figure 6c The directional image disclosed in the present application is obtained by performing linear filtering transformation processing on the original image;

[0051] Figure 7a and Figure 7b The enhanced image and detail display diagram obtained after linear filtering transformation disclosed in this application;

[0052] Figure 7c and Figure 7d The image and detail display diagram obtained by performing directional filter transformation processing on the enhanced image disclosed in the present application;

[0053] Figure 8 A flowchart of a specific positioning table data splicing method based on a sample structure disclosed in this application;

[0054] Fig. 9 A display diagram of the horizontal stitching registration results of microtubule images of various algorithms disclosed in this application;

[0055] Fig.10 A display diagram of the vertical seam registration results of microtubule images of various algorithms disclosed in this application;

[0056] Figures 11a to 11c This is a data splicing result diagram of a simulated panoramic microtubule disclosed in this application;

[0057] Fig.12aand Figure 12b A real panoramic microtubule splicing result and detailed display diagram disclosed in this application;

[0058] Fig.13 This is a schematic diagram of the structure of a positioning table data splicing device based on a sample structure disclosed in this application;

[0059] Fig.14 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] The existing super-resolution panoramic image stitching algorithm borrows the idea of ​​point cloud registration to perform super-resolution positioning data registration. Due to the large number of positioning points and the randomness of the excitation, when the sample structure at the stitching seam is discontinuous and the data structure is incomplete, it is impossible to accurately calculate the registration offset, resulting in errors in image registration and serious stitching misalignment, which affects subsequent analysis. The stitching performance needs to be improved.

[0062] In order to overcome the above technical problems, the present application provides a positioning table data splicing method based on a sample structure to reduce the splicing misalignment problem caused by errors in image registration.

[0063] See also Figure 1 As shown, an embodiment of the present invention discloses a positioning table data splicing method based on a sample structure, comprising:

[0064] Step S11, determining a plurality of seam positioning data in the positioning table to be spliced, and performing image rendering on the seam positioning data by using bilinear interpolation to obtain a plurality of rendered images.

[0065] In this embodiment, the positioning data of the seams in the positioning tables to be stitched are first determined, and then the images at the seams corresponding to the positioning data are rendered and reconstructed using bilinear interpolation to obtain a number of rendered images. It can be understood that since the original positioning data carried by the images to be stitched contain structural characteristics of the biological sample, such as a continuous skeleton structure, etc., and the continuous skeleton structure characteristics in the biological sample can not only make up for the shortcomings of incomplete distribution of the positioning data at the seams, but also reduce the interference of a large amount of fluorescent background noise in the positioning point alignment process on the alignment process, which helps to improve the alignment accuracy of the positioning image. Therefore, in this embodiment, the images at the seams corresponding to the positioning data in the images to be stitched are rendered and reconstructed to obtain the overall structure of the biological sample at the seam. Figure 2 The overall structure timing diagram of the positioning table data splicing method based on the sample structure of the present application is shown. Specifically, the present embodiment first obtains the corresponding positioning data to be spliced ​​from the positioning tables to be spliced, and extracts the positioning data at the seams from the positioning data to be spliced, so as to render and reconstruct the images at the seams corresponding to the positioning data at the seams by using bilinear difference, and obtains the rendered images.

[0066] It should be pointed out that the processing flow of rendering and reconstructing several seam images corresponding to the seam positioning data using bilinear interpolation in this embodiment is as follows: determine several seam images corresponding to the seam positioning data, and determine several positioning points in the several seam images; determine the fluorescent molecules corresponding to the several positioning points based on the preset pixel units, and assign pixel weights to each pixel in each of the fluorescent molecules; render the several seam images based on the pixel weights and the fluorescent molecules to obtain several rendered images. Among them, the preset pixel unit can be four pixels, and the weight allocation calculation formula for assigning pixel weights to each pixel in each of the fluorescent molecules can be specifically:

[0067] ;

[0068] in, is the tensor product, Indicates the horizontal coordinate of the nth positioning point, Indicates that the horizontal coordinate of the positioning point is fixed to the nearest integer multiple of the selected pixel size, Indicates the ordinate of the nth positioning point, Indicates that the vertical coordinate of the positioning point is fixed to the nearest integer multiple of the selected pixel size. Specifically, first determine a number of seam images corresponding to the seam positioning data, and determine a number of positioning points in the seam images, then determine the four nearest pixel points near the several positioning points respectively, that is, the four pixel points are the preset pixel units, use the four pixel points to describe the fluorescent molecules corresponding to the corresponding positioning points, and assign different pixel weights to the four nearest pixel points by weight allocation, such as Figure 3 The figure shows a weight distribution diagram of three positioning points, in which the four nearest pixels near the coordinates of the positioning points are selected respectively, and respective pixel weights are assigned to the four pixels respectively, so as to describe a fluorescent molecule through the four pixels. Finally, the images at the seams are rendered based on the pixel weights and the fluorescent molecules to obtain several rendered images. In this way, the present embodiment only renders and reconstructs the images at the seams corresponding to the positioning data at the seams in the positioning tables to be spliced, without occupying additional storage space, so as to convert the result of image registration into the registration offset of the positioning data, without affecting the subsequent quantitative analysis, and at the same time, the positioning data at the seams are quickly rendered and reconstructed into the rendered image using bilinear difference, which retains the advantages of convenient and fast histogram rendering while improving the accuracy of the algorithm, so as to obtain the overall structure of the biological sample at the seams, reduce errors, and help improve the registration accuracy of the positioning images.

[0069] Step S12: Integrate the pixel value of each pixel in the rendered images based on a linear filtering transformation to obtain a plurality of continuity enhanced images and a target integration direction and a target linear filtering transformation amplitude corresponding to each pixel in the plurality of continuity enhanced images.

[0070] In this embodiment, the pixel value of each pixel in the several rendered images is integrated using the technique of Line Filter Transform (LFT) to enhance the continuity of the sample structure of the several rendered images, thereby obtaining several continuity-enhanced images and target integration directions and target linear filter transform amplitudes corresponding to each pixel in the several continuity-enhanced images. It is understandable that due to the random activation characteristics of the single-molecule localization microscope, the data obtained by positioning have specific artifacts, which will hinder the task of image registration and affect the accuracy of registration. Therefore, this embodiment can adopt a processing method based on local context, using linear filter transform to integrate the pixel value of each pixel in the several rendered images, so as to obtain several continuity-enhanced images with enhanced sample structure features. Among them, the manifestation of the specific artifacts includes but is not limited to random fluctuations in labeling density, sample structure discontinuities caused by non-specific positioning points, etc. Figure 4a as well as Figure 4b Shown are two schematic diagrams of specific artifacts, where Figure 4a Random fluctuations in labeling density rendered for microtubules, Figure 4b The sample structure discontinuity and noise map are caused by non-specific positioning points.

[0071] It should be pointed out that the specific processing flow of step S12 of this embodiment is as follows: integrate each pixel in the plurality of rendered images according to a preset integral range and a preset integral direction to obtain a plurality of continuity enhanced images corresponding to the plurality of rendered images; record the maximum integral corresponding to each pixel, and use the angle value corresponding to the maximum integral as the target integral direction corresponding to each pixel, and use the integral value corresponding to the maximum integral as the target linear filter transformation amplitude corresponding to each pixel. The preset integral range is a rotation path of a rotating line segment with a length of twice the preset radius rotating around each pixel in the plurality of rendered images, and the preset integral direction is the angle of the rotating line segment from negative ninety degrees to ninety degrees, such as Figure 5 The figure shows a schematic diagram of the preset integration range of LFT, and the radius of the circle in the schematic diagram of the preset integration range is the preset radius. That is, this embodiment integrates the image intensity along a rotating line segment whose length is twice the preset radius at each pixel point in the several rendered images, wherein the angle of the rotating line segment is from -90° to 90°, so as to enhance the structural continuity of the sample, and obtain the several continuity enhanced images after the sample structure is enhanced, and at the same time, record the maximum integral along the radial direction during the rotation of the rotating line segment of each pixel, and the angle value when the maximum integral is obtained, and use the several recorded maximum integrals as the target integration directions corresponding to the corresponding pixels, and use the several angle values ​​as the target linear filter transformation amplitudes corresponding to the corresponding pixels. It should be further pointed out that the formula for determining the target linear filter transformation amplitude corresponding to each pixel can be specifically:

[0072] ;

[0073] in, represents the target linear filter transformation amplitude, I represents the image before processing, Indicates the position of the pixel in image I, and are respectively represented by the preset radius and direction, It means taking the maximum value of the function. The formula for determining the target integral direction corresponding to each pixel may specifically be:

[0074] ;

[0075] in, represents the target integration direction, It is understood that by calculating the above target integral direction and the above target linear filter transformation amplitude determination formula, the direction of the sample structure on the current pixel and the intensity of all fluorescent molecules in the direction corresponding to the direction can be determined, thereby enhancing the structural continuity of the sample, such as Figure 6a is the original image obtained by single-molecule localization microscopy, such as Figure 6b is the intensity image obtained after the original image is processed by LFT, such as Figure 6c is the directional image obtained after the original image is processed by LFT. In this way, this embodiment uses linear filtering transformation to integrate the pixel value of each pixel in a number of rendered images to reduce the probability of discontinuity of the sample structure, enhance the characteristics of the sample structure, and meet the requirements of robustness and accuracy in the registration task.

[0076] Step S13, using a preset linear functional to determine the target pixels that meet the preset enhancement conditions from the plurality of continuity enhanced images, and performing structural enhancement on the target pixels based on the target integral direction corresponding to the target pixels and the target linear filter transformation amplitude according to the directional filter transformation to obtain a plurality of target enhanced images.

[0077] In this embodiment, a preset linear functional function is used to determine the target pixels that meet the preset enhancement conditions from the plurality of continuous enhanced images, and based on the technology of Orientation Filter Transform (OFT), the target pixel is structurally enhanced using the target integral direction corresponding to the target pixel and the target linear filter transform amplitude, so as to obtain a plurality of target enhanced images. The target pixel that meets the preset enhancement conditions refers to the pixel with the maximum directional consistency within the area with a radius of the preset radius; the preset linear functional can be expressed as an extreme direction and a unit vector The dot product of . as well as represents the estimated value of the response variable in the linear regression model. That is, the determination formula of the preset linear functional can be specifically:

[0078] ;

[0079] Among them, when When the angle between the direction and the vector is greater than 45°, the direction and the vector are considered to be consistent and related; when When the angle between the direction and the vector is less than 45°, it can be considered that the direction and the vector are inconsistent but related; when When the value is close to zero, the angle between the direction and the vector differs by about 45°, which means that the consistency and correlation between the direction and the vector are not strong. , the current pixel can be considered as the pixel with the greatest directional consistency. It is understandable that although LFT processing of images can reduce the occurrence of sample structure discontinuity, it will also enhance artifacts such as high-density positioning points. These artifacts are common and difficult to avoid in positioning data, such as artifacts caused by calibration reference points or non-specific backgrounds. These artifacts that still exist after LFT processing will affect the subsequent registration process. Therefore, this embodiment can use OFT to process the several continuity-enhanced images after LFT enhancement to improve the registration accuracy.

[0080] It should be pointed out that the specific method of performing OFT processing on the several continuity enhanced images in this embodiment is to process each pixel in the several continuity enhanced images, and selectively enhance the target pixel with the maximum directional consistency of each pixel in the area with a radius of the preset radius. It can be understood that in order to enhance the target pixel, it is necessary to first determine the target pixel. This embodiment determines the direction of the sample structure by measuring the alignment degree between the target integral direction corresponding to each pixel and the local line segment. The specific processing flow is as follows: based on the target integral direction corresponding to each pixel and the target linear filter transformation amplitude corresponding to each pixel, the polar coordinates corresponding to each pixel are determined; the dot product between the unit vector corresponding to each pixel in the several continuity enhanced images and the polar coordinates corresponding to each pixel is calculated to obtain several pixel dot products; the dot products of several target pixels whose dot product values ​​are higher than the preset dot product threshold are determined in the several pixel dot products, and the pixels corresponding to the dot products of the several target pixels are used as target pixels that meet the preset enhancement conditions. Wherein, the preset dot product threshold is zero. That is, this embodiment can first determine the polar coordinates corresponding to each pixel through the target integral direction corresponding to each pixel and the target linear filter transformation amplitude corresponding to each pixel. Specifically, the target integral direction corresponding to each pixel is used as the polar angle of each pixel, and the target linear filter transformation amplitude corresponding to each pixel is used as the polar diameter of each pixel to determine the polar coordinates corresponding to each pixel. Then, the dot product between the unit vector corresponding to each pixel in the several continuity enhanced images and the determined polar coordinates corresponding to each pixel is calculated to obtain several pixel dot products. Finally, several target pixel dot products whose dot product values ​​are higher than the preset dot product threshold are determined from the several pixel dot products, and the pixels corresponding to the several target pixel dot products are used as target pixels that meet the preset enhancement conditions. It should be further pointed out that the determination formula for the target pixel that meets the preset enhancement conditions can be specifically:

[0081] ;

[0082] in, Represents the weight of the unit vector corresponding to the target pixel. After determining the target pixel that meets the preset enhancement condition, the target pixel can be selectively enhanced, and the processing flow is as follows: based on the preset integral range, the unit vector corresponding to the target pixel and the polar coordinates corresponding to the target pixel are subjected to dot product processing to obtain several target enhanced images corresponding to the several continuity enhanced images. That is, within a region with a radius of a preset radius, the unit vector corresponding to the target pixel and the polar coordinates corresponding to the target pixel are subjected to dot product processing to enhance the several continuity enhanced images with reduced artifacts to obtain the several target enhanced images. Among them, the calculation formula for enhancing the target pixel can be specifically:

[0083] ;

[0084] in, represents the unit vector corresponding to the target pixel, and the determination formula of the unit vector may be specifically:

[0085] ;

[0086] like Figure 7a as well as Figure 7b The enhanced image and detail display diagram after LFT enhancement are shown. Figure 7c as well as Figure 7d The image obtained after performing OFT processing on the enhanced image and the detailed display diagram are shown. In this way, after obtaining a plurality of continuous enhanced images by performing continuity enhancement processing on the plurality of rendered images, this embodiment also performs directional filter transformation on the plurality of continuous enhanced images, so as to reduce the influence of artifacts on the registration accuracy while enhancing the sample structure information, thereby improving the registration accuracy.

[0087] Step S14, determining the image registration offsets corresponding to the several target enhanced images based on the phase correlation method, and performing data splicing on the several positioning tables to be spliced ​​according to the image registration offsets to obtain a target splicing positioning table corresponding to the several positioning tables to be spliced, and then rendering the target splicing positioning table, and verifying the splicing effect of the obtained target splicing image.

[0088] In this embodiment, the phase correlation method is used to determine the image registration offset in the several target enhanced images, so as to splice the several target enhanced images according to the determined image registration offset, so as to obtain a target splicing positioning table corresponding to the several positioning tables to be spliced, and then the target splicing positioning table is rendered, and the splicing effect of the obtained target splicing image is verified to complete the work of splicing the several images to be spliced. Among them, the phase correlation method is a registration method commonly used in the field of image processing. That is, this embodiment achieves accurate alignment of the images by analyzing the phase information in the several target enhanced images. It is understandable that other image stitching algorithms are limited by the influence of pixel scale, and the stitched images can only achieve pixel-level accuracy, which will cause precision loss for positioning data and cannot meet the requirements of precise registration; at the same time, in the process of super-resolution positioning imaging, the camera mainly undergoes horizontal displacement for the sample, and its rotation transformation can usually be ignored, which can just meet the displacement result that can be obtained by the phase correlation method. Therefore, this embodiment can use the phase correlation method to calculate the image registration offset, that is, the displacement of the positioning table data, to reduce the loss of positioning data precision and meet the requirements of precise registration. Specifically, the similarity of the images is evaluated by calculating the phase correlation coefficients of the several target enhanced images, and the optimal registration position of the image is determined according to the maximum value of the phase correlation coefficient, that is, the image registration offset. That is, this embodiment can first map the several target enhanced images to the frequency domain to obtain the target impulse response function, that is, the phase correlation coefficient, by calculation, and finally determine the parameter that makes the value of the target impulse response function the largest as the optimal translation amount of the overlapping area between the images, and splice the positioning data at the enhanced seams corresponding to the several target enhanced images according to the optimal translation amount to obtain a target splicing positioning table corresponding to the several positioning tables to be spliced, and then render the target splicing positioning table, and verify the splicing effect of the obtained target splicing image. In this way, this embodiment uses the phase correlation method to splice the several target enhanced images to achieve precise alignment at the sub-pixel level, and converts the alignment result into the alignment offset of the positioning data, thereby realizing panoramic super-resolution positioning image stitching.

[0089] As can be seen from the above, the embodiment of the present application determines the positioning data at the seam from several positioning tables to be spliced, and uses bilinear interpolation to perform image rendering on the positioning data at the seam to obtain several rendered images, and enhances the several rendered images based on linear filtering transformation and directional filter transformation to obtain several target enhanced images, and finally analyzes the several target enhanced images to determine the image registration offset, and splices several positioning tables to be spliced ​​based on the image registration offset to obtain a target splicing positioning table, and then renders the target splicing positioning table, and verifies the splicing effect of the obtained target spliced ​​image. In this way, through the above process of the embodiment of the present application, on the one hand, only the images at the seams corresponding to the positioning data at the seams in the positioning tables to be stitched are rendered and reconstructed, without occupying additional storage space, so as to convert the result of image registration into the registration offset of the positioning data, and do not affect the subsequent quantitative analysis; on the one hand, the positioning data at the seams are quickly rendered and reconstructed into a rendered image by using bilinear difference, while retaining the advantages of convenient and fast histogram rendering, the accuracy of the algorithm is improved to obtain the overall structure of the biological sample at the seams, reduce errors, and help improve the registration accuracy of the positioning image; on the one hand, the pixel value of each pixel in the rendered images is integrated by linear filtering transformation to reduce the probability of occurrence of discontinuous sample structure, enhance the characteristics of the sample structure, and meet the requirements of robustness and accuracy in the registration task; on the one hand, the directional filter transformation is performed on the several continuity enhanced images, so as to reduce the influence of artifacts on the registration accuracy while enhancing the sample structure information, thereby improving the registration accuracy; on the other hand, the phase correlation method is used to splice the several target enhanced images to achieve sub-pixel level precise registration, thereby reducing the splicing misalignment problem caused by errors in image registration.

[0090] Based on the above embodiments, it can be known that, through the method of the present application, when determining the image registration offset in a plurality of target enhanced images, the image pairs to be registered in the plurality of target enhanced images can be first determined, and the image registration offsets corresponding to the image pairs to be registered can be determined according to the phase correlation method, so as to splice the positioning table to be spliced ​​according to the obtained image registration offsets. For this reason, this embodiment provides a detailed description of how to determine the image registration offsets corresponding to the image pairs to be registered. Figure 8 As shown, an embodiment of the present invention discloses a positioning table data splicing method based on a sample structure, comprising:

[0091] Step S21, determining a plurality of seam positioning data in the positioning table to be spliced, and performing image rendering on the seam positioning data by using bilinear interpolation to obtain a plurality of rendered images.

[0092] Step S22: Integrate the pixel value of each pixel in the rendered images based on a linear filtering transformation to obtain a plurality of continuity enhanced images and a target integration direction and a target linear filtering transformation amplitude corresponding to each pixel in the plurality of continuity enhanced images.

[0093] Step S23, using a preset linear functional to determine the target pixels that meet the preset enhancement conditions from the plurality of continuity enhanced images, and performing structural enhancement on the target pixels based on the target integral direction corresponding to the target pixels and the target linear filter transformation amplitude according to the directional filter transformation to obtain a plurality of target enhanced images.

[0094] Step S24, determining the image pairs to be registered in the several target enhanced images, and determining the image registration offsets corresponding to the image pairs to be registered according to the phase correlation method, so as to perform data splicing on the several positioning tables to be spliced ​​according to the image registration offsets, so as to obtain a target splicing positioning table corresponding to the several positioning tables to be spliced; the image pairs to be registered are two images in the several target enhanced images that need to be stitched.

[0095] In this embodiment, two images that need to be stitched among the several target enhanced images, that is, the image pair to be registered, are first determined, and then the stitching positioning tables corresponding to the images to be registered are stitched based on the determined image registration offset corresponding to the image pair to be registered to obtain the target stitching positioning table.

[0096] It should be pointed out that after determining the image pair to be registered, it is necessary to determine the image registration offsets corresponding to the image pair to be registered, and the processing flow is as follows: based on Fourier transform, the two images to be registered corresponding to the image pair to be registered are respectively mapped to the frequency domain to obtain a first matrix and a second matrix corresponding to the two images to be registered; the mutual power spectrum of the first matrix and the second matrix is ​​calculated, and the mutual power spectrum is inversely Fourier transformed to obtain a target impulse response function; the maximum function value corresponding to the target impulse response function is determined, and the coordinate parameters corresponding to the maximum function value are used as the image registration offset; based on the image registration offset, the data of the to-be-stitched positioning table corresponding to any image to be registered in the image pair to be registered is translated to complete the registration with the to-be-stitched positioning table corresponding to the other image to be registered in the image pair to be registered, and obtain the target splicing positioning table. That is, the image registration offset is a translation vector that enables the pair of images to be registered to achieve image alignment through displacement transformation, and the relationship between the two images to be registered corresponding to the pair of images to be registered can be determined by the formula:

[0097] ;

[0098] in, and are two images to be registered corresponding to the pair of images to be registered, is the translation vector, that is, the image registration offset. In this embodiment, the two images to be registered corresponding to the image pair to be registered are first mapped to the frequency domain using the Fourier transform method to obtain a first matrix and a second matrix corresponding to the two images to be registered. The determination formulas of the first matrix and the second matrix can be specifically:

[0099] ;

[0100] ;

[0101] in, For images The first matrix corresponding to For images The corresponding second matrix is is the Fourier transform (i.e., Discrete Fourier Transform), Represents the frequency component of the image in the horizontal direction, It is understood that the relationship between the first matrix and the second matrix can be expressed as follows:

[0102] ;

[0103] in, is the natural base, It is an imaginary unit, representing the imaginary number in mathematics That is, the two images to be registered have the same amplitude in the frequency domain and different phases. Then, the cross power spectrum of the first matrix and the second matrix is ​​calculated. The formula for determining the cross power spectrum may be:

[0104] ;

[0105] in, for The complex conjugate of for Then, the cross power spectrum is inversely transformed by Fourier transform to obtain a target impulse response function. The formula for determining the target impulse response function may be specifically:

[0106] ;

[0107] in, is the inverse Fourier transform (i.e., Inverse Discrete Fourier Transform), is the target impulse response function. Finally, the maximum function value corresponding to the target impulse response function is determined, and the coordinate parameter corresponding to the maximum function value is used as the image registration offset. The formula for determining the image registration offset may be:

[0108] ;

[0109] That is, the optimal translation amount of the overlapping area between the two images to be registered. This embodiment uses the determined image registration offset to translate the data of the positioning table to be spliced ​​corresponding to any image to be registered in the pair of images to be registered, so as to complete the registration with the positioning table to be spliced ​​corresponding to the other image to be registered in the pair of images to be registered, and obtain the target spliced ​​positioning table. In this way, this embodiment uses the phase correlation method to splice the positioning tables to be spliced, which can reduce the loss of positioning data accuracy, meet the requirements of precise registration, and achieve precise registration at the sub-pixel level.

[0110] Step S25: Render the target stitching positioning table to obtain a target stitching image, and verify the target stitching image to determine whether there are stitching traces in the target stitching image.

[0111] In this embodiment, the target stitching positioning table needs to be rendered, and then the obtained target stitching image is verified to determine the stitching effect of the target stitching positioning table, wherein the stitching effect can be determined by whether the stitching traces in the rendered target stitching image are obvious.

[0112] Step S26: If there is no stitching trace, it indicates that the target stitching positioning table corresponding to the target stitching image has passed the verification.

[0113] In this embodiment, if there is no stitching mark in the target stitching image, or the stitching mark is not obvious, it can be determined that the target stitching positioning table corresponding to the target stitching image has passed the verification, that is, the effect of stitching with the stitching positioning table can meet the user's requirements.

[0114] The specific implementation process of steps S21, S22 and S23 may refer to the contents of the aforementioned embodiments and will not be described in detail here.

[0115] As can be seen from the above, the embodiment of the present application determines the seam positioning data from a number of images to be stitched, and renders and reconstructs a number of seam images corresponding to the seam positioning data to obtain a number of rendered images, enhances the number of rendered images based on linear filtering transformation and directional filter transformation to obtain a number of target enhanced images, and finally determines the image pairs to be registered in the number of target enhanced images, and analyzes the image pairs to be registered to determine the image registration offset, and stitches the splicing positioning table according to the image registration offset to obtain a target splicing positioning table, and then the target splicing positioning table can be rendered, and the splicing effect of the obtained target spliced ​​image can be verified. In this way, through the above process of the embodiment of the present application, when determining the image registration offset in several target enhanced images, the image pairs to be registered in the several target enhanced images are first determined, and the image registration offsets corresponding to the image pairs to be registered are determined according to the phase correlation method, so as to realize the splicing of the positioning table to be spliced ​​through the obtained image registration offset, which can reduce the loss of positioning data accuracy, meet the requirements of precise registration, and realize precise registration at the sub-pixel level.

[0116] The following is an example of an image stitching process of several microscopic images collected based on a certain imaging experiment sample to illustrate the technical solution in this application.

[0117] Assume that the imaging experiment sample is COS-7 cells (an African green monkey kidney cell transformed by simian vacuolating virus), use AF647 dye (a far-red fluorescent dye) to mark the microtubule structure of the sample, and use CF568 dye (a red fluorescent dye) to mark the mitochondria of the sample. The laser power at the AF647 sample end is 704mw, and the laser power at the CE568 sample end is 1274mW. Automatic feedback adjustment is performed during the imaging process, and the signal density of the fluorescent molecule is monitored in real time, and the laser is adjusted according to the signal density. The imaging system includes a microscope, an imaging objective lens, and a detector; the camera exposure time is set to 10ms, and the imaging field of view (i.e., Field of View, FOV) size is 100μm×100μm; the imaging uses a 405nm laser to activate the sample, and then a 640nm laser is used to flash the fluorescent molecules; during imaging, AF647 is first collected, and AF647 is bleached, and then CF568 is collected. The maximum number of frames collected for each FOV of the two dyes is 5000.

[0118] This embodiment can respectively use the representative algorithms in the field of image stitching, namely JRMPC (Joint Registration of Multiple Point Clouds), ICP (Iterative Closest Point), NDT (Normal Distributions Transform) and CPD (Coherent Point Drift) and the positioning table data stitching method based on the sample structure provided in this application to perform experiments on paired registration of microtubule data with regular structure and mitochondrial data with irregular structure. Specifically, the experiment of pairwise registration of the microtubule data in two directions, horizontal and vertical, will be divided into two directions to compare the registration results respectively. The following Table 1 compares the paired registration results of five different algorithms: JRMPC, ICP, NDT, CPD and the positioning table data stitching method based on the sample structure provided in this application on microtubule data.

[0119] Table 1

[0120]

[0121] Through the analysis of Table 1 above, it can be determined that in the vertical and horizontal alignment, the MSE (Mean Squared Error) index and SSIM (Structure Similarity Index Measure) index of the positioning table data splicing method based on the sample structure provided by the present application are better than other algorithms. Among them, the MSE value is about half or less of other algorithms, and the SSIM value is close to 0.8, which indicates that the alignment result of the positioning table data splicing method based on the sample structure provided by the present application has a strong structural similarity with the actual situation. The combination of a lower MSE value and a higher SSIM value shows that the structural information of the biological sample can better solve the problem of incomplete data at the seam.

[0122] In order to observe the registration results of each algorithm more intuitively, this embodiment shows the rendering of the microtubule image seams in the horizontal and vertical directions. Fig. 9 as well as Fig.10 As shown in the figure, the horizontal direction seam registration result display diagram and the vertical direction seam registration result display diagram of each algorithm of the microtubule image provided by the present application are respectively shown. In the horizontal and vertical directions, the seam positioning data of the same position of the microtubule are enlarged to show its specific splicing details. Fig. 9The horizontal registration result display diagram shown in the figure shows that the purple box of the sample structure-based positioning table data splicing method provided by the present application in (f) has the highest similarity with the red box of the real situation in (a), and the misalignment is the least, while the yellow box of ICP in (b) has obvious misalignment. The other algorithms in (c), (d) and (e) all have slight misalignment, resulting in unclear appearance of the rendered microtubule image structure. Fig.10 It can be determined from the vertical alignment result display diagram that the purple box of the positioning table data splicing method based on the sample structure provided by the present application in Figure (f) has the highest similarity in details with the red box of the actual situation (a), while other algorithms have different degrees of misalignment compared with the actual situation at the same position due to poor alignment accuracy. This shows that the positioning table data splicing method based on the sample structure provided by the present application has better splicing accuracy on microtubules with obvious structures than other related algorithms.

[0123] In order to determine the performance of the positioning table data stitching method based on the sample structure provided by the present application for super-resolution panoramic image stitching, so as to realize the stitching of positioning data of multiple small fields of view into positioning data of a larger field of view, this embodiment conducts a panoramic super-resolution positioning image stitching experiment on 3×3 microtubule data and mitochondrial panoramic simulation data set, and compares it with the latest stitching framework based on positioning data in the field of super-resolution positioning imaging. As shown in the data in Table 2 below, it can be determined that the positioning table data stitching method based on the sample structure provided by the present application performs well in panoramic super-resolution positioning imaging stitching, whether for microtubule data or mitochondrial data, and its results are closer to the actual situation and the calculation speed is faster, which can effectively improve the performance and speed of panoramic image stitching.

[0124] Table 2

[0125]

[0126] In order to further observe the stitching details of the panoramic super-resolution positioning image, this embodiment analyzes the stitching details of different algorithms in detail. Fig.11a , Fig.11b as well as Fig.11c The data stitching results of a simulated panoramic microtubule provided in this application are respectively the data stitching results of the real situation, the data stitching results obtained by processing the stitching framework based on positioning data, and the data stitching results obtained by processing the positioning table data stitching method based on the sample structure provided in this application. After comparison, it can be seen that the two methods have no obvious misalignment in the overall structure, but the data stitching results based on the stitching framework of positioning data and the real situation have differences in details, such as Fig.11b Yellow frame contrast Fig.11aThe yellow frame has a blurry effect. The positioning table data splicing method based on the sample structure provided in this application is as follows Fig.11c As shown in the yellow box, the details are clear and there is no blur. At the same time, it is consistent with the details in the real situation. Compared with the stitching framework based on positioning data, it has a better effect in panoramic stitching.

[0127] In order to verify the stitching performance of the positioning table data stitching method based on sample structure provided by the present application in actual panoramic experimental data stitching, this embodiment compares the results of panoramic stitching tasks on microtubules with regular structures and mitochondria with irregular structures using the stitching framework based on positioning data and the positioning table data stitching method based on sample structure provided by the present application. It is understandable that in order to intuitively experience the stitching effect of the panoramic microtubule image, this embodiment does not smooth or deduplicate the stitched positioning table and image, so the image will have obvious stitching seams. Fig.12a as well as Figure 12b The splicing result and detail display of a real panoramic microtubule provided in this application show that there is no obvious misalignment in both methods from the comparison of the visual effect and details of the splicing. However, in terms of splicing details, the horizontal seams are Fig.12a The red arrow Figure 12b The red arrows and the vertical seams are as follows: Fig.12a The arrows in other color boxes are Figure 12b At the arrows in other color boxes, the positioning table data splicing method based on the sample structure provided in the present application can more accurately achieve the splicing of the seams, and the microtubule lines are clearer, the sample structure is more continuous, and the detailed information can be better presented.

[0128] Accordingly, see Fig.13 As shown, the embodiment of the present application also provides a positioning table data splicing device based on a sample structure, including:

[0129] An image rendering module 11 is used to determine the positioning data of the seams in a plurality of positioning tables to be spliced, and perform image rendering on the positioning data of the seams by using bilinear interpolation to obtain a plurality of rendered images;

[0130] A pixel integration module 12 is used to integrate the pixel value of each pixel in the plurality of rendered images based on a linear filtering transformation to obtain a plurality of continuity enhanced images and a target integration direction and a target linear filtering transformation amplitude corresponding to each pixel in the plurality of continuity enhanced images;

[0131] A pixel enhancement module 13 is used to determine a target pixel that meets a preset enhancement condition from the plurality of continuous enhanced images by using a preset linear functional, and to perform structural enhancement on the target pixel based on the target integral direction corresponding to the target pixel and the target linear filter transformation amplitude according to a directional filter transformation, so as to obtain a plurality of target enhanced images;

[0132] The positioning table stitching module 14 is used to determine the image registration offsets corresponding to the several target enhanced images based on the phase correlation method, and to perform data stitching on the several positioning tables to be stitched according to the image registration offsets to obtain a target stitching positioning table corresponding to the several positioning tables to be stitched, and then to render the target stitching positioning table and to verify the stitching effect of the obtained target stitching image.

[0133] As can be seen from the above, the embodiment of the present application determines the positioning data at the seam from several positioning tables to be spliced, and uses bilinear interpolation to render the positioning data at the seam to obtain several rendered images, enhances the several rendered images based on linear filtering transformation and directional filter transformation to obtain several target enhanced images, and finally analyzes the several target enhanced images to determine the image registration offset, and splices several positioning tables to be spliced ​​based on the image registration offset to obtain a target splicing positioning table, and then renders the target splicing positioning table, and verifies the splicing effect of the obtained target splicing image. In this way, the images to be spliced ​​are spliced ​​through the above process of the embodiment of the present application, and the sample structure of the image is enhanced by using linear filtering transformation and directional filter transformation to reduce the problems of incomplete structure and specific artifacts in the positioning data to be spliced, thereby reducing the splicing misalignment problem caused by errors in image registration.

[0134] In some specific implementations, the image rendering module 11 may specifically include:

[0135] A positioning point determination unit, used to determine a plurality of seam images corresponding to the seam positioning data, and to determine a plurality of positioning points in the plurality of seam images;

[0136] A weight allocation unit, used for determining the fluorescent molecules corresponding to the plurality of positioning points based on a preset pixel unit, and allocating a pixel weight to each pixel in each of the fluorescent molecules;

[0137] An image rendering unit is used to render the plurality of seam images based on the pixel weights and the fluorescent molecules to obtain a plurality of rendered images.

[0138] In some specific implementations, the pixel integration module 12 may specifically include:

[0139] a pixel integration unit, configured to integrate each pixel in the plurality of rendered images according to a preset integration range and a preset integration direction, so as to obtain a plurality of continuity-enhanced images corresponding to the plurality of rendered images;

[0140] The amplitude determination unit is used to record the maximum integral corresponding to each pixel, and use the angle value corresponding to the maximum integral as the target integral direction corresponding to each pixel, and use the integral value corresponding to the maximum integral as the target linear filtering transformation amplitude corresponding to each pixel.

[0141] In some specific implementations, the pixel enhancement module 13 may specifically include:

[0142] A polar coordinate determination unit, configured to determine the polar coordinates corresponding to each pixel based on a target integral direction corresponding to each pixel and a target linear filter transformation amplitude corresponding to each pixel;

[0143] a dot product calculation unit, used to calculate the dot product between the unit vector corresponding to each pixel in the plurality of continuity enhanced images and the polar coordinates corresponding to each pixel, so as to obtain a plurality of pixel dot products;

[0144] The pixel determination unit is used to determine a number of target pixel dot products whose dot product values ​​are higher than a preset dot product threshold among the number of pixel dot products, and use pixels corresponding to the number of target pixel dot products as target pixels that meet a preset enhancement condition.

[0145] In some specific implementations, the pixel enhancement module 13 may specifically include:

[0146] A dot product processing unit is used to perform dot product processing on the unit vector corresponding to the target pixel and the polar coordinates corresponding to the target pixel based on the preset integration range to obtain a plurality of target enhanced images corresponding to the plurality of continuity enhanced images.

[0147] In some specific implementations, the positioning table splicing module 14 may specifically include:

[0148] An offset determination submodule is used to determine the image pairs to be registered in the plurality of target enhanced images, and determine the image registration offsets respectively corresponding to the image pairs to be registered according to the phase correlation method, so as to perform data splicing on the plurality of positioning tables to be spliced ​​according to the image registration offsets, so as to obtain a target splicing positioning table corresponding to the plurality of positioning tables to be spliced; the image pairs to be registered are two images to be spliced ​​in the plurality of target enhanced images;

[0149] A splicing verification unit, used for rendering the target splicing positioning table to obtain a target splicing image, and verifying the target splicing image to determine whether there are splicing traces in the target splicing image;

[0150] The verification passing judgment unit is used for indicating that if there is no splicing trace, then the target splicing positioning table corresponding to the target splicing image has passed the verification.

[0151] In some specific implementations, the offset determination submodule may specifically include:

[0152] An image mapping unit, used for mapping the two images to be registered corresponding to the pair of images to be registered to the frequency domain respectively based on Fourier transform, so as to obtain a first matrix and a second matrix corresponding to the two images to be registered;

[0153] an inverse transform unit, configured to calculate a cross power spectrum of the first matrix and the second matrix, and perform an inverse Fourier transform on the cross power spectrum to obtain a target impulse response function;

[0154] An offset determination unit, used to determine a maximum function value corresponding to the target impulse response function, and use a coordinate parameter corresponding to the maximum function value as an image registration offset;

[0155] The stitching unit is used to translate the data of the to-be-stitched positioning table corresponding to any one of the to-be-registered images in the to-be-registered image pair based on the image registration offset, so as to complete the registration with the to-be-stitched positioning table corresponding to the other to-be-registered image in the to-be-registered image pair, and obtain the target stitching positioning table.

[0156] Furthermore, the present application also discloses an electronic device. Fig.14 1 is a structural diagram of an electronic device 20 according to an exemplary embodiment, and the content in the figure cannot be regarded as any limitation on the scope of use of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the positioning table data splicing method based on the sample structure disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0157] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0158] In addition, the memory 22 as a carrier for resource storage may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon may include an operating system 221, a computer program 222, etc., and the storage method may be temporary storage or permanent storage.

[0159] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, and can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program that can be used to complete the positioning table data splicing method based on the sample structure performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include computer programs that can be used to complete other specific tasks.

[0160] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned disclosed method for splicing positioning table data based on a sample structure is implemented. For the specific steps of the method, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, and no further description will be given here.

[0161] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0162] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0163] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0164] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0165] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A positioning table data splicing method based on sample structure, characterized in that: include: Determine a plurality of seam location data in the to-be-joined location tables, and use bilinear interpolation to perform image rendering on the seam location data to obtain a plurality of rendered images; Integrating the pixel value of each pixel in the plurality of rendered images based on a linear filtering transformation to obtain a plurality of continuity-enhanced images and a target integration direction and a target linear filtering transformation amplitude corresponding to each pixel in the plurality of continuity-enhanced images; Determine a target pixel that meets a preset enhancement condition from the plurality of continuous enhanced images by using a preset linear functional, and perform structural enhancement on the target pixel based on the target integral direction corresponding to the target pixel and the target linear filter transformation amplitude according to a directional filter transformation, so as to obtain a plurality of target enhanced images; Based on the phase correlation method, the image registration offsets corresponding to the several target enhanced images are determined, and data splicing is performed on the several positioning tables to be spliced ​​according to the image registration offsets to obtain a target splicing positioning table corresponding to the several positioning tables to be spliced, and then the target splicing positioning table is rendered, and the splicing effect of the obtained target spliced ​​image is verified.

2. The positioning table data splicing method based on sample structure according to claim 1 is characterized in that: The rendering and reconstruction of a plurality of seam images corresponding to the seam positioning data by bilinear interpolation to obtain a plurality of rendered images includes: Determine a plurality of seam images corresponding to the seam positioning data, and determine a plurality of positioning points in the plurality of seam images; Determine the fluorescent molecules corresponding to the plurality of positioning points based on a preset pixel unit, and assign a pixel weight to each pixel in each of the fluorescent molecules; The plurality of seam images are rendered based on the pixel weights and the fluorescent molecules to obtain a plurality of rendered images.

3. The positioning table data splicing method based on sample structure according to claim 1, characterized in that: The step of integrating the pixel value of each pixel in the plurality of rendered images based on the linear filtering transformation to obtain a plurality of continuity enhanced images and a target integration direction and a target linear filtering transformation amplitude corresponding to each pixel in the plurality of continuity enhanced images comprises: Integrating each pixel in the plurality of rendered images according to a preset integration range and a preset integration direction to obtain a plurality of continuity-enhanced images corresponding to the plurality of rendered images; The maximum integral corresponding to each pixel is recorded, and the angle value corresponding to the maximum integral is used as the target integral direction corresponding to each pixel, and the integral value corresponding to the maximum integral is used as the target linear filter transformation amplitude corresponding to each pixel.

4. The positioning table data splicing method based on sample structure according to claim 3 is characterized in that: The step of using a preset linear functional to determine a target pixel satisfying a preset enhancement condition from the plurality of continuity enhanced images includes: Determine the polar coordinates corresponding to each pixel based on the target integral direction corresponding to each pixel and the target linear filter transformation amplitude corresponding to each pixel; Calculating the dot product between the unit vector corresponding to each pixel in the plurality of continuity enhanced images and the polar coordinates corresponding to each pixel to obtain a plurality of pixel dot products; A plurality of target pixel dot products whose dot product values ​​are higher than a preset dot product threshold value are determined among the plurality of pixel dot products, and pixels corresponding to the plurality of target pixel dot products are used as target pixels that meet a preset enhancement condition.

5. The positioning table data splicing method based on sample structure according to claim 4 is characterized in that: The target pixel is enhanced based on the target integral direction corresponding to the target pixel and the target linear filter transformation amplitude according to the directional filter transformation to obtain a plurality of target enhanced images, including: Based on the preset integral range, dot product processing is performed on the unit vector corresponding to the target pixel and the polar coordinates corresponding to the target pixel to obtain a plurality of target enhanced images corresponding to the plurality of continuity enhanced images.

6. The positioning table data splicing method based on sample structure according to any one of claims 1 to 5, characterized in that: The method of determining the image registration offsets corresponding to the plurality of target enhanced images based on the phase correlation method, and performing data splicing on the plurality of positioning tables to be spliced ​​according to the image registration offsets to obtain a target splicing positioning table corresponding to the plurality of positioning tables to be spliced, and then rendering the target splicing positioning table, and verifying the splicing effect of the obtained target splicing image, includes: Determine the image pairs to be registered in the plurality of target enhanced images, and determine the image registration offsets respectively corresponding to the image pairs to be registered according to the phase correlation method, so as to perform data splicing on the plurality of positioning tables to be spliced ​​according to the image registration offsets, so as to obtain a target splicing positioning table corresponding to the plurality of positioning tables to be spliced; the image pairs to be registered are two images to be spliced ​​in the plurality of target enhanced images; Rendering the target stitching positioning table to obtain a target stitching image, and verifying the target stitching image to determine whether there are stitching traces in the target stitching image; If there is no stitching trace, then the target stitching positioning table corresponding to the target stitching image is verified.

7. The positioning table data splicing method based on sample structure according to claim 6 is characterized in that: The step of determining the image registration offsets respectively corresponding to the pair of images to be registered according to the phase correlation method, and performing data splicing on the plurality of positioning tables to be spliced ​​according to the image registration offsets, so as to obtain a target splicing positioning table corresponding to the plurality of positioning tables to be spliced, comprises: Based on Fourier transform, two images to be registered corresponding to the pair of images to be registered are respectively mapped to the frequency domain to obtain a first matrix and a second matrix corresponding to the two images to be registered; Calculating a cross power spectrum of the first matrix and the second matrix, and performing an inverse Fourier transform on the cross power spectrum to obtain a target impulse response function; Determine a maximum function value corresponding to the target impulse response function, and use a coordinate parameter corresponding to the maximum function value as an image registration offset; The data of the to-be-stitched positioning table corresponding to any one of the to-be-registered image pairs are translated based on the image registration offset to complete the registration with the to-be-stitched positioning table corresponding to another to-be-registered image in the to-be-registered image pair to obtain a target stitching positioning table.

8. A positioning table data splicing device based on sample structure, characterized in that: include: An image rendering module is used to determine the positioning data of the seams in the positioning tables to be spliced, and perform image rendering on the positioning data of the seams by using bilinear interpolation to obtain a plurality of rendered images; A pixel integration module, used for integrating the pixel value of each pixel in the plurality of rendered images based on a linear filtering transformation, so as to obtain a plurality of continuity-enhanced images and a target integration direction and a target linear filtering transformation amplitude corresponding to each pixel in the plurality of continuity-enhanced images; A pixel enhancement module, used to determine a target pixel satisfying a preset enhancement condition from the plurality of continuous enhanced images by using a preset linear functional, and to perform structural enhancement on the target pixel based on the target integral direction corresponding to the target pixel and the target linear filter transformation amplitude according to a directional filter transformation, so as to obtain a plurality of target enhanced images; A positioning table stitching module is used to determine the image registration offsets corresponding to the several target enhanced images based on the phase correlation method, and to perform data stitching on the several positioning tables to be stitched according to the image registration offsets to obtain a target stitching positioning table corresponding to the several positioning tables to be stitched, and then to render the target stitching positioning table and to verify the stitching effect of the obtained target stitching image.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor is used to execute the computer program to implement the positioning table data splicing method based on the sample structure as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program; wherein, when the computer program is executed by a processor, the positioning table data splicing method based on the sample structure as described in any one of claims 1 to 7 is implemented.

Images