Positioning table data splicing method and device based on sample structure, equipment and medium

Abstract

The application discloses a sample structure-based positioning table data splicing method and device, equipment and medium, and relates to the technical field of image processing. The method comprises the following steps: determining the splicing seam position data in the to-be-spliced positioning table, performing image rendering on the splicing seam position data by using bilinear interpolation to obtain a plurality of rendered images; integrating the pixel value size of each pixel in the plurality of rendered images based on linear filter transformation to obtain a plurality of continuity-enhanced images; determining a target pixel meeting a preset enhancement condition by using a preset linear functional, and enhancing the target pixel to obtain a plurality of target-enhanced images; determining an image registration offset, splicing the to-be-spliced positioning table according to the image registration offset to obtain a target spliced positioning table, then rendering the target spliced positioning table, and verifying the splicing effect of the obtained target spliced image. Thus, the splicing dislocation problem caused by the error of image registration can be reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular relates to a positioning table data splicing method and device based on sample structure, equipment and medium. BACKGROUND

[0002] Super-resolution microscopy breaks the optical diffraction limit, enabling biologists to resolve the ultrastructure of biological molecules at nanometer resolution. Due to the size of the micrograph collected at a time is often smaller than the size of the biological sample. Therefore, it is necessary to splice a large number of small field super-resolution images to form a super-resolution panoramic image to meet the needs of pathologists to observe the microstructure of the entire sample. However, the existing super-resolution panoramic image splicing algorithm borrows the idea of point cloud registration to register super-resolution positioning data. Affected by the large number of positioning points and random excitation, when the sample structure is discontinuous at the seam and the data structure is incomplete, the registration offset cannot be accurately calculated, resulting in errors in image registration, and thus serious misregistration in splicing, affecting subsequent analysis, and the splicing performance needs to be improved.

[0003] From the above, it can be seen that how to reduce the misregistration problem caused by errors in image registration is a technical problem to be solved at present. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a positioning table data splicing method and device based on sample structure, which can reduce the misregistration problem caused by errors in image registration. The specific scheme is as follows:

[0005] In a first aspect, the present application provides a positioning table data splicing method based on sample structure, comprising:

[0006] Determine the seam positioning data in the plurality of to-be-spliced positioning tables, and perform image rendering on the seam positioning data using bilinear interpolation to obtain a plurality of rendered images;

[0007] Integrate the pixel value size of each pixel in the plurality of rendered images based on linear filter transformation to obtain a plurality of continuity-enhanced images, and a target integral direction and a target linear filter transformation amplitude corresponding to each pixel in the plurality of continuity-enhanced images;

[0008] Determine a target pixel satisfying a preset enhancement condition from the plurality of continuity-enhanced images using a preset linear functional, and perform structure enhancement on the target pixel based on the target integral direction and the target linear filter transformation amplitude corresponding to the target pixel according to the direction filter transformation to obtain a plurality of target enhanced images;

[0009] determine image registration offsets corresponding to the plurality of target enhanced images based on the phase correlation method, and perform data stitching on the plurality of to-be-stitched positioning tables according to the image registration offsets to obtain a target stitched positioning table corresponding to the plurality of to-be-stitched positioning tables, then render the target stitched positioning table, and perform stitching effect verification on the obtained target stitched image.

[0010] Optionally, the rendering reconstruction of the plurality of joint seam images corresponding to the joint seam positioning data by using the bilinear interpolation comprises:

[0011] determine a plurality of joint seam images corresponding to the joint seam positioning data, and determine a plurality of positioning points in the plurality of joint seam images;

[0012] determine a plurality of fluorescent molecules corresponding to the plurality of positioning points based on a preset pixel unit, and assign a pixel weight to each pixel in each of the fluorescent molecules;

[0013] render the plurality of joint seam images based on the pixel weight and the fluorescent molecules to obtain a plurality of rendered images.

[0014] Optionally, the integration of the pixel value size of each pixel in the plurality of rendered images based on the linear filter transformation comprises:

[0015] integrate each pixel in the plurality of rendered images according to a preset integration range and a preset integration direction to obtain a plurality of continuity enhanced images corresponding to the plurality of rendered images;

[0016] record the maximum integration corresponding to each pixel, and take the angle value corresponding to the maximum integration as the target integration direction corresponding to each pixel, and take the integration value corresponding to the maximum integration as the target linear filter transformation amplitude value corresponding to each pixel.

[0017] Optionally, the determination of a target pixel satisfying a preset enhancement condition from the plurality of continuity enhanced images by using a preset linear functional comprises:

[0018] determine the polar coordinates corresponding to each pixel based on the target integration direction corresponding to each pixel and the target linear filter transformation amplitude value corresponding to each pixel;

[0019] calculate the dot product between the unit vector corresponding to each pixel in the plurality of continuity enhanced images and the polar coordinates corresponding to each pixel to obtain a plurality of pixel dot products;

[0020] Determine a plurality of target pixel products whose dot product values are higher than a preset dot product threshold value in the plurality of pixel products, and take the pixels corresponding to the plurality of target pixel products as target pixels satisfying a preset enhancement condition.

[0021] Optionally, the target pixel is enhanced according to the target integral direction corresponding to the target pixel and the target linear filter transform amplitude value to obtain a plurality of target enhanced images based on the direction filter transform, including:

[0022] The unit vector corresponding to the target pixel and the polar coordinate corresponding to the target pixel are dot product processed based on the preset integral range to obtain a plurality of target enhanced images corresponding to the plurality of continuity enhanced images.

[0023] Optionally, the image registration offset corresponding to the plurality of target enhanced images is determined based on the phase correlation method, and the plurality of to-be-stitched positioning tables are data-stitched according to the image registration offset to obtain a target stitching positioning table corresponding to the plurality of to-be-stitched positioning tables, and then the target stitching positioning table is rendered, and the target stitching image obtained is verified for stitching effect, including:

[0024] A to-be-registered image pair in the plurality of target enhanced images is determined, and the image registration offset corresponding to the to-be-registered image pair is determined according to the phase correlation method, so as to data-stitch the plurality of to-be-stitched positioning tables according to the image registration offset to obtain a target stitching positioning table corresponding to the plurality of to-be-stitched positioning tables; the to-be-registered image pair is two images in the plurality of target enhanced images that need to be image-stitched;

[0025] The target stitching positioning table is rendered to obtain a target stitching image, and the target stitching image is verified to determine whether there is a stitching trace in the target stitching image;

[0026] If there is no stitching trace, it is indicated that the target stitching positioning table corresponding to the target stitching image passes the verification.

[0027] Optionally, the image registration offset corresponding to the to-be-registered image pair is determined, and the plurality of to-be-stitched positioning tables are data-stitched according to the image registration offset to obtain a target stitching positioning table corresponding to the plurality of to-be-stitched positioning tables, including:

[0028] The two to-be-registered images corresponding to the to-be-registered image pair are respectively mapped to a frequency domain based on Fourier transform to obtain a first matrix and a second matrix corresponding to the two to-be-registered images;

[0029] calculating a cross-power spectrum of the first matrix and the second matrix, and performing an inverse Fourier transform on the cross-power spectrum to obtain a target impulse response function;

[0030] determining a maximum function value corresponding to the target impulse response function, and taking a coordinate parameter corresponding to the maximum function value as an image registration offset;

[0031] translating data of a to-be-stitched positioning table corresponding to any one of the to-be-registered image pairs based on the image registration offset, to complete registration of a to-be-stitched positioning table corresponding to another to-be-registered image in the to-be-registered image pairs, and obtain a target stitching positioning table.

[0032] In a second aspect, the present application provides a positioning table data stitching device based on a sample structure, comprising:

[0033] an image rendering module, configured to determine stitching seam positioning data in a plurality of to-be-stitched positioning tables, and perform image rendering on the stitching seam positioning data by using bilinear interpolation to obtain a plurality of rendered images;

[0034] a pixel integration module, configured to integrate pixel value sizes of each pixel in the plurality of rendered images based on a linear filter transform to obtain a plurality of continuity-enhanced images, a target integration direction corresponding to each pixel in the plurality of continuity-enhanced images, and a target linear filter transform amplitude value;

[0035] a pixel enhancement module, configured to determine a target pixel satisfying a preset enhancement condition from the plurality of continuity-enhanced images by using a preset linear functional, and perform structure enhancement on the target pixel based on the target integration direction corresponding to the target pixel and the target linear filter transform amplitude value according to a directional filter transform to obtain a plurality of target enhanced images;

[0036] a positioning table stitching module, configured to determine an image registration offset corresponding to the plurality of target enhanced images based on a phase correlation method, and perform data stitching on the plurality of to-be-stitched positioning tables according to the image registration offset to obtain a target stitching positioning table corresponding to the plurality of to-be-stitched positioning tables, then render the target stitching positioning table, and verify a stitching effect of a target stitching image obtained.

[0037] In a third aspect, the present application provides an electronic device, comprising:

[0038] a memory, configured to save a computer program;

[0039] a processor, configured to execute the computer program to implement the foregoing positioning table data stitching method based on a sample structure.

[0040] In a fourth aspect, the present application provides a computer readable storage medium for storing a computer program, wherein the computer program is executed by a processor to implement the aforementioned sample structure-based positioning table data splicing method.

[0041] In the present application, the positioning data at the seams in the plurality of to-be-spliced positioning tables is determined, and the positioning data at the seams is rendered into a plurality of rendered images by using bilinear interpolation. The pixel value of each pixel in the plurality of rendered images is integrated based on a linear filter transform to obtain a plurality of continuity-enhanced images, a target integral direction corresponding to each pixel in the plurality of continuity-enhanced images, and a target linear filter transform amplitude. A target pixel satisfying a preset enhancement condition is determined from the plurality of continuity-enhanced images by using a preset linear functional, and the target pixel is structurally enhanced based on the target integral direction and the target linear filter transform amplitude corresponding to the target pixel according to a directional filter transform to obtain a plurality of target enhanced images. The image registration offset corresponding to the plurality of target enhanced images is determined based on a phase correlation method, and the plurality of to-be-spliced positioning tables are spliced based on the image registration offset to obtain a target spliced positioning table corresponding to the plurality of to-be-spliced positioning tables. Then, the target spliced positioning table is rendered, and the target spliced image obtained is verified for splicing effect. As can be seen from the above, in the present application, the positioning data at the seams in the plurality of to-be-spliced positioning tables is determined, and the positioning data at the seams is rendered into a plurality of rendered images by using bilinear interpolation. The plurality of rendered images are enhanced based on a linear filter transform and a directional filter transform to obtain a plurality of target enhanced images. Finally, the plurality of target enhanced images are analyzed to determine an image registration offset, and the plurality of to-be-spliced positioning tables are spliced based on the image registration offset to obtain a target spliced positioning table. Then, the target spliced positioning table is rendered, and the target spliced image obtained is verified for splicing effect. In this way, by splicing the to-be-spliced positioning tables through the above process of the present application, the sample structure of the image is enhanced by using a linear filter transform and a directional filter transform, so as to reduce the problems of incomplete structure and specific artifacts in the to-be-spliced positioning data, and further reduce the misalignment problem caused by errors in image registration. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on the provided drawings.

[0043] Figure 1A flow chart of a sample structure-based positioning table data splicing method disclosed in the present application;

[0044] Figure 2 A whole structure timing diagram of a sample structure-based positioning table data splicing method disclosed in the present application;

[0045] Figure 3 A weight distribution diagram of three positioning points disclosed in the present application;

[0046] Figure 4a And Figure 4b Two specific artifact diagrams disclosed in the present application;

[0047] Figure 5 A preset integral range diagram of a linear filter transformation disclosed in the present application;

[0048] Figure 6a An original image obtained by a single molecule localization microscope disclosed in the present application;

[0049] Figure 6b An intensity image obtained by linear filter transformation processing of the original image disclosed in the present application;

[0050] Figure 6c A direction image obtained by linear filter transformation processing of the original image disclosed in the present application;

[0051] Figure 7a And Figure 7b An enhanced image and a detail display diagram obtained by linear filter transformation disclosed in the present application;

[0052] Figure 7c And Figure 7d An image obtained by direction filter transformation processing of the enhanced image disclosed in the present application and a detail display diagram;

[0053] Figure 8 A specific sample structure-based positioning table data splicing method flow chart disclosed in the present application;

[0054] Figure 9 A microtubule image horizontal direction splicing registration result display diagram of each algorithm disclosed in the present application;

[0055] Figure 10 A microtubule image vertical direction splicing registration result display diagram of each algorithm disclosed in the present application;

[0056] Figures 11a to 11c A data splicing result diagram of a simulated panoramic microtubule disclosed in the present application;

[0057] Figure 12aWith Figure 12b A real panorama microtubule splicing result and a detail display diagram disclosed by the application;

[0058] Figure 13 A positioning table data splicing device structure schematic diagram based on sample structure disclosed by the application;

[0059] Figure 14 An electronic device structure diagram disclosed by the application. DETAILED DESCRIPTION

[0060] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative work belong to the protection scope of the application.

[0061] The existing super-resolution panoramic image splicing algorithm borrows the idea of point cloud registration to perform super-resolution positioning data registration. Influenced by the large number of positioning points and random excitation, when the sample structure at the seam is discontinuous and the data structure is incomplete, the registration offset cannot be accurately calculated, errors occur in image registration, and serious misalignment occurs in splicing, which affects subsequent analysis, and the splicing performance needs to be improved.

[0062] In order to overcome the above technical problems, the application provides a positioning table data splicing method based on sample structure to reduce the misalignment problem caused by errors in image registration.

[0063] Referring to Figure 1 The embodiments of the application disclose a positioning table data splicing method based on sample structure, which comprises the following steps:

[0064] Step S11, determine the seam positioning data in the plurality of to-be-spliced positioning tables, and perform image rendering on the seam positioning data by using bilinear interpolation to obtain a plurality of rendered images.

[0065] In this embodiment, the positioning data of the seams in the positioning tables to be spliced ​​are first determined, and then the bilinear difference is used to render and reconstruct the images at the seams corresponding to the positioning data at the seams to obtain a number of rendered images. It can be understood that since the original positioning data carried by the images to be spliced ​​contain the structural characteristics of the biological sample, such as the continuous skeleton structure, etc., and the continuous skeleton structure characteristics in the biological sample can not only make up for the shortcomings of incomplete distribution of the positioning data at the seams, but also reduce the interference of a large amount of fluorescent background noise in the positioning point registration process on the registration process, which helps to improve the registration accuracy of the positioning image. Therefore, in this embodiment, the images at the seams corresponding to the positioning data at the seams in the images to be spliced ​​are rendered and reconstructed to obtain the overall structure of the biological sample at the seams. Figure 2 The overall structure sequence diagram of the positioning table data splicing method based on a sample structure of the present application is shown. Specifically, this embodiment first obtains corresponding positioning data to be spliced ​​from multiple positioning tables to be spliced, and then extracts the multiple seam positioning data from the positioning data to be spliced, and then uses bilinear interpolation to render and reconstruct multiple seam images corresponding to the seam positioning data to obtain multiple rendered images.

[0066] It should be noted that the processing flow of rendering and reconstructing the plurality of seam images corresponding to the seam positioning data using bilinear interpolation in this embodiment is as follows: determining the plurality of seam images corresponding to the seam positioning data, and determining the plurality of positioning points in the plurality of seam images; determining the fluorescent molecules corresponding to the plurality of positioning points based on preset pixel units, and assigning pixel weights to each pixel in each of the fluorescent molecules; rendering the plurality of seam images based on the pixel weights and the fluorescent molecules to obtain a plurality of rendered images. Among them, the preset pixel unit can be four pixels, and the weight allocation calculation formula for assigning pixel weights to each pixel in each of the fluorescent molecules can be specifically:

[0067] ;

[0068] in, is the tensor product, Indicates the horizontal coordinate of the nth positioning point, Indicates that the horizontal coordinate of the positioning point is fixed to the nearest integer multiple of the selected pixel size, Indicates the vertical coordinate of the nth positioning point, The positioning point ordinate is fixed as an integer multiple closest to the selected pixel size. Specifically, a plurality of seam images corresponding to the seam positioning data are determined, and a plurality of positioning points in the plurality of seam images are determined. Then, the nearest four pixel points near the plurality of positioning points are determined, i.e., the four pixel points are the preset pixel units. The corresponding fluorescent molecules of the positioning points are described by using the four pixel points, and different pixel weights are assigned to the nearest four pixel points by weight distribution. Figure 3 As shown in a weight distribution diagram of three positioning points, the nearest four pixels near the positioning point coordinates are selected, and the four pixels are respectively assigned with respective pixel weights to describe a fluorescent molecule by using the four pixels. Finally, the plurality of seam images are rendered based on the pixel weights and the fluorescent molecules to obtain a plurality of rendered images. In this way, the embodiment only renders the plurality of seam images corresponding to the seam positioning data in the plurality of to-be-stitched positioning tables, does not occupy extra storage space, converts the registration result of the image registration into a registration offset of the positioning data, does not affect subsequent quantitative analysis, quickly renders the seam positioning data to the rendered images by using the bilinear difference value, retains the advantages of the histogram rendering, improves the accuracy of the algorithm, obtains the overall structure of the seam biological sample, reduces the error, and helps to improve the registration accuracy of the positioning image.

[0069] In step S12, the pixel value size of each pixel in the plurality of rendered images is integrated based on a linear filter transform to obtain a plurality of continuity enhancement images and a target integral direction and a target linear filter transform amplitude corresponding to each pixel in the plurality of continuity enhancement images.

[0070] In the embodiment, the pixel value size of each pixel in the plurality of rendered images is integrated by using the linear filter transform (Line Filter Transform, LFT) to enhance the sample structure continuity of the plurality of rendered images, and a plurality of continuity enhancement images and a target integral direction and a target linear filter transform amplitude corresponding to each pixel in the plurality of continuity enhancement images are obtained. It can be understood that due to the random activation characteristics of the single molecule localization microscope, the positioning data obtained has specific artifacts, which hinders the image registration task and affects the accuracy of the registration. Therefore, the embodiment can adopt a processing method based on local context, integrate the pixel value size of each pixel in the plurality of rendered images by using the linear filter transform, and obtain a plurality of continuity enhancement images with enhanced sample structure features. The specific artifacts include but are not limited to random fluctuations of marker density, sample structure discontinuity caused by non-specific positioning points, etc.Figure 4a and Figure 4b are two specific artifact schematic diagrams, wherein, Figure 4a is a random fluctuation image of the marker density of the microtubule rendering, Figure 4b is a sample structure discontinuity caused by a non-specific positioning point and a noise map.

[0071] It should be pointed out that the specific process flow of step S12 of the embodiment is as follows: integrating each pixel in the plurality of rendered images according to a preset integration range and a preset integration direction to obtain a plurality of continuity enhancement images corresponding to the plurality of rendered images; recording the maximum integration corresponding to each pixel, taking the angle value corresponding to the maximum integration as the target integration direction corresponding to each pixel, and taking the integration value corresponding to the maximum integration as the target linear filter transform amplitude value corresponding to each pixel. Wherein, the preset integration range is a rotating path obtained by rotating a rotating line segment with a length of twice a preset radius with each pixel point in the plurality of rendered images as the center, and the preset integration direction is an angle of the rotating line segment from negative ninety degrees to ninety degrees, as shown in Figure 5 is a preset integration range schematic diagram of LFT, and the radius of the circle in the preset integration range schematic diagram is the preset radius. That is, in the embodiment, the image intensity is integrated along the rotating line segment with a length of twice the preset radius at each pixel point in the plurality of rendered images, wherein the angle of the rotating line segment is from -90° to 90°, so as to enhance the structural continuity of the sample, obtain the plurality of continuity enhancement images after the structural continuity of the sample is enhanced, record the maximum integration along the radial direction in the rotating process of the rotating line segment of each pixel, obtain the angle value at the maximum integration, and take the plurality of recorded maximum integrations as the target integration direction corresponding to the corresponding pixel respectively, and take the plurality of angle values as the target linear filter transform amplitude value corresponding to the corresponding pixel. It should be further pointed out that the determination formula of the target linear filter transform amplitude value corresponding to each pixel can be specifically:

[0072] ;

[0073] wherein, represents the target linear filter transform amplitude value, I represents the image before processing, represents the position of the pixel in the image I, and respectively represent the preset radius and direction, represents taking the maximum value of the function, is a summation symbol. The determination formula of the target integration direction corresponding to each pixel can be specifically:

[0074] ;

[0075] wherein, denotes the target integral direction, denotes the parameter that makes the function value maximum. It can be understood that, through the calculation of the above target integral direction and the determination formula of the above target linear filter transform amplitude, the orientation of the sample structure on the current pixel and the intensity of all fluorescent molecules in the direction corresponding to the orientation can be determined, so as to enhance the structural continuity of the sample, such as Figure 6a is the original image obtained by single molecule localization microscopy observation, such as Figure 6b is the intensity image obtained after the original image is processed by LFT, such as Figure 6c is the direction image obtained after the original image is processed by LFT. In this way, the embodiment utilizes linear filter transform to integrate the pixel value size of each pixel in a plurality of rendered images, so as to reduce the probability of occurrence of sample structure discontinuity, enhance the features of sample structure, and meet the requirements of robustness and accuracy in registration tasks.

[0076] In step S13, a target pixel satisfying a preset enhancement condition is determined from the plurality of continuity enhanced images by using a preset linear functional, and structure enhancement is performed on the target pixel based on the target integral direction corresponding to the target pixel and the target linear filter transform amplitude based on the orientation filter transform, so as to obtain a plurality of target enhanced images.

[0077] In the embodiment, a target pixel satisfying a preset enhancement condition is determined from the plurality of continuity enhanced images by using a preset linear functional, and structure enhancement is performed on the target pixel based on the target integral direction corresponding to the target pixel and the target linear filter transform amplitude based on the orientation filter transform (i.e., Orientation Filter Transform, OFT), so as to obtain a plurality of target enhanced images. The target pixel satisfying the preset enhancement condition refers to a pixel having maximum directional consistency within a field with a radius of the preset radius; the preset linear functional can be represented as the dot product of a polar direction and a unit vector . Wherein, and denote the estimated value of the response variable in the linear regression model. That is, the determination formula of the preset linear functional can be specifically:

[0078] ;

[0079] When the angle between the direction and the vector is greater than 45°, it can be considered that the direction and the vector are consistent and related; when ​When the angle between the direction and the vector is less than 45°, the direction and the vector can be considered to be inconsistent but related; when When the value of the angle between the direction and the vector is close to zero, the angle between the direction and the vector is about 45°, and the consistency and the correlation of the direction and the vector are not strong, and thus, when the current pixel can be considered to be a pixel with the maximum direction consistency. It can be understood that although the LFT processing of the image can reduce the occurrence of the sample structure discontinuity, the high-density positioning points and other artifacts are also enhanced, which are common in the positioning data and difficult to avoid, such as artifacts caused by the calibration reference points or non-specific backgrounds, and the artifacts still existing after the LFT processing affect the subsequent registration process, and thus, the embodiment can adopt the OFT to process the plurality of continuity-enhanced images enhanced by the LFT to improve the registration accuracy.

[0080] It should be noted that the specific method of OFT processing of the plurality of continuity enhanced images in the embodiment is to process each pixel in the plurality of continuity enhanced images, and selectively enhance the target pixel having maximum directional consistency in the field with a radius of the preset radius. It can be understood that in order to enhance the target pixel, the target pixel needs to be determined first. The embodiment determines the direction of the sample structure by measuring the alignment degree between the target integral direction corresponding to each pixel and the local line segment. The specific processing procedure is as follows: based on the target integral direction corresponding to each pixel and the target linear filter transform amplitude corresponding to each pixel, the polar coordinates corresponding to each pixel are determined; the dot product between the unit vector corresponding to each pixel in the plurality of continuity enhanced images and the polar coordinates corresponding to each pixel is calculated to obtain a plurality of pixel dot products; a plurality of target pixel dot products whose dot product values are higher than a preset dot product threshold are determined from the plurality of pixel dot products, and the pixels corresponding to the plurality of target pixel dot products are taken as target pixels satisfying a preset enhancement condition. The preset dot product threshold is zero. That is, the polar coordinates corresponding to each pixel can be determined by the target integral direction corresponding to each pixel and the target linear filter transform amplitude corresponding to each pixel. Specifically, the target integral direction corresponding to each pixel is taken as the polar angle of each pixel, and the target linear filter transform amplitude corresponding to each pixel is taken as the polar radius of each pixel to determine the polar coordinates corresponding to each pixel. Then the dot product between the unit vector corresponding to each pixel in the plurality of continuity enhanced images and the determined polar coordinates corresponding to each pixel is calculated to obtain a plurality of pixel dot products. Finally, a plurality of target pixel dot products whose dot product values are higher than a preset dot product threshold are determined from the plurality of pixel dot products, and the pixels corresponding to the plurality of target pixel dot products are taken as target pixels satisfying a preset enhancement condition. It should be further pointed out that the determination formula of the target pixel satisfying the preset enhancement condition can be specifically:

[0081] ;

[0082] wherein, represents the weight of the unit vector corresponding to the target pixel. After determining the target pixel satisfying the preset enhancement condition, the target pixel can be selectively enhanced, and the processing flow is as follows: based on the preset integral range, the unit vector corresponding to the target pixel and the polar coordinates corresponding to the target pixel are dot product processed to obtain a plurality of target enhancement images corresponding to the plurality of continuity enhancement images. That is, the unit vector corresponding to the target pixel and the polar coordinates corresponding to the target pixel are dot product processed within a field with a preset radius to enhance the artifact reduction of the plurality of continuity enhancement images to obtain the plurality of target enhancement images. The calculation formula for enhancing the target pixel can be specifically:

[0083] ;

[0084] wherein, represents the unit vector corresponding to the target pixel, and the determination formula of the unit vector can be specifically:

[0085] ;

[0086] As shown in Figure 7a and Figure 7b are the enhanced images and detail display images after LFT enhancement. As shown in Figure 7c and Figure 7d are the images and detail display images obtained after OFT processing of the enhanced images. In this way, after obtaining the plurality of continuity enhancement images for continuity enhancement processing of the plurality of rendered images, the plurality of continuity enhancement images are subjected to a direction filter transformation, so as to enhance the sample structure information while reducing the influence of artifacts on the registration accuracy, thereby improving the registration accuracy.

[0087] In step S14, the image registration offset corresponding to the plurality of target enhancement images is determined based on the phase correlation method, and the plurality of to-be-stitched positioning tables are data-stitched according to the image registration offset to obtain a target stitching positioning table corresponding to the plurality of to-be-stitched positioning tables. Then, the target stitching positioning table is rendered, and the obtained target stitching image is subjected to stitching effect verification.

[0088] In this embodiment, the phase correlation method is used to determine the image registration offset in the plurality of target enhanced images, to stitch the plurality of target enhanced images according to the determined image registration offset, to obtain a target stitched positioning table corresponding to the plurality of to-be-stitched positioning tables, then render the target stitched positioning table, and perform stitching effect verification on the obtained target stitched image, to complete the stitching of the plurality of to-be-stitched images. The phase correlation method is a registration method commonly used in the field of image processing. That is, this embodiment realizes accurate alignment of images by analyzing phase information in the plurality of target enhanced images. It can be understood that other image stitching algorithms will be affected by pixel size, resulting in images stitched by the algorithms only reaching pixel-level accuracy, which will cause a loss of accuracy in positioning data and cannot meet the requirement of accurate registration. Meanwhile, in the process of super-resolution positioning imaging, the camera mainly undergoes horizontal displacement relative to the sample, and its rotation transformation can be generally ignored, which can meet the displacement result obtained by the phase correlation method. Therefore, this embodiment can use the phase correlation method to calculate the image registration offset, i.e., the displacement of the positioning table data, to reduce the loss of positioning data accuracy and meet the requirement of accurate registration. Specifically, the phase correlation coefficient of the plurality of target enhanced images is calculated to evaluate the similarity of the images, and the maximum value of the phase correlation coefficient is used to determine the best registration position of the images, i.e., the image registration offset. That is, this embodiment can first map the plurality of target enhanced images to the frequency domain, to calculate the target impulse response function, i.e., the phase correlation coefficient, and finally determine the parameter that makes the value of the target impulse response function maximum as the best translation amount of the overlapping region between the images, to stitch the positioning data at the enhanced seam according to the best translation amount, to obtain a target stitched positioning table corresponding to the plurality of to-be-stitched positioning tables, then render the target stitched positioning table, and perform stitching effect verification on the obtained target stitched image. In this way, this embodiment uses the phase correlation method to stitch the plurality of target enhanced images to realize accurate registration at the sub-pixel level, and converts the registration result into the registration offset of the positioning data, thereby realizing panoramic super-resolution positioning image stitching.

[0089] As can be seen from the above, the embodiment of the present application determines the joint location data in the plurality of to-be-stitched location tables, and performs image rendering on the joint location data by using bilinear interpolation to obtain a plurality of rendered images, enhances the plurality of rendered images based on linear filter transformation and directional filter transformation to obtain a plurality of target enhanced images, finally analyzes the plurality of target enhanced images to determine the image registration offset, and stitches the plurality of to-be-stitched location tables based on the image registration offset to obtain a target stitched location table, then renders the target stitched location table, and verifies the stitching effect of the obtained target stitched image. In this way, through the above process of the embodiment of the present application, on the one hand, only the plurality of joint images corresponding to the joint location data in the plurality of to-be-stitched location tables are rendered and reconstructed, without occupying additional storage space, so that the result of image registration is converted into the registration offset of location data, without affecting subsequent quantitative analysis; on the other hand, the joint location data is quickly rendered and reconstructed into rendered images by using bilinear interpolation, which retains the advantages of histogram rendering, such as convenience and speed, while improving the accuracy of the algorithm, so as to obtain the overall structure of the joint biological sample, reduce errors, and help improve the registration accuracy of the location image; on the one hand, the pixel value of each pixel in the plurality of rendered images is integrated by using linear filter transformation, so as to reduce the probability of occurrence of discontinuous sample structures and enhance the features of the sample structures, so as to meet the requirements of robustness and accuracy in the registration task; on the one hand, the plurality of continuity-enhanced images are subjected to directional filter transformation, so as to enhance the sample structure information while reducing the influence of artifacts on the registration accuracy, thereby improving the registration accuracy; on the other hand, the plurality of target enhanced images are stitched by using the phase correlation method to realize accurate registration at the sub-pixel level, thereby reducing the misregistration problem caused by errors in image registration.

[0090] Based on the foregoing embodiment, through the method of the present application, when determining the image registration offset in the plurality of target enhanced images, the to-be-registered image pair in the plurality of target enhanced images can be determined first, and the image registration offset corresponding to the to-be-registered image pair is determined according to the phase correlation method, so as to stitch the to-be-stitched location table according to the obtained image registration offset, and for this purpose, the embodiment of the present application makes a detailed description of how to determine the image registration offset corresponding to the to-be-registered image pair. Referring to Figure 8 The embodiment of the present application discloses a sample structure-based location table data stitching method, which comprises the following steps:

[0091] In step S21, the joint location data in the plurality of to-be-stitched location tables is determined, and the joint location data is subjected to image rendering by using bilinear interpolation to obtain a plurality of rendered images.

[0092] Step S22, integrating pixel value sizes of each pixel in the plurality of rendered images based on the linear filter transform to obtain a plurality of continuity enhanced images and a target integral direction and a target linear filter transform amplitude corresponding to each pixel in the plurality of continuity enhanced images.

[0093] Step S23, determining a target pixel satisfying a preset enhancement condition from the plurality of continuity enhanced images using a preset linear functional, and performing structure enhancement on the target pixel based on the target integral direction and the target linear filter transform amplitude corresponding to the target pixel according to the directional filter transform to obtain a plurality of target enhanced images.

[0094] Step S24, determining a to-be-registered image pair in the plurality of target enhanced images, and determining an image registration offset corresponding to the to-be-registered image pair respectively according to a phase correlation method, to perform data stitching on the plurality of to-be-stitched positioning tables according to the image registration offset to obtain a target stitched positioning table corresponding to the plurality of to-be-stitched positioning tables; the to-be-registered image pair is two images in the plurality of target enhanced images that need to be stitched.

[0095] In this embodiment, two images in the plurality of target enhanced images that need to be stitched, i.e., a to-be-registered image pair, are determined first, and then the to-be-stitched positioning tables corresponding to the to-be-registered image pair are stitched based on the image registration offset corresponding to the to-be-registered image pair to obtain a target stitched positioning table.

[0096] It should be noted that after the to-be-registered image pair is determined, the image registration offset corresponding to the to-be-registered image pair respectively needs to be determined, and the processing procedure is as follows: the two to-be-registered images corresponding to the to-be-registered image pair are respectively mapped to a frequency domain based on Fourier transform to obtain a first matrix and a second matrix corresponding to the two to-be-registered images; the cross power spectrum of the first matrix and the second matrix is calculated, and inverse Fourier transform is performed on the cross power spectrum to obtain a target impulse response function; the maximum function value corresponding to the target impulse response function is determined, and the coordinate parameter corresponding to the maximum function value is taken as an image registration offset; the data of the to-be-stitched positioning table corresponding to any to-be-registered image in the to-be-registered image pair is translated based on the image registration offset to complete the registration of the to-be-stitched positioning table corresponding to the other to-be-registered image in the to-be-registered image pair to obtain a target stitched positioning table. That is, the image registration offset is a translation vector that can make the to-be-registered image pair achieve image alignment through displacement transform, and the formula for determining the relationship between the two to-be-registered images corresponding to the to-be-registered image pair can be as follows:

[0097] ;

[0098] wherein, and are two to-be-registered images corresponding to the to-be-registered image pair respectively, is the translation vector, i.e., the image registration offset. The embodiment first maps two to-be-registered images corresponding to the to-be-registered image pair to the frequency domain respectively by using the Fourier transform method, to obtain a first matrix and a second matrix corresponding to the two to-be-registered images. The determination formula of the first matrix and the second matrix can be specifically as follows:

[0099] ;

[0100] ;

[0101] wherein, is the first matrix corresponding to image , is the second matrix corresponding to image , is the Fourier transform (i.e., Discrete Fourier Transform), represents the frequency component of the image in the horizontal direction, represents the frequency component of the image in the vertical direction. It can be understood that the relationship expression formula between the first matrix and the second matrix can be specifically as follows:

[0102] ;

[0103] wherein, is the natural base, is the imaginary unit, representing the imaginary number in mathematics. That is, the amplitudes of the two to-be-registered images in the frequency domain are the same, and the phases are different. Then, the cross power spectrum of the first matrix and the second matrix is calculated. The determination formula of the cross power spectrum can be specifically as follows:

[0104] ;

[0105] wherein, is the complex conjugate of , is the complex conjugate of . Then, the inverse Fourier transform is performed on the cross power spectrum to obtain a target impulse response function. The determination formula of the target impulse response function can be specifically as follows:

[0106] ;

[0107] wherein, Inverse Discrete Fourier Transform (IDFT) is used to calculate the target impulse response function. The target impulse response function is determined. Finally, the maximum function value corresponding to the target impulse response function is determined, and the coordinate parameters corresponding to the maximum function value are taken as the image registration offset. The determination formula of the image registration offset can be specifically:

[0108]

[0109] That is, the optimal translation amount of the overlapping region between the two images to be registered. In this embodiment, the image registration offset is used to translate the data of the to-be-stitched positioning table corresponding to any one of the to-be-registered image pairs, so as to complete the registration of the to-be-stitched positioning table corresponding to the other to-be-registered image in the to-be-registered image pair, and obtain a target stitching positioning table. In this way, the phase correlation method is used to stitch the to-be-stitched positioning table, which can reduce the loss of positioning data accuracy, meet the requirement of accurate registration, and realize sub-pixel level accurate registration.

[0110] Step S25, rendering the target stitching positioning table to obtain a target stitching image, and verifying the target stitching image to determine whether there is a stitching trace in the target stitching image.

[0111] In this embodiment, the target stitching positioning table needs to be rendered, and then the target stitching image obtained is verified to determine the stitching effect of the target stitching positioning table, wherein the stitching effect can be determined by whether the stitching trace in the target stitching image after rendering is obvious.

[0112] Step S26, if there is no stitching trace, it is determined that the target stitching positioning table corresponding to the target stitching image passes the verification.

[0113] In this embodiment, if there is no stitching trace or the stitching trace is not obvious in the target stitching image, it is determined that the target stitching positioning table corresponding to the target stitching image passes the verification, that is, the stitching effect of the to-be-stitched positioning table can meet the requirements of the user.

[0114] The specific implementation process of steps S21, S22 and S23 can refer to the content in the foregoing embodiments, which will not be repeated here.

[0115] ​From the above, the embodiment of the present application determines the seam position data from a plurality of images to be spliced, and performs rendering reconstruction on a plurality of seam position data corresponding images to obtain a plurality of rendered images, enhances the plurality of rendered images based on linear filter transformation and directional filter transformation to obtain a plurality of target enhanced images, finally determines the image registration offset in the plurality of target enhanced images, analyzes the image pair to be registered, determines the image registration offset, splices the image registration offset to obtain the target splicing positioning table, and then performs rendering on the target splicing positioning table and verifies the splicing effect of the target splicing image. In this way, through the above process of the embodiment of the present application, when determining the image registration offset in the plurality of target enhanced images, the image pair to be registered in the plurality of target enhanced images is first determined, and the image registration offset corresponding to the image pair to be registered is determined according to the phase correlation method, so as to realize the splicing of the image to be spliced positioning table through the obtained image registration offset, reduce the loss of positioning data accuracy, meet the requirement of accurate registration, and realize the accurate registration at the sub-pixel level.

[0116] The following takes a certain image splicing process of a plurality of microscopic images collected based on a certain imaging experiment sample as an example to describe the technical solutions in the present application.

[0117] Suppose the imaging experiment sample is COS-7 cells (a kind of African green monkey kidney cells transformed by simian vacuolating virus), the microtubule structure of the sample is marked with AF647 dye (a far-red fluorescent dye), and the mitochondria of the sample are marked with CF568 dye (a red fluorescent dye). The laser power of the AF647 sample end is 704 mw, and the laser power of the CE568 sample end is 1274 mW. During the imaging process, automatic feedback adjustment is performed, the fluorescence molecular signal density is monitored in real time, and the laser is adjusted according to the signal density. The imaging system includes a microscope, an imaging objective lens, and a detector; the camera exposure time is set to 10 ms, and the imaging field of view (FOV) size is 100 μm x 100 μm; the imaging uses a 405 nm laser to activate the sample, and then uses a 640 nm laser to excite the fluorescence molecules; during imaging, AF647 is collected first, and then AF647 is bleached, and CF568 is collected. The maximum number of FOVs of each dye is 5000 frames.

[0118] The present embodiment can respectively use the algorithms representative in the field of image stitching, JRMPC (i.e. Joint Registration of Multiple Point Clouds, multiple point cloud joint registration algorithm), ICP (i.e. Iterative Closest Point, iterative closest point algorithm), NDT (i.e. Normal Distributions Transform, normal distribution transform algorithm) and CPD (i.e. Coherent Point Drift, coherent point drift algorithm) and the positioning table data splicing method based on sample structure provided by the present application to conduct pairwise registration experiments on microtubule data with regular structure and mitochondria data with irregular structure. Specifically, the microtubule data is divided into horizontal and vertical directions for pairwise registration experiments to compare the registration results respectively. As shown in Table 1, the pairwise registration results of five different algorithms, JRMPC, ICP, NDT, CPD and the positioning table data splicing method based on sample structure provided by the present application on microtubule data are compared respectively.

[0119] Table 1

[0120]

[0121] From the analysis of the above Table 1, it can be determined that in the registration in the vertical and horizontal directions, the MSE (i.e. Mean Squared Error, mean squared error) index and the SSIM (i.e. Structure Similarity Index Measure, structure similarity index) index of the positioning table data splicing method based on sample structure provided by the present application are better than those of other algorithms. Among them, the MSE value is about half or less of that of other algorithms, and the SSIM value is close to 0.8, indicating that the registration result of the positioning table data splicing method based on sample structure provided by the present application has strong structure similarity with the true situation. The combination of low MSE value and high SSIM value indicates that the structure information of the biological sample can better solve the problem of incomplete data at the joint.

[0122] In order to more intuitively observe the registration results of each algorithm, the present embodiment respectively displays the rendering graphs of the microtubule image joint in the horizontal and vertical directions. As shown in Figure 9 and Figure 10 , they are respectively a microtubule image horizontal joint registration result display graph and a vertical joint registration result display graph of each algorithm provided by the present application. In the horizontal and vertical directions, the joint positioning data of the same position of the microtubule is enlarged to show the specific splicing details. Among them, as shown in Figure 9The horizontal registration result display diagram shown in the figure shows that the purple box of the positioning table data splicing method based on the sample structure provided by the present application in the figure (f) has the highest similarity with the red box of the real situation in (a), and the misalignment is the least, while the yellow box of ICP in (b) has obvious misalignment. As shown in (c), (d) and (e), other algorithms all have slight misalignment, resulting in unclear appearance of the rendered microtubule image structure. Figure 10 It can be determined from the vertical alignment result display diagram that the purple box of the positioning table data splicing method based on the sample structure provided by the present application in Figure (f) has the highest similarity in details with the red box of the actual situation (a), while other algorithms have different degrees of misalignment compared with the actual situation at the same position due to unsatisfactory alignment accuracy. It can be seen that the positioning table data splicing method based on the sample structure provided by the present application has better splicing accuracy on microtubules with obvious structures than other related algorithms.

[0123] To determine the performance of the sample structure-based positioning table data stitching method provided in this application for super-resolution panoramic image stitching, thereby enabling the stitching of positioning data from multiple small fields of view into positioning data from a larger field of view, this embodiment conducted panoramic super-resolution positioning image stitching experiments on 3×3 microtubule data and a mitochondrial panoramic simulation dataset, and compared them with a state-of-the-art positioning data-based stitching framework in the field of super-resolution positioning imaging. As shown in Table 2 below, it can be determined that the sample structure-based positioning table data stitching method provided in this application performs excellently in panoramic super-resolution positioning imaging stitching for both microtubule data and mitochondrial data. Its results are closer to reality and the calculation speed is faster, effectively improving the performance and speed of panoramic image stitching.

[0124] Table 2

[0125]

[0126] In order to further observe the stitching details of the panoramic super-resolution positioning image, this embodiment analyzes the stitching details of different algorithms in detail. Figure 11a 、 Figure 11b as well as Figure 11c The data splicing results of a simulated panoramic microtubule provided in this application are respectively the data splicing results of the actual situation, the data splicing results obtained by processing the splicing framework based on positioning data, and the data splicing results obtained by processing the positioning table data splicing method based on the sample structure provided in this application. After comparison, it can be seen that there is no obvious misalignment in the overall structure of the two methods, but there are differences in details between the data splicing results of the splicing framework based on positioning data and the data splicing results of the actual situation, such as Figure 11b Yellow frame contrast Figure 11aThe yellow box has a blurred effect. The sample structure-based positioning table data splicing method provided in the present application has Figure 11c The yellow box shows that the structure is clear in detail and has no blurred phenomenon, and is consistent with the details in the real situation, and has better effect on panorama splicing than the splicing framework based on positioning data.

[0127] In order to verify the splicing performance of the sample structure-based positioning table data splicing method provided in the present application in actual panorama experiment data splicing, the present embodiment compares the results of the splicing framework based on positioning data and the sample structure-based positioning table data splicing method provided in the present application on the regular structure microtubule and the irregular structure mitochondria data set on the panorama splicing task. It can be understood that in order to intuitively feel the splicing effect of the panorama microtubule image, the present embodiment does not smooth and remove the spliced positioning table and image, so the image will have obvious seams. As shown in Figure 12a and Figure 12b The present application provides a real panorama microtubule splicing result and a detail display diagram. From the comparison of the visual effect and the details of the splicing, it can be seen that neither of the two methods has obvious misplacement. However, in the splicing details, the horizontal seam such as the red box arrow in Figure 12a and the red box arrow in Figure 12b , and the vertical seam such as the arrow of the other color box in Figure 12a and the arrow of the other color box in Figure 12b , the sample structure-based positioning table data splicing method provided in the present application can more accurately realize the splicing of the seam, and the microtubule line is clearer, the sample structure is more continuous, and the details can be better presented.

[0128] Correspondingly, as shown in Figure 13 the present embodiment also provides a sample structure-based positioning table data splicing device, which comprises:

[0129] An image rendering module 11 is configured to determine seam positioning data in a plurality of to-be-spliced positioning tables, and perform image rendering on the seam positioning data by using bilinear interpolation to obtain a plurality of rendered images.

[0130] A pixel integration module 12 is configured to integrate the pixel value of each pixel in the plurality of rendered images based on linear filter transformation to obtain a plurality of continuity-enhanced images, and a target integral direction and a target linear filter transformation amplitude corresponding to each pixel in the plurality of continuity-enhanced images.

[0131] The pixel enhancement module 13 is configured to determine target pixels satisfying a preset enhancement condition from the plurality of continuity enhancement images by using a preset linear functional, and perform structure enhancement on the target pixels based on the target integral direction corresponding to the target pixels and the target linear filter transform amplitude according to the directional filter transform, to obtain a plurality of target enhancement images.

[0132] The positioning table splicing module 14 is configured to determine image registration offsets corresponding to the plurality of target enhancement images based on a phase correlation method, and splice the plurality of to-be-spliced positioning tables according to the image registration offsets, to obtain a target spliced positioning table corresponding to the plurality of to-be-spliced positioning tables, and then render the target spliced positioning table and verify a splicing effect of a target spliced image obtained.

[0133] As can be seen from the above, the embodiment of the present application determines the seam positioning data from the plurality of to-be-spliced positioning tables, performs image rendering on the seam positioning data by using a bilinear interpolation, to obtain a plurality of rendered images, performs enhancement on the plurality of rendered images based on a linear filter transform and a directional filter transform, to obtain a plurality of target enhancement images, analyzes the plurality of target enhancement images to determine image registration offsets, and splices the plurality of to-be-spliced positioning tables based on the image registration offsets, to obtain a target spliced positioning table, and then renders the target spliced positioning table and verifies a splicing effect of a target spliced image obtained. In this way, the to-be-spliced images are spliced through the above process of the embodiment of the present application, the sample structure of the images is enhanced by using the linear filter transform and the directional filter transform, to reduce the problems of incomplete structure and specific artifacts in the to-be-spliced positioning data, and further reduce the splicing misplacement problem caused by errors in image registration.

[0134] In some specific embodiments, the image rendering module 11 can specifically include:

[0135] The positioning point determination unit is configured to determine a plurality of seam images corresponding to the seam positioning data, and determine a plurality of positioning points in the plurality of seam images;

[0136] The weight allocation unit is configured to determine fluorescent molecules corresponding to the plurality of positioning points based on a preset pixel unit, and allocate a pixel weight to each pixel in each of the fluorescent molecules;

[0137] The image rendering unit is configured to render the plurality of seam images based on the pixel weight and the fluorescent molecules, to obtain a plurality of rendered images.

[0138] In some specific embodiments, the pixel integral module 12 can specifically include:

[0139] a pixel integration unit, configured to integrate each pixel in the plurality of rendered images according to a preset integration range and a preset integration direction, to obtain a plurality of continuity-enhanced images corresponding to the plurality of rendered images;

[0140] an amplitude determination unit, configured to record a maximum integration corresponding to each pixel, and take an angle value corresponding to the maximum integration as a target integration direction corresponding to each pixel, and take an integration value corresponding to the maximum integration as a target linear filter transform amplitude corresponding to each pixel.

[0141] In some embodiments, the pixel enhancement module 13 can specifically include:

[0142] a polar coordinate determination unit, configured to determine a polar coordinate corresponding to each pixel based on the target integration direction corresponding to each pixel and the target linear filter transform amplitude corresponding to each pixel;

[0143] a dot product calculation unit, configured to calculate a dot product between a unit vector corresponding to each pixel in the plurality of continuity-enhanced images and the polar coordinate corresponding to each pixel, to obtain a plurality of pixel dot products;

[0144] a pixel determination unit, configured to determine a plurality of target pixel dot products in which a dot product value is higher than a preset dot product threshold in the plurality of pixel dot products, and take pixels corresponding to the plurality of target pixel dot products as target pixels satisfying a preset enhancement condition.

[0145] In some embodiments, the pixel enhancement module 13 can specifically include:

[0146] a dot product processing unit, configured to perform dot product processing on a unit vector corresponding to the target pixel and a polar coordinate corresponding to the target pixel based on the preset integration range, to obtain a plurality of target enhanced images corresponding to the plurality of continuity-enhanced images.

[0147] In some embodiments, the positioning table splicing module 14 can specifically include:

[0148] an offset determination sub-module, configured to determine a to-be-registered image pair in the plurality of target enhanced images, determine an image registration offset corresponding to the to-be-registered image pair respectively according to a phase correlation method, perform data splicing on the plurality of to-be-spliced positioning tables according to the image registration offset, to obtain a target spliced positioning table corresponding to the plurality of to-be-spliced positioning tables; the to-be-registered image pair is two images in the plurality of target enhanced images that need to be spliced;

[0149] The stitching verification unit is configured to render the target stitching positioning table to obtain a target stitching image, and verify the target stitching image to determine whether there is a stitching trace in the target stitching image.

[0150] The verification passing judgment unit is configured to, if there is no stitching trace, represent that the target stitching positioning table corresponding to the target stitching image passes the verification.

[0151] In some embodiments, the offset determination sub-module specifically can include:

[0152] The image mapping unit is configured to map the two to-be-registered images corresponding to the to-be-registered image pair into a frequency domain based on Fourier transform to obtain a first matrix and a second matrix corresponding to the two to-be-registered images.

[0153] The inverse transformation unit is configured to calculate a cross power spectrum of the first matrix and the second matrix, and perform inverse Fourier transform on the cross power spectrum to obtain a target impulse response function.

[0154] The offset determination unit is configured to determine a maximum function value corresponding to the target impulse response function, and take a coordinate parameter corresponding to the maximum function value as an image registration offset.

[0155] The stitching unit is configured to translate data of a to-be-stitched positioning table corresponding to any to-be-registered image in the to-be-registered image pair based on the image registration offset to complete registration of a to-be-stitched positioning table corresponding to another to-be-registered image in the to-be-registered image pair, and obtain a target stitching positioning table.

[0156] Further, the embodiment of the present application further discloses an electronic device, Figure 14 The electronic device 20 shown in the figure is a structure diagram according to an exemplary embodiment, and the content in the figure cannot be considered as any limitation on the use range of the present application. The electronic device 20 specifically can include at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25 and a communication bus 26. The memory 22 is configured to store a computer program, the computer program is loaded and executed by the processor 21 to realize the related steps in the positioning table data stitching method based on the sample structure disclosed in any of the preceding embodiments. In addition, the electronic device 20 in the embodiment specifically can be an electronic computer.

[0157] In this embodiment, the power supply 23 is configured to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 is configured to create a data transmission channel between the electronic device 20 and external devices, and the communication protocol followed by the communication interface 24 can be any communication protocol applicable to the technical solution of the present application, which will not be specifically limited herein; the input and output interface 25 is configured to obtain external input data or output data to the outside, and the specific interface type can be selected according to the specific application needs, which will not be specifically limited herein.

[0158] In addition, the memory 22 as a carrier for storing resources can be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage mode can be temporary storage or permanent storage.

[0159] The operating system 221 is configured to manage and control each hardware device and the computer program 222 on the electronic device 20, and can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program capable of completing the sample structure-based positioning table data splicing method executed by the electronic device 20 disclosed in any of the preceding embodiments, the computer program 222 can further include a computer program capable of completing other specific work.

[0160] Further, the present application also discloses a computer readable storage medium for storing a computer program; wherein the computer program is executed by a processor to implement the sample structure-based positioning table data splicing method disclosed above. For the specific steps of the method, please refer to the corresponding content disclosed in the preceding embodiments, which will not be described here.

[0161] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0162] The skilled person can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been described in a general manner in the above description. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0163] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in RAM, flash memory, ROM, electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. The

[0164] Finally, it should be noted that, in the description of the application, relational terms such as first and second, and the like can be used solely to distinguish one entity or action from another entity or action without necessarily implying any actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0165] The above provides a detailed description of the technical solutions of the present application. The principles and implementation modes of the present application are described by applying specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed; in summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A positioning table data splicing method based on sample structure, characterized in that: include: Determining a plurality of seam location data in a positioning table to be spliced, and performing image rendering on the seam location data using bilinear interpolation to obtain a plurality of rendered images; Integrating the pixel value of each pixel in the plurality of rendered images based on a linear filtering transformation to obtain a plurality of continuity-enhanced images and a target integration direction and a target linear filtering transformation amplitude corresponding to each pixel in the plurality of continuity-enhanced images; Determining target pixels that meet preset enhancement conditions from the plurality of continuity enhanced images using a preset linear functional, and performing structural enhancement on the target pixels based on the target integral direction corresponding to the target pixels and the target linear filter transform amplitude according to a directional filter transform to obtain a plurality of target enhanced images; Based on the phase correlation method, the image registration offsets corresponding to the several target enhanced images are determined, and data splicing is performed on the several positioning tables to be spliced ​​according to the image registration offsets to obtain a target splicing positioning table corresponding to the several positioning tables to be spliced, and then the target splicing positioning table is rendered, and the splicing effect of the obtained target spliced ​​image is verified.

2. The positioning table data splicing method based on sample structure according to claim 1, characterized in that: The image rendering of the positioning data at the seam by using bilinear interpolation to obtain a plurality of rendered images includes: Determining a plurality of seam images corresponding to the seam positioning data, and determining a plurality of positioning points in the plurality of seam images; Determining fluorescent molecules corresponding to the plurality of positioning points based on a preset pixel unit, and assigning a pixel weight to each pixel in each of the fluorescent molecules; The plurality of seam images are rendered based on the pixel weights and the fluorescent molecules to obtain a plurality of rendered images.

3. The positioning table data splicing method based on sample structure according to claim 1, characterized in that: The step of integrating the pixel value of each pixel in the plurality of rendered images based on the linear filtering transformation to obtain a plurality of continuity-enhanced images and a target integration direction and a target linear filtering transformation amplitude corresponding to each pixel in the plurality of continuity-enhanced images includes: Integrating each pixel in the plurality of rendered images according to a preset integration range and a preset integration direction to obtain a plurality of continuity-enhanced images corresponding to the plurality of rendered images; The maximum integral corresponding to each pixel is recorded, and the angle value corresponding to the maximum integral is used as the target integral direction corresponding to each pixel, and the integral value corresponding to the maximum integral is used as the target linear filtering transformation amplitude corresponding to each pixel.

4. The positioning table data splicing method based on sample structure according to claim 3 is characterized in that: The determining of target pixels satisfying a preset enhancement condition from the plurality of continuity enhanced images using a preset linear functional includes: Determining the polar coordinates corresponding to each pixel based on the target integration direction corresponding to each pixel and the target linear filter transformation amplitude corresponding to each pixel; Calculating a dot product between a unit vector corresponding to each pixel in the plurality of continuity-enhanced images and a polar coordinate corresponding to each pixel to obtain a plurality of pixel dot products; A plurality of target pixel dot products whose dot product values ​​are higher than a preset dot product threshold value are determined among the plurality of pixel dot products, and pixels corresponding to the plurality of target pixel dot products are used as target pixels that meet a preset enhancement condition.

5. The positioning table data splicing method based on sample structure according to claim 4 is characterized in that: The step of enhancing the target pixel based on the target integral direction and the target linear filter transform amplitude corresponding to the target pixel according to the directional filter transform to obtain a plurality of target enhanced images includes: Based on the preset integration range, dot product processing is performed on the unit vector corresponding to the target pixel and the polar coordinates corresponding to the target pixel to obtain a plurality of target enhanced images corresponding to the plurality of continuity enhanced images.

6. The positioning table data splicing method based on sample structure according to any one of claims 1 to 5, characterized in that: The method of determining the image registration offsets corresponding to the plurality of target enhanced images based on the phase correlation method, performing data splicing on the plurality of positioning tables to be spliced ​​according to the image registration offsets to obtain a target splicing positioning table corresponding to the plurality of positioning tables to be spliced, then rendering the target splicing positioning table, and verifying the splicing effect of the obtained target spliced ​​image, includes: Determining an image pair to be registered from the plurality of target enhanced images, and determining image registration offsets corresponding to the image pairs to be registered according to a phase correlation method, so as to perform data splicing on the plurality of positioning tables to be stitched according to the image registration offsets, so as to obtain a target stitching positioning table corresponding to the plurality of positioning tables to be stitched; the image pair to be registered is two images to be stitched from the plurality of target enhanced images; Rendering the target stitching positioning table to obtain a target stitching image, and verifying the target stitching image to determine whether there are stitching traces in the target stitching image; If there is no stitching trace, it indicates that the target stitching positioning table corresponding to the target stitching image has passed the verification.

7. The positioning table data splicing method based on sample structure according to claim 6, characterized in that: The step of determining the image registration offsets corresponding to the pair of images to be registered according to the phase correlation method, and performing data splicing on the plurality of positioning tables to be spliced ​​according to the image registration offsets to obtain a target splicing positioning table corresponding to the plurality of positioning tables to be spliced, comprises: Mapping the two images to be registered corresponding to the pair of images to be registered to the frequency domain based on Fourier transform to obtain a first matrix and a second matrix corresponding to the two images to be registered; Calculating a cross power spectrum between the first matrix and the second matrix, and performing an inverse Fourier transform on the cross power spectrum to obtain a target impulse response function; Determining a maximum function value corresponding to the target impulse response function, and using coordinate parameters corresponding to the maximum function value as image registration offsets; The data of the to-be-stitched positioning table corresponding to any one of the to-be-registered images in the to-be-registered image pair is translated based on the image registration offset to complete the registration with the to-be-stitched positioning table corresponding to the other to-be-registered image in the to-be-registered image pair, thereby obtaining a target stitching positioning table.

8. A positioning table data splicing device based on a sample structure, characterized in that: include: An image rendering module is used to determine the seam positioning data in a plurality of to-be-joined positioning tables, and perform image rendering on the seam positioning data using bilinear interpolation to obtain a plurality of rendered images; a pixel integration module, configured to integrate the pixel value of each pixel in the plurality of rendered images based on a linear filtering transformation to obtain a plurality of continuity-enhanced images and a target integration direction and a target linear filtering transformation amplitude corresponding to each pixel in the plurality of continuity-enhanced images; a pixel enhancement module, configured to determine target pixels that meet preset enhancement conditions from the plurality of continuity enhanced images using a preset linear functional, and perform structural enhancement on the target pixels based on the target integral direction corresponding to the target pixels and the target linear filter transform amplitude according to a directional filter transform, so as to obtain a plurality of target enhanced images; A positioning table stitching module is used to determine the image registration offsets corresponding to the several target enhanced images based on the phase correlation method, and to perform data stitching on the several positioning tables to be stitched according to the image registration offsets to obtain a target stitching positioning table corresponding to the several positioning tables to be stitched, and then to render the target stitching positioning table and to verify the stitching effect of the obtained target stitching image.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor is configured to execute the computer program to implement the positioning table data splicing method based on the sample structure according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that Used to store a computer program; wherein, when the computer program is executed by a processor, the positioning table data splicing method based on the sample structure according to any one of claims 1 to 7 is implemented.

Images