Real-time image splicing method and system without illumination influence

Abstract

The invention provides a real-time image splicing method and system without illumination influence. The real-time image splicing method comprises the following steps: acquiring a first prefabricated template and a second prefabricated template; determining a first correction parameter according to the first prefabricated template; determining a second correction parameter according to the second prefabricated template; scanning the pathological section by using a scanner, and in the scanning process, directly splicing each to-be-spliced pathological section image obtained by scanning through the scanner after the pathological section image is corrected through the first correction parameter and the second correction parameter, so that the pathological section image can be spliced while being scanned in the scanning process to obtain a complete pathological section image; the method has the advantages that the orthographic projection image of the to-be-spliced pathological section image is restored and the position of the to-be-spliced pathological section image is corrected by using the first correction parameter and the second correction parameter, so that the image integrity can be kept, the image containing blank samples, sparse samples or few features can be spliced, and the image splicing accuracy is improved. Scanning and splicing can be carried out at the same time.

Description

Technical Field

[0001] The present invention relates to the technical field of image stitching, and in particular to a real-time image stitching method and system without illumination influence. Background Art

[0002] In digital pathology, in order to obtain a complete pathology slice image, it is usually necessary to stitch multiple high-resolution images. These images often cover different areas of the entire slice, so they must be combined into a complete image through stitching technology. However, since the resolution of pathology images is very high, usually reaching billions of pixels, the amount of calculation in the stitching process is very large and the processing speed is very slow, which seriously affects the efficiency of real-time diagnosis and analysis.

[0003] Defects and shortcomings of existing technologies: 1. Highly dependent on image matching and unable to cope with poor matching: First, the matching algorithm is sensitive to deformation. Since pathological sections may deform during production and scanning, existing spatial domain jigsaw puzzle methods are less robust to these deformations and prone to calculation errors; Second, for samples lacking feature points, it is impossible to calculate a relatively accurate displacement distance and even produce a large offset error. 2. Unable to scan and stitch at the same time: The digital imaging process of the entire slide requires the calculation of the global position of each image. This process requires waiting for all images to be scanned and the displacement deviation to be calculated before global optimization can be performed. Then the machine scanning motion process cannot be imaged, resulting in too slow imaging time. Summary of the invention

[0004] In view of the above-mentioned shortcomings of the prior art, an object of the present invention is to provide a real-time image stitching method and system without lighting influence, so as to solve the problem that the prior art is highly dependent on matching between images and thus cannot cope with poor matching and cannot stitch while scanning.

[0005] To achieve the above objectives and other related objectives, the present invention provides the following technical solutions: A real-time image stitching method without illumination influence comprises the following steps: obtaining a first prefabricated template and a second prefabricated template, wherein the first prefabricated template and the second prefabricated template are both templates shaped like a chessboard with evenly distributed black and white chessboards, and a uniform, light-transmitting single-color coating is provided on the white grid of the first prefabricated template, and the thickness of the single-color coating is 5 , because the conventional white checkerboard grid is a transparent grid, due to the light transmittance of different glass materials, it cannot be guaranteed that the glass slide has different brightness distribution when it is uneven, so the setting of the monochrome coating makes the light transmittance at different distances have different attenuation conditions; determine the first correction parameter for calculating the orthographic projection image according to the first prefabricated template; determine the second correction parameter for correcting the image position according to the second prefabricated template; The pathological slices are scanned using a scanner, and the pathological slice images to be spliced ​​are obtained according to the scanning results; during the scanning process, the plane projection of the pathological slice images is corrected by the first correction parameter, and the poorly matched pathological slice images are corrected by the second correction parameter, and each of the pathological slice images can be directly spliced ​​after being corrected by the first correction parameter and the second correction parameter, so that the complete pathological slice image can be obtained by scanning and splicing at the same time during the scanning process, wherein the complete pathological slice image includes a plurality of spliced ​​pathological slice images, and the content of each of the pathological slice images includes different areas of the entire pathological slice.

[0006] A real-time image stitching system without illumination influence comprises: an acquisition module, used for acquiring a first prefabricated template and a second prefabricated template, wherein the first prefabricated template and the second prefabricated template are both templates shaped like a chessboard with evenly distributed black and white chessboards, and a uniform, thin and light-transmitting monochrome coating is provided on the white grids of the first prefabricated template, because the conventional chessboard white grids are transparent grids, and due to the light transmittance of different glass materials, it cannot be guaranteed that there are different brightness distributions when the glass slide is uneven, so the setting of the monochrome coating makes the light transmittance at different distances have different attenuation conditions; a determination module, used for determining a first correction parameter for calculating an orthographic projection image according to the first prefabricated template; and determining a second correction parameter for correcting an image position according to the second prefabricated template; A scanning and stitching module is used to scan pathological slices using a scanner and obtain pathological slice images to be stitched according to the scanning results; during the scanning process, the plane projection of the pathological slice image is corrected by the first correction parameter, and the poorly matched pathological slice images are corrected by the second correction parameter. Each of the pathological slice images can be directly stitched after being corrected by the first correction parameter and the second correction parameter, so that the complete pathological slice image can be obtained by scanning and stitching during the scanning process, wherein the complete pathological slice image includes a plurality of stitched pathological slice images, and the content of each pathological slice image contains different areas of the entire pathological slice.

[0007] An electronic device comprises: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the above-mentioned real-time image stitching method without illumination influence.

[0008] In one embodiment of the present invention, the method of determining the first correction parameter for calculating the orthographic projection image based on the first prefabricated template includes: placing the first prefabricated template under the lens of a scanner for shooting, and obtaining a plane projection image of the first prefabricated template after rotation according to the shooting result; obtaining an equivalent image on the rotation plane corresponding to the plane projection image, and determining the rotation angle of the rotation plane relative to the XOY plane according to the plane projection image and the equivalent image on the rotation plane, wherein two solutions are generated when determining the rotation angle of the rotation plane relative to the XOY plane, and whether the rotation plane is warped or sunken is judged by the different attenuation of the light transmittance at different distances, and the correct direction is obtained according to the judgment result; determining a rotation matrix according to the rotation angle, and inversely transforming the rotation matrix, and obtaining the first correction parameter for calculating the orthographic projection image according to the inverse transformation result.

[0009] In one embodiment of the present invention, determining the rotation angle of the rotation plane relative to the XOY plane according to the planar projection image and the equivalent image on the rotation plane includes: determining the rotation angle of the rotation plane relative to the XOY plane according to the following formula: , in, is the rotation angle of the rotation plane relative to the XOY plane; the general equation of the rotation plane is , , and is the normal vector of the rotation plane ,and is a constant term; is the normal vector to the XOY plane.

[0010] In one embodiment of the present invention, a rotation matrix is ​​determined according to the rotation angle, and an inverse transformation is performed on the rotation matrix, and a first correction parameter for calculating the orthographic projection image is obtained according to the inverse transformation result, including: determining the rotation matrix and obtaining the first correction parameter for calculating the orthographic projection image according to the following formula: ; ;in, is the rotation matrix; is the identity matrix; is the rotation angle of the rotation plane relative to the XOY plane; is the antisymmetric matrix form of the rotation, ; It is the inverse matrix of the rotation matrix and also the transposed matrix, and is also the first correction parameter used to calculate the forward projection image.

[0011] In one embodiment of the present invention, determining the second correction parameter for correcting the image position based on the second prefabricated template includes: scanning the second prefabricated template with a scanner, and obtaining blocks with an ideal grid distribution based on the scanning results; obtaining the coordinates of the upper left corner point of each ideal block, and recording the coordinate set as a first list; recording the coordinate set of the upper left corner point of the block obtained by the SURF alignment algorithm as a second list; and determining the second correction parameter for correcting the image position based on the first list and the second list.

[0012] In one embodiment of the present invention, determining the second correction parameter for correcting the image position according to the first list and the second list includes: performing linear regression with the first list as an independent variable and the second list as a dependent variable, obtaining a coefficient matrix according to the linear regression result, and obtaining the second correction parameter for correcting the image position according to the coefficient matrix; wherein the coefficient matrix is , Indicates input Variable pair output The impact of Indicates input Variable pair output The impact of Indicates input Variable pair output The impact of Indicates input Variable pair output

[0013] The influence of; so the linear equation can be expressed as: , linearly transformed into the second correction parameter for correcting the image position.

[0014] In one embodiment of the present invention, during the scanning process, the plane projection of the pathological slice image is corrected by the first correction parameter, and the poorly matched pathological slice image is corrected by the second correction parameter. Each of the pathological slice images can be directly spliced ​​after being corrected by the first correction parameter and the second correction parameter, so that the complete pathological slice image can be obtained by scanning and stitching during the scanning process, including: during the scanning process, the positive projection image of the pathological slice image is calculated by the first correction parameter, and then the displacement error caused by mechanical movement is calculated by the second correction parameter, and the poorly matched pathological slice image is corrected according to the displacement error. Each of the pathological slice images can be directly spliced ​​after being corrected by the first correction parameter and the second correction parameter, so that the complete pathological slice image can be obtained by scanning and stitching during the scanning process.

[0015] As described above, the real-time image stitching method and system without illumination influence of the present invention have the following beneficial effects: The present invention uses a first prefabricated template to correct the direction of the platform rising or sinking to provide a first correction parameter for the orthographic projection image, and restores the orthographic projection image of the pathological slice image to be spliced ​​obtained by scanning the scanner device through the first correction parameter; the present invention also obtains a second correction parameter through the simultaneous application of a linear fitting and matching algorithm, and corrects the displacement error caused by mechanical movement through the second correction parameter, thereby ensuring the accuracy of the displacement correction, so that the pathological slice images to be spliced ​​can be seamlessly spliced, and a more accurate position correction can be made for the pathological slice images to be spliced ​​with poor matching or excessive matching errors; therefore, the present invention can make the calculation speed fast under the action of the first correction parameter and the second correction parameter, and can enable the scanner device to scan and splice at the same time, thereby solving the problem of being highly dependent on matching between images and being unable to cope with poor matching and being unable to scan and splice at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of a real-time image stitching method without illumination influence in a first embodiment of the present invention; Figure 2 is a flow chart of a real-time image stitching system without illumination influence in a second embodiment of the present invention; Figure 3 is a schematic diagram of an electronic device in a third embodiment of the present invention; Figure 4 is a schematic diagram of the attenuation of brightness as the distance from the light source increases in the present invention; Figure 5 It is a schematic diagram of the present invention in which the captured image is not an orthographic image due to uneven assembly; Figure 6 It is a schematic diagram of the spatial relationship between the image and the projection in the present invention; Figure 7 It is a schematic diagram of two situations of plane projection restoration of the S rotation plane in the present invention. DETAILED DESCRIPTION

[0017] The following describes the embodiments of the present invention through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0018] The first embodiment of the present invention relates to a real-time image stitching method without illumination influence, the process is as follows: Figure 1 As shown, the details are as follows: S101, obtaining a first prefabricated template and a second prefabricated template.

[0019] Specifically, for the first prefabricated template: design a scanning template for pre-scanning, and select a template shaped like a chessboard for the scanning template, because the chessboard has a high contrast under the microscope and can well detect the projection shape; for the chessboard size, the width and height need to be designed to be 360 , the tolerance is less than or equal to 0.2 However, the conventional white checkerboard grid is transparent. Due to the light transmittance of different glass materials, it is impossible to ensure that the glass has different brightness distribution when the glass is not flat. Therefore, a uniform, light-transmitting single-color coating is applied on the transparent checkerboard grid. The thickness of the single-color coating is 5 , ensuring that when the camera is not vertical or the platform is not horizontal, different distances (the distance between the plane and the light source) will cause different attenuation of the light intensity; e.g. Figure 4 As shown, there is less light transmission away from the light source, and its light transmission intensity is low; the light transmission intensity is high near the light source, so the light transmission intensity can be used to determine which direction the glass slide is tilted or sunken.

[0020] For the second prefabricated template: Customize a scanning template that presents a black and white checkerboard distribution, where the white checkerboard is transparent and the checkerboard size is set to 30 ×30 , tolerance ≤0.2 , and the tolerance is ≤0.2 The reason is that the error of the captured image can be less than 1 pixel.

[0021] S102: Determine a first correction parameter for calculating an orthographic projection image according to the first prefabricated template.

[0022] Specifically, the projection calculation platform rotation angle is used to correct the positive projection image: first, the first prefabricated template is placed under the mirror and photographed to ensure that there is only one complete monochrome quadrilateral (non-black) under the mirror, such as Figure 5 As shown, during the scanning process, the platform may be uneven due to platform assembly, so that the image captured by the camera is not an orthographic projection image, but a rotated plane projection, which leads to errors (it should be noted that in this embodiment, when a scanner is used to scan a pathological section, a slide containing the pathological section is placed on the platform, and then scanned using a scanner device); Figure 6 As shown, assuming that pattern A is the positive projection of the image, is its equivalent image on the rotation plane, The rotation plane is denoted as S, and the irregular quadrilateral B is The projection on the XOY plane needs to be done through The rotation angle of plane S relative to the XOY plane is solved with B, and then the orthographic projection image A can be obtained by inverse transformation of the rotation angle; Rectangle A coincides with the XOY plane, and its four vertex coordinates can be assumed to be: , , , ,in, , because it coincides with the XOY plane; the rotated rectangle (On plane S): After spatial rotation, the rectangle Each vertex of the rectangle is transformed from the XOY plane to another spatial plane S. The vertex coordinates are: , , , ; Irregular quadrilateral B projected onto the XOY plane: In the rotated state, the rectangle Projecting onto the XOY plane will form an irregular quadrilateral B. When projecting onto the XOY plane, the z coordinates of all vertices are ignored, and the projected vertex coordinates are: , , , ; Size constraints of rectangle A: The side lengths of rectangle A are known. Assuming that the side lengths of rectangle A are w (width) and h (height), then no matter how it is rotated, the lengths of the sides after rotation (that is, the distance between vertices) should remain unchanged in 3D space. To explain this in detail: the distance between adjacent vertices of the rectangle, such as the distance between A1 and A2, should remain w before and after rotation, that is: ①, Similarly, for the other direction of the rectangle, the distance between A1 and A4 should be h: ②, after rotation, the vertices of rectangle A , , , On the new plane S, the distance between their vertices should still satisfy the geometric constraints. Since the x and y coordinates of each vertex on the projected quadrilateral B are known, and the edge length constraint can provide additional geometric conditions, it helps to determine the normal vector of plane S and its information in the Z direction. ③, among which , are known projection coordinates, is the unknown quantity that needs to be solved; for the two opposite sides of the rectangle, the following equations can be written respectively: Width constraint equation: ④, Height constraint equation: ⑤, combined with ④ and ⑤, let: , ; available , ; Solve to get: ⑥; Because ⑥ provides Relative to and , select a value as the benchmark. Usually, you can set the vertex The z coordinate of = 0), thereby simplifying the solution (setting the z coordinate of any point does not affect the spatial relationship); at this time, two sets of solutions will be generated, and the corresponding situations of the two sets of solutions are as follows Figure 7 As shown, the light transmittance in the first prefabricated template is used to determine whether the plane S is warped or sunken. After the determination, one of the two sets of solutions will be discarded. Then, there is only one set of solutions, and the spatial rotation angle is unique and determined. To calculate the angle between plane S and the XOY plane (the angle between plane S and the X, Y, and Z axes), we need to calculate the normal vector of plane S. The general equation of plane S can be written as: ⑦, among which, , and is the normal vector of the plane ,and is a constant term; in order to calculate the angle between plane S and the XOY plane, you need to find the normal vector first , and then determine from a known point on the plane ; Through the above calculation, there are three vertices of rectangle A on plane S, which are , , , , , , then we can construct the vector , the normal vector ; By normal vector , we can know the rotation angle between plane S and X, Y, and Z axes, and thus construct the inverse transformation , then the orthographic image , to explain this in detail: To construct the inverse transformation matrix , its meaning and construction steps need to be clarified, inverse transformation is used to transform the normal vector of plane S Align to a reference plane (such as the XOY plane); the rotation matrix R transforms the normal vector of the XOY plane Normal vector rotated to the target plane S direction, so the inverse transformation That is to Rotate back The direction of; normal vector normalization: , we can get the rotation axis, , and unitize, ; Use the Rodrigues formula to construct the rotation matrix R. The Rodrigues formula defines the rotation around the axis v The rotation matrix R of , ,so ,in, is the identity matrix, used to preserve the non-rotated parts, is the antisymmetric matrix form of the rotation: ; The inverse of a rotation matrix is ​​its transposed matrix, so , substituting the numerical value into it, we can get: , orthographic image , we get the coordinates of the orthographic projection image, as the first correction parameter.

[0023] S103: Determine a second correction parameter for correcting the image position according to the second prefabricated template.

[0024] Specifically, the deviation caused by the motor movement is obtained by linear fitting and used as a correction parameter to correct the image position: first, the second prefabricated template is scanned, and a set of very regular grid-distributed blocks is obtained according to the second prefabricated template. Specifically, the coordinates of the upper left corner point of each ideal block are obtained, and the coordinate set is recorded as a list pos( , ), let the coordinate set of the upper left corner of the tile calculated by the SURF registration algorithm be recorded as the list image_coordinates( , ), take the coordinates in the pos list as the independent variable, and the coordinates in the image_coordinates list as the dependent variable, and perform linear regression; that is, use the original coordinates pos as the input (X), and use the calculated coordinates image_coordinates as the output (Y). According to the mapping relationship, we can get: Y=X Coefficient matrix + intercept, the form of this matrix is ​​as follows: The coefficient matrix is , Indicates input Variable pair output The impact of Indicates input Variable pair output The impact of Indicates input Variable pair output The impact of Indicates input Variable pair output The influence of; so the linear equation can be expressed as: , by finding the coefficient matrix, to constrain the image error when the match is poor, and use this linear transformation as the second correction parameter.

[0025] S104, scanning the pathological slices using a scanner, and obtaining the pathological slice images to be spliced ​​according to the scanning results.

[0026] S105, during the scanning process, the plane projection of the pathological slice image is corrected by the first correction parameter, and the poorly matched pathological slice image is corrected by the second correction parameter. Each pathological slice image can be directly spliced ​​after being corrected by the first correction parameter and the second correction parameter, so that the pathological slice image can be obtained by scanning and splicing at the same time during the scanning process.

[0027] Specifically, a complete pathological slice image includes several spliced ​​pathological slice images, the content of each pathological slice image includes different areas of the entire pathological slice, and the complete pathological slice image can be considered as a grid map composed of several images, each small grid of the grid map is a pathological slice image, and the more important technical point in the present invention is to calculate the first correction parameter and the second correction parameter, and to correct each pathological slice image by the first correction parameter and the second correction parameter; and in the scanning process, the orthographic projection image of the pathological slice image is calculated by the first correction parameter, and then the displacement error caused by mechanical movement is calculated by the second correction parameter, and the poorly matched pathological slice images are corrected according to the displacement error, and each pathological slice image can be directly spliced ​​after being corrected by the first correction parameter and the second correction parameter, so that the complete pathological slice image can be obtained by scanning and splicing during the scanning process.

[0028] Further, the above process can also be specifically as follows: Step 1: Scan the customized pre-scan template (that is, the first prefabricated template); Step 2: Determine the platform warping direction according to the light transmittance. When calculating the rotation angle of the rotating plane S relative to the XOY plane, two solutions will be generated. One direction will be discarded by determining the direction, and the correct direction will be retained; Step 3: Restore the orthographic projection image. The purpose is to obtain the orthographic projection coordinates through the current projection. This is explained in detail by performing a monochromatic rectangular block-pattern of shooting, and stipulates The coordinates of the orthographic projection image A are , , , , determine the image The projection of the irregular quadrilateral B is: , , , ; Then here, the direction obtained by the mathematical operation process of step 102 discards a mathematical solution, and finally the inverse transformation matrix is ​​obtained , then you can pass The current projection B is calculated to get Orthographic projection coordinates , thereby obtaining the first correction parameter; Step 4: Calculate the displacement error caused by mechanical movement: Here we need to scan the scale (that is, the second prefabricated template). The original grid position of the scanned block is calculated under ideal conditions. The actual scan position is obtained through the SURF matching algorithm, and then the linear conversion relationship of the position is calculated, that is: actual position = transformation matrix Ideal position + deviation, that is , taking the transformation relationship as the second correction parameter; the fifth step: using a scanner to scan the pathological slice to obtain the pathological slice image to be spliced; the sixth step: calculating the orthographic projection image of the pathological slice image through the first correction parameter; the seventh step: correcting the poor match through the second correction parameter, so after using the scanner to scan a pathological slice image, it can be corrected through the first correction parameter and the second correction parameter, and after correction, it can be directly spliced, so it can be scanned and spliced ​​during the scanning process.

[0029] The second embodiment of the present invention relates to a real-time image stitching system without illumination influence, see Figure 2 ,include: The acquisition module is used to acquire a first prefabricated template and a second prefabricated template, wherein the first prefabricated template and the second prefabricated template are both templates shaped like a chessboard with evenly distributed black and white chessboards, and a uniform, light-transmitting single-color coating is provided on the white grid of the first prefabricated template, and the thickness of the single-color coating is 5 ,Because the conventional chessboard white grid is transparent grid, due to the light transmittance of different glass materials, it cannot be guaranteed that the brightness distribution will be different when the glass slide is not flat, so the setting of the single-color coating makes the light transmittance intensity at different distances have different attenuation; A determination module, configured to determine a first correction parameter for calculating an orthographic projection image according to a first prefabricated template; and to determine a second correction parameter for correcting an image position according to a second prefabricated template; The scanning and stitching module is used to scan the pathological slices with a scanner and obtain the pathological slice images to be stitched according to the scanning results; during the scanning process, the plane projection of the pathological slice images is corrected by the first correction parameter, and the poorly matched pathological slice images are corrected by the second correction parameter. Each pathological slice image can be directly stitched after being corrected by the first correction parameter and the second correction parameter, so that the complete pathological slice image can be obtained by scanning and stitching during the scanning process, wherein the complete pathological slice image includes several stitched pathological slice images, and the content of each pathological slice image includes different areas of the entire pathological slice.

[0030] It is not difficult to find that this embodiment is a system embodiment corresponding to the first embodiment, and this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment are still valid in this embodiment, and in order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the first embodiment.

[0031] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of the present invention, this embodiment does not introduce units that are not closely related to solving the technical problem proposed by the present invention, but this does not mean that there are no other units in this embodiment.

[0032] The third embodiment of the present invention relates to an electronic device, see Figure 3 ,include: At least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned real-time image stitching method without lighting influence, wherein the electronic device can be arranged inside the scanner, or can be an external terminal device such as a computer.

[0033] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium via an antenna, and further, the antenna also receives data and transmits the data to the processor.

[0034] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0035] A fourth embodiment of the present invention relates to a computer-readable storage medium storing a computer program, and the computer program implements the above method embodiment when executed by a processor.

[0036] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0037] In summary, the present invention uses the first correction parameter and the second correction parameter to restore the orthographic projection image of the pathological section image to be stitched and correct the position of the pathological section image to be stitched, which can not only maintain the integrity of the image and stitch images containing blank samples, sparse samples or very few features, but also perform stitching while scanning, and finally stitch all the pathological section images to be stitched into a complete digital image, so as to achieve the purpose of real-time stitching of large-scale images and generation of digital pathological sections, and the image stitching result is more accurate by correcting the stitching position of the image.

[0038] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. All equivalent modifications or changes made by those skilled in the art without departing from the spirit and technical ideas disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A real-time image stitching method without illumination influence, characterized in that: The following steps are involved: A first prefabricated template and a second prefabricated template are obtained, wherein the first prefabricated template and the second prefabricated template are both chessboard-shaped templates with evenly distributed black and white chessboard grids, and a uniform, light-transmitting single-color coating is provided on the white grids of the first prefabricated template, and the thickness of the single-color coating is 5 ,Because the conventional chessboard white grid is transparent grid, due to the light transmittance of different glass materials, it cannot be guaranteed that the brightness distribution will be different when the glass slide is not flat, so the setting of the single-color coating makes the light transmittance intensity at different distances have different attenuation; Determining a first correction parameter for calculating the orthographic projection image according to the first prefabricated template; Determining a second correction parameter for correcting the image position according to the second prefabricated template; Scanning the pathological sections with a scanner, and obtaining the pathological section images to be spliced ​​according to the scanning results; During the scanning process, the plane projection of the pathological slice image is corrected by the first correction parameter, and the poorly matched pathological slice image is corrected by the second correction parameter. Each of the pathological slice images can be directly spliced ​​after being corrected by the first correction parameter and the second correction parameter, so that the scanning and splicing can be performed simultaneously to obtain a complete pathological slice image, wherein the complete pathological slice image includes several spliced ​​pathological slice images, and the content of each pathological slice image contains different areas of the entire pathological slice.

2. The real-time image stitching method without illumination influence according to claim 1, characterized in that: The step of determining a first correction parameter for calculating the orthographic projection image according to the first prefabricated template comprises: Placing the first prefabricated template under a scanner lens for photographing, and obtaining a plane projection image of the first prefabricated template after rotation according to the photographing result; Obtain an equivalent image on the rotation plane corresponding to the plane projection image, and determine the rotation angle of the rotation plane relative to the XOY plane according to the plane projection image and the equivalent image on the rotation plane, wherein two solutions are generated when determining the rotation angle of the rotation plane relative to the XOY plane, and judge whether the rotation plane is upturned or downturned by different attenuation conditions of the light transmission intensity at different distances, and obtain the correct direction according to the judgment result; A rotation matrix is ​​determined according to the rotation angle, and an inverse transformation is performed on the rotation matrix. A first correction parameter for calculating the orthographic projection image is obtained according to the inverse transformation result.

3. The real-time image stitching method without illumination influence according to claim 2, characterized in that: Determining the rotation angle of the rotation plane relative to the XOY plane based on the plane projection image and the equivalent image on the rotation plane includes: Determine the rotation angle of the rotation plane relative to the XOY plane according to the following formula: ， ； in, is the rotation angle of the rotation plane relative to the XOY plane; the general equation of the rotation plane is , , and is the normal vector of the rotation plane ,and is a constant term; is the normal vector to the XOY plane.

4. The real-time image stitching method without illumination influence according to claim 2, characterized in that: Determining a rotation matrix according to the rotation angle, and performing an inverse transformation on the rotation matrix, and obtaining a first correction parameter for calculating the orthographic projection image according to the inverse transformation result, including: The rotation matrix and the first correction parameters for calculating the orthographic projection image are determined according to the following formula: ； ； in, is the rotation matrix; is the identity matrix; is the rotation angle of the rotation plane relative to the XOY plane; is the antisymmetric matrix form of the rotation, ; It is the inverse matrix of the rotation matrix and also the transposed matrix, and is also the first correction parameter used to calculate the forward projection image.

5. The real-time image stitching method without illumination influence according to claim 1, characterized in that: The step of determining a second correction parameter for correcting the image position according to the second prefabricated template comprises: Scanning the second prefabricated template using a scanner, and obtaining blocks with ideal grid distribution according to the scanning result; Get the coordinates of the upper left corner of each ideal tile, and record the coordinate set as the first list; The coordinate set of the upper left corner point of the tile obtained by the SURF registration algorithm is recorded as the second list; A second correction parameter for correcting the image position is determined based on the first list and the second list.

6. The real-time image stitching method without illumination influence according to claim 5, characterized in that: The determining of the second correction parameter for correcting the image position according to the first list and the second list comprises: Performing linear regression with the first list as an independent variable and the second list as a dependent variable, obtaining a coefficient matrix according to the linear regression result, and obtaining a second correction parameter for correcting the image position according to the coefficient matrix; Among them, the coefficient matrix is , Indicates input Variable pair output The impact of Indicates input Variable pair output The impact of Indicates input Variable pair output The impact of Indicates input Variable pair output The influence of; so the linear equation can be expressed as: , linearly transformed into the second correction parameter for correcting the image position.

7. The real-time image stitching method without illumination influence according to claim 1, characterized in that: During the scanning process, the plane projection of the pathological slice image is corrected by the first correction parameter, and the poorly matched pathological slice image is corrected by the second correction parameter. Each of the pathological slice images can be directly spliced ​​after being corrected by the first correction parameter and the second correction parameter, so that the complete pathological slice image can be obtained by scanning and splicing during the scanning process, including: During the scanning process, the first correction parameter is used to calculate the orthographic projection image of the pathological slice image, and then the displacement error caused by mechanical movement is calculated by the second correction parameter. The poorly matched pathological slice image is corrected according to the displacement error. Each pathological slice image can be directly spliced ​​after being corrected by the first correction parameter and the second correction parameter, so that the pathological slice image can be scanned and spliced ​​at the same time during the scanning process to obtain a complete pathological slice image.

8. A real-time image stitching system without lighting influence, characterized in that: include: The acquisition module is used to acquire a first prefabricated template and a second prefabricated template, wherein the first prefabricated template and the second prefabricated template are both templates shaped like a chessboard with evenly distributed black and white chessboards, and a uniform, light-transmitting single-color coating is provided on the white grid of the first prefabricated template, and the thickness of the single-color coating is 5 ,Because the conventional chessboard white grid is transparent grid, due to the light transmittance of different glass materials, it cannot be guaranteed that the brightness distribution will be different when the glass slide is not flat, so the setting of the single-color coating makes the light transmittance intensity at different distances have different attenuation; A determination module, configured to determine a first correction parameter for calculating an orthographic projection image according to the first prefabricated template; and to determine a second correction parameter for correcting an image position according to the second prefabricated template; A scanning and stitching module is used to scan pathological slices using a scanner and obtain pathological slice images to be stitched according to the scanning results; during the scanning process, the plane projection of the pathological slice image is corrected by the first correction parameter, and the poorly matched pathological slice images are corrected by the second correction parameter. Each of the pathological slice images can be directly stitched after being corrected by the first correction parameter and the second correction parameter, so that the complete pathological slice image can be obtained by scanning and stitching during the scanning process, wherein the complete pathological slice image includes a plurality of stitched pathological slice images, and the content of each pathological slice image contains different areas of the entire pathological slice.

9. An electronic device, characterized in that: include: at least one processor; as well as, A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the real-time image stitching method without illumination influence as described in any one of claims 1 to 7.

Images