Image generation method, microscope device, and storage medium

Abstract

The embodiment of the invention provides an image generation method, a microscope device and a storage medium, and relates to the technical field of image processing, and the method comprises the steps: obtaining observation images of a to-be-observed object collected by a microscope according to a specified magnification in the illumination directions of at least three preset light sources; determining the illumination direction and illumination intensity of each pixel position in each observation image; calculating an initial normal vector of each pixel position by combining the pixel value, the illumination direction and the illumination intensity of each pixel position in each observation image; the modulus length of the initial normal vector of any pixel position represents a diffuse reflection coefficient of a real position represented by the pixel position; calculating an initial gray value of the pixel position based on the vector representing the specified observation direction and the initial normal vector of the pixel position; and mapping the initial gray value of the pixel position to a gray value range to obtain a target gray value of the pixel position. The detailed structure of the object to be observed can be effectively observed.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image generation method, a microscope device, and a storage medium. Background Technology

[0002] In industrial production, it is often necessary to observe minute parts during the production process to detect defects or monitor their processing status. For example, a microscope can be used to observe these parts. However, the detailed surface structure of minute parts is often difficult to observe. Therefore, how to effectively observe the detailed structure of minute parts (which can be referred to as the objects to be observed) has become an urgent problem to be solved. Summary of the Invention

[0003] The purpose of this application is to provide an image generation method, a microscope device, and a storage medium to effectively observe the detailed structure of an object under a specified observation direction. The specific technical solution is as follows:

[0004] A first aspect of this application provides an image generation method, the method comprising:

[0005] Acquire observation images of the object to be observed by the microscope at a specified magnification under illumination from at least three preset light sources;

[0006] For each preset light source, determine the illumination direction and intensity of each pixel in the observed image under the illumination direction of that preset light source;

[0007] For each pixel location, the initial normal vector is calculated by combining the pixel value, illumination direction, and illumination intensity of that pixel location in the observed image under the illumination direction of each preset light source. The magnitude of the initial normal vector at any pixel location represents the diffuse reflection coefficient of the true location on the surface of the object being observed, as represented by that pixel location. The direction of the initial normal vector at any pixel location represents the direction of the normal vector at the true location on the surface of the object being observed, as represented by that pixel location.

[0008] The initial grayscale value of the pixel is calculated based on the direction vector representing the specified observation direction and the initial normal vector of the pixel position.

[0009] The initial grayscale value of the pixel location is mapped to a grayscale range to obtain the target grayscale value of the pixel location in the grayscale image of the object to be observed under the specified observation direction.

[0010] Optionally, before calculating the initial grayscale value of the pixel position based on the direction vector representing the specified observation direction and the initial normal vector of the pixel position, the method further includes:

[0011] Based on a preset energy function and the initial normal vector of each pixel position, the depth value of each pixel position in the depth image of the object to be observed is determined.

[0012] According to the pre-defined transformation coefficients between the pixel coordinate system and the world coordinate system, the pixel coordinates and depth values ​​of each pixel position are transformed to obtain the world coordinates corresponding to each pixel position; wherein, the first coordinate axis of the world coordinate system is perpendicular to the plane to which the microscope's platform belongs;

[0013] The plane to be corrected is obtained by performing plane fitting based on the world coordinates corresponding to each pixel position.

[0014] Calculate the rotation matrix between the normal vector of the plane to be corrected and the direction of the first coordinate axis;

[0015] The step of calculating the initial grayscale value of the pixel position based on the direction vector representing the specified observation direction and the initial normal vector of the pixel position includes:

[0016] Calculate the product of the rotation matrix and the initial normal vector at the pixel position to obtain the normal vector to be used at the pixel position;

[0017] The initial grayscale value of the pixel is obtained by multiplying the direction vector representing the specified observation direction with the normal vector to be used at the pixel location.

[0018] Optionally, the step of performing plane fitting based on the world coordinates corresponding to each pixel position to obtain the plane to be corrected includes:

[0019] Randomly select the first number of world coordinates corresponding to each pixel position;

[0020] The selected world coordinates are used to perform plane fitting to obtain the current candidate plane; and the process returns to the step of randomly selecting a first number of world coordinates from the world coordinates corresponding to each pixel position until a preset number of candidate planes are fitted.

[0021] Obtain the number of interior points of each candidate plane, which is taken as the number of points corresponding to each candidate plane; wherein, the interior point of any candidate plane represents the position represented by world coordinates at a distance less than a preset distance threshold from the candidate plane.

[0022] The plane to be corrected is determined based on the candidate plane with the largest number of corresponding points.

[0023] Optionally, determining the plane to be corrected based on the candidate plane with the largest number of corresponding points includes:

[0024] The plane to be corrected is obtained by fitting the world coordinates of the interior points of the candidate plane with the largest number of corresponding points.

[0025] Optionally, the preset energy function is as follows:

[0026] J(z)=∫∫((z u -p) 2 +(z v -q) 2 )dudv+λ(∑|z u |+|z v |)

[0027] Where J(z) represents the energy of the depth image; z u and z v These represent the gradients of the depth image along the two coordinate axes of the pixel coordinate system; p = n x / n z q = n y / n z ,(n x n y n z ) represents the unit normal vector at each pixel position; λ represents the regularization constraint strength coefficient.

[0028] Optionally, determining the illumination direction and intensity of each pixel in the observed image under the illumination direction of each preset light source includes:

[0029] For each preset light source, the illumination direction of each pixel in the observed image under the illumination direction of that preset light source is calculated according to the first formula; wherein, the first formula is as follows:

[0030]

[0031] K(u,v) represents the illumination direction of pixel position (u,v); m represents the coordinates of the preset light source in the world coordinate system; p(u,v) represents the coordinates of the actual position on the surface of the object to be observed, represented by pixel position (u,v), in the world coordinate system.

[0032] When the specified magnification is greater than a preset magnification threshold, for each preset light source, the illumination intensity at each pixel position in the observed image under the illumination direction of that preset light source is calculated according to the second formula; wherein, the second formula is as follows:

[0033]

[0034] K(u,v) represents the illumination direction at pixel position (u,v); m represents the coordinates of the preset light source in the world coordinate system; p(u,v) represents the coordinates of the actual position on the surface of the object to be observed, represented by pixel position (u,v), in the world coordinate system; s(u,v) represents the illumination intensity at pixel position (u,v); Ψ represents the illumination intensity at the position of the preset light source; ns represents the principal direction of the preset light source.

[0035] When the specified magnification is not greater than the preset magnification threshold, for each preset light source, the observed image under the illumination direction of the preset light source is uniformly divided according to the grid matrix to obtain a second number of rectangular regions of equal size;

[0036] The illumination intensity at the actual position on the surface of the object to be observed, represented by the center of each rectangular region, is measured and taken as the illumination intensity corresponding to that rectangular region.

[0037] Based on the illumination intensity corresponding to each rectangular region obtained by division, linear interpolation is performed to obtain the illumination intensity of each pixel position in the observed image under the illumination direction of the preset light source.

[0038] Optionally, the step of mapping the initial grayscale value of the pixel location to a grayscale range to obtain the target grayscale value of the pixel location in the grayscale image of the object to be observed under the specified observation direction includes:

[0039] According to the mapping formula, the initial grayscale value of the pixel position is mapped to a grayscale value range to obtain the target grayscale value of the pixel position in the grayscale image of the object to be observed under the specified observation direction; wherein, the mapping formula is as follows:

[0040]

[0041] G(u,v) represents the target grayscale value at pixel position (u,v) in the grayscale image of the object to be observed under the specified observation direction; G c (u,v) represents the initial grayscale value at pixel position (u,v); max(abs(G c )) represents the maximum absolute value among the initial grayscale values ​​at each pixel location.

[0042] Optionally, for each pixel location, the initial normal vector for that pixel location is calculated by combining the pixel value, illumination direction, and illumination intensity of the observed image under the illumination direction of each preset light source, including:

[0043] For each pixel location, based on the pixel value of that pixel location in the observed image under the illumination direction of each preset light source, calculate the brightness value of that pixel location under the illumination direction of each preset light source.

[0044] According to the diffuse reflection formula, and combining the brightness value, illumination direction, and illumination intensity of the pixel location under the illumination direction of each preset light source, the product of the diffuse reflection coefficient and the unit normal vector representing the true position on the surface of the object to be observed at that pixel location is calculated to obtain the initial normal vector of that pixel location; wherein, the diffuse reflection formula is as follows:

[0045] I i (u,v)=s i (u,v)·l i (u,v)·N(u,v)

[0046] st

[0047] N(u,v)=ρ(u,v)·n(u,v)

[0048] I i (u,v) represents the brightness value of pixel position (u,v) under the illumination direction of the i-th preset light source; i (u,v), s i (u,v) represent the illumination direction and illumination intensity of pixel position (u,v) in the observed image under the illumination direction of the i-th preset light source, respectively; N(u,v) represents the initial normal vector of pixel position (u,v), ρ(u,v) represents the diffuse reflection coefficient of the true position on the surface of the object to be observed represented by pixel position (u,v), and n(u,v) represents the unit normal vector of the true position on the surface of the object to be observed represented by pixel position (u,v).

[0049] A second aspect of this application also provides an image generation apparatus, the apparatus comprising:

[0050] The observation image acquisition module is used to acquire observation images of the object to be observed by the microscope at a specified magnification under the illumination directions of at least three preset light sources.

[0051] The illumination data determination module is used to determine the illumination direction and intensity of each pixel in the observed image under the illumination direction of each preset light source.

[0052] The initial normal vector calculation module is used to calculate the initial normal vector for each pixel position by combining the pixel value, illumination direction, and illumination intensity of that pixel position in the observed image under the illumination direction of each preset light source; wherein, the magnitude of the initial normal vector of any pixel position represents the diffuse reflection coefficient of the true position on the surface of the object to be observed represented by that pixel position; the direction of the initial normal vector of any pixel position represents the direction of the normal vector of the true position on the surface of the object to be observed represented by that pixel position.

[0053] The grayscale value calculation module is used to calculate the initial grayscale value of the pixel position based on the direction vector representing the specified observation direction and the initial normal vector of the pixel position.

[0054] The grayscale value mapping module is used to map the initial grayscale value of the pixel position to a grayscale value range, so as to obtain the target grayscale value of the pixel position in the grayscale image of the object to be observed under the specified observation direction.

[0055] Optionally, the device further includes:

[0056] The depth value determination module is used to determine the depth value of each pixel position in the depth image of the object to be observed, based on a preset energy function and the initial normal vector of each pixel position, before calculating the initial gray value of the pixel position based on the direction vector representing the specified observation direction and the initial normal vector of the pixel position.

[0057] The coordinate transformation module is used to transform the pixel coordinates and depth values ​​of each pixel position according to the pre-calibrated transformation coefficients between the pixel coordinate system and the world coordinate system, so as to obtain the world coordinates corresponding to each pixel position; wherein, the first coordinate axis of the world coordinate system is perpendicular to the plane to which the microscope's platform belongs;

[0058] The plane fitting module is used to perform plane fitting based on the world coordinates corresponding to each pixel position to obtain the plane to be corrected.

[0059] A rotation matrix calculation module is used to calculate the rotation matrix between the normal vector of the plane to be corrected and the direction of the first coordinate axis;

[0060] The grayscale value calculation module is specifically used to calculate the product of the rotation matrix and the initial normal vector of the pixel position to obtain the normal vector to be used at the pixel position; and to calculate the product of the direction vector representing the specified observation direction and the normal vector to be used at the pixel position to obtain the initial grayscale value at the pixel position.

[0061] Optionally, the plane fitting module includes:

[0062] The coordinate selection submodule is used to randomly select a first number of world coordinates from the world coordinates corresponding to each pixel position;

[0063] The alternative plane fitting submodule is used to perform plane fitting based on the selected world coordinates to obtain the current alternative plane; and return to execute the step of randomly selecting a first number of world coordinates from the world coordinates corresponding to each pixel position until a preset number of alternative planes are fitted.

[0064] The point count acquisition module is used to acquire the number of interior points of each candidate plane, which is used as the number of points corresponding to each candidate plane; wherein, the interior point of any candidate plane represents the position represented by world coordinates at a distance less than a preset distance threshold from the candidate plane.

[0065] The plane determination submodule is used to determine the plane to be corrected based on the candidate plane with the largest number of corresponding points.

[0066] Optionally, the plane determination submodule is specifically used to perform plane fitting based on the world coordinates of the interior points of the candidate plane with the largest number of corresponding points to obtain the plane to be corrected.

[0067] Optionally, the preset energy function is as follows:

[0068] J(z)=∫∫((z u -p) 2 +(z v -q) 2 )dudv+λ(Σ|z u |+|z v |)

[0069] Where J(z) represents the energy of the depth image; z u and z v These represent the gradients of the depth image along the two coordinate axes of the pixel coordinate system; p = n x / n z q = n y / n z ,(n x n y n z ) represents the unit normal vector at each pixel position; λ represents the regularization constraint strength coefficient.

[0070] Optionally, the illumination data determination module is specifically used to calculate, for each preset light source, the illumination direction of each pixel position in the observed image under the illumination direction of that preset light source according to a first formula; wherein, the first formula is as follows:

[0071]

[0072] K(u,v) represents the illumination direction of pixel position (u,v); m represents the coordinates of the preset light source in the world coordinate system; p(u,v) represents the coordinates of the actual position on the surface of the object to be observed, represented by pixel position (u,v), in the world coordinate system.

[0073] When the specified magnification is greater than a preset magnification threshold, for each preset light source, the illumination intensity at each pixel position in the observed image under the illumination direction of that preset light source is calculated according to the second formula; wherein, the second formula is as follows:

[0074]

[0075] K(u,v) represents the illumination direction at pixel position (u,v); m represents the coordinates of the preset light source in the world coordinate system; p(u,v) represents the coordinates of the actual position on the surface of the object to be observed, represented by pixel position (u,v), in the world coordinate system; s(u,v) represents the illumination intensity at pixel position (u,v); Ψ represents the illumination intensity at the position of the preset light source; ns represents the principal direction of the preset light source.

[0076] When the specified magnification is not greater than the preset magnification threshold, for each preset light source, the observed image under the illumination direction of the preset light source is uniformly divided according to the grid matrix to obtain a second number of rectangular regions of equal size;

[0077] The illumination intensity at the actual position on the surface of the object to be observed, represented by the center of each rectangular region, is measured and taken as the illumination intensity corresponding to that rectangular region.

[0078] Based on the illumination intensity corresponding to each rectangular region obtained by division, linear interpolation is performed to obtain the illumination intensity of each pixel position in the observed image under the illumination direction of the preset light source.

[0079] Optionally, the grayscale value mapping module is specifically used to map the initial grayscale value of the pixel location to a grayscale value range according to a mapping formula, so as to obtain the target grayscale value of the pixel location in the grayscale image of the object to be observed under the specified observation direction; wherein, the mapping formula is as follows:

[0080]

[0081] G(u,v) represents the target grayscale value at pixel position (u,v) in the grayscale image of the object to be observed under the specified observation direction; G c (u,v) represents the initial grayscale value at pixel position (u,v); max(abs(G c )) represents the maximum absolute value among the initial grayscale values ​​at each pixel location.

[0082] Optionally, the initial normal vector calculation module is specifically used to calculate the brightness value of each pixel position under the illumination direction of each preset light source, based on the pixel value of that pixel position in the observed image under the illumination direction of each preset light source.

[0083] According to the diffuse reflection formula, and combining the brightness value, illumination direction, and illumination intensity of the pixel location under the illumination direction of each preset light source, the product of the diffuse reflection coefficient and the unit normal vector representing the true position on the surface of the object to be observed at that pixel location is calculated to obtain the initial normal vector of that pixel location; wherein, the diffuse reflection formula is as follows:

[0084] I i (u,v)=s i (u,v)·l i (u,v)·N(u,v)

[0085] st

[0086] N(u,v)=ρ(u,v)·n(u,v)

[0087] I i (u,v) represents the brightness value of pixel position (u,v) under the illumination direction of the i-th preset light source; i (u,v), s i (u,v) represent the illumination direction and illumination intensity of pixel position (u,v) in the observed image under the illumination direction of the i-th preset light source, respectively; N(u,v) represents the initial normal vector of pixel position (u,v), ρ(u,v) represents the diffuse reflection coefficient of the true position on the surface of the object to be observed represented by pixel position (u,v), and n(u,v) represents the unit normal vector of the true position on the surface of the object to be observed represented by pixel position (u,v).

[0088] A third aspect of the embodiments of this application also provides an electronic device, including:

[0089] Memory, used to store computer programs;

[0090] The processor, when executing a program stored in memory, implements any of the image generation methods described above.

[0091] This application embodiment also provides a microscope device, the device including: a microscope and at least three preset light sources, the microscope being provided with an image sensor and a processor;

[0092] The image sensor is used to acquire observation images of the object to be observed under the illumination directions of the at least three preset light sources at a specified magnification, and to send the acquired observation images to the processor.

[0093] The processor is configured to execute any of the image generation methods described above.

[0094] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the image generation methods described above.

[0095] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute any of the image generation methods described above.

[0096] Beneficial effects of the embodiments in this application:

[0097] This application provides an image generation method that can acquire observation images of an object to be observed under the illumination directions of at least three preset light sources using a microscope at a specified magnification. For each preset light source, the illumination direction and intensity of each pixel position in the observation image under that light source's illumination direction are determined. For each pixel position, an initial normal vector is calculated by combining the pixel value, illumination direction, and intensity of that pixel position in the observation images under the illumination directions of each preset light source. The magnitude of the initial normal vector at any pixel position represents the diffuse reflection coefficient of the true position on the surface of the object to be observed, as represented by that pixel position. The direction of the initial normal vector at any pixel position represents the direction of the normal vector of the true position on the surface of the object to be observed, as represented by that pixel position. Based on the direction vector representing the specified observation direction and the initial normal vector of the pixel position, an initial grayscale value is calculated for that pixel position. The initial grayscale value of the pixel position is mapped to a grayscale value range to obtain the target grayscale value of that pixel position in the grayscale image of the object to be observed under the specified observation direction.

[0098] Based on the above processing, since the diffuse reflection coefficient of the detailed structure (e.g., scratches) on the surface of the object being observed is often large, while the diffuse reflection coefficient at the real location where noise is easily generated is often small, the initial normal vector of each pixel position in the grayscale image of the object being observed under a specified observation direction is combined during the process of obtaining the target grayscale value of each pixel position in the grayscale image of the object being observed under a specified observation direction. The magnitude of the initial normal vector of any pixel position can represent the diffuse reflection coefficient of the real location on the surface of the object being observed represented by that pixel position. Therefore, the target grayscale value of the pixel position corresponding to the detailed structure of the surface of the object being observed in the grayscale image can be increased, while the target grayscale value of the pixel position corresponding to the real location on the surface of the object being observed under a specified noise generation in the grayscale image can be decreased. In this way, the detailed structure of the surface of the object being observed in the grayscale image can be highlighted, and noise in the grayscale image can be suppressed. That is, the grayscale image can effectively reflect the detailed structure of the surface of the object being observed under a specified observation direction with low noise. Correspondingly, based on the grayscale image, the detailed structure of the object being observed under a specified observation direction can be effectively observed.

[0099] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0100] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0101] Figure 1 This is a schematic diagram of a first flowchart of the image generation method provided in the embodiments of this application;

[0102] Figure 2 This is a schematic diagram of the light intensity corresponding to each rectangular region measured in the image generation method provided in the embodiments of this application;

[0103] Figure 3 This is a second flowchart illustrating the image generation method provided in the embodiments of this application;

[0104] Figure 4a A schematic diagram of a depth image of an object to be observed, provided as an embodiment of this application;

[0105] Figure 4b A schematic diagram of another depth image of an object to be observed, provided as an embodiment of this application;

[0106] Figure 5 This is a schematic diagram of a third process for the image generation method provided in the embodiments of this application;

[0107] Figure 6a A schematic diagram of an observation image provided in an embodiment of this application;

[0108] Figure 6b A schematic diagram of a grayscale image obtained by an image generation method according to an embodiment of this application;

[0109] Figure 7 This is a schematic diagram of the structure of an image generation device provided in an embodiment of this application;

[0110] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0111] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0112] In industrial production, it is often necessary to observe tiny parts during the manufacturing process to detect defects or monitor their processing status. For example, a microscope can be used to observe tiny parts. However, the detailed surface structure of tiny parts is often difficult to observe.

[0113] To effectively observe the detailed structure of minute parts (which can be referred to as the object to be observed), embodiments of this application provide an image generation method, see [link to relevant documentation]. Figure 1 , Figure 1 This is a schematic diagram of a first flowchart of an image generation method provided in an embodiment of this application. The method includes:

[0114] Step S101: Acquire observation images of the object to be observed by the microscope at a specified magnification under illumination directions from at least three preset light sources.

[0115] Step S102: For each preset light source, determine the illumination direction and illumination intensity of each pixel in the observed image under the illumination direction of the preset light source.

[0116] Step S103: For each pixel location, calculate the initial normal vector of that pixel location by combining the pixel value, illumination direction, and illumination intensity of the observed image under the illumination direction of each preset light source.

[0117] Wherein, the magnitude of the initial normal vector at any pixel position represents the diffuse reflection coefficient of the true position on the surface of the object to be observed, as represented by that pixel position; the direction of the initial normal vector at any pixel position represents the direction of the normal vector of the true position on the surface of the object to be observed, as represented by that pixel position.

[0118] Step S104: Calculate the initial gray value of the pixel position based on the direction vector representing the specified observation direction and the initial normal vector of the pixel position.

[0119] Step S105: Map the initial gray value of the pixel location to a gray value range to obtain the target gray value of the pixel location in the grayscale image of the object to be observed under the specified observation direction.

[0120] Based on the above processing, since the diffuse reflection coefficient of the detailed structure (e.g., scratches) on the surface of the object being observed is often large, while the diffuse reflection coefficient at the real location where noise is easily generated is often small, the initial normal vector of each pixel position in the grayscale image of the object being observed under a specified observation direction is combined during the process of obtaining the target grayscale value of each pixel position in the grayscale image of the object being observed under a specified observation direction. The magnitude of the initial normal vector of any pixel position can represent the diffuse reflection coefficient of the real location on the surface of the object being observed represented by that pixel position. Therefore, the target grayscale value of the pixel position corresponding to the detailed structure of the surface of the object being observed in the grayscale image can be increased, while the target grayscale value of the pixel position corresponding to the real location on the surface of the object being observed under a specified noise generation in the grayscale image can be decreased. In this way, the detailed structure of the surface of the object being observed in the grayscale image can be highlighted, and noise in the grayscale image can be suppressed. That is, the grayscale image can effectively reflect the detailed structure of the surface of the object being observed under a specified observation direction with low noise. Correspondingly, based on the grayscale image, the detailed structure of the object being observed under a specified observation direction can be effectively observed.

[0121] Based on the image generation method provided in this application, the resulting grayscale image can effectively reflect the detailed structure of the surface of the object under observation in a specified viewing direction with low noise. That is, it can highlight the concave and convex detailed structure information of the surface of the object under observation in a specified viewing direction, and can be applied to defect detection and observation of micro-parts under a microscope, such as observing the processing status of micro-parts.

[0122] For step S101, at least three preset light sources (e.g., point light sources) with different illumination directions can be set to illuminate the object to be observed from different directions. For example, one preset light source can be set first, followed by another preset light source with a different illumination direction, and so on, setting at least three preset light sources sequentially, ensuring that only one preset light source illuminates the object to be observed at a time. The object to be observed can be a tiny part that needs to be observed. The number of preset light sources can be set as needed; for example, four or five preset light sources with different illumination directions can be set. For each preset light source, an observation image of the object to be observed can be acquired by the microscope at a specified magnification under the illumination direction of that preset light source. For example, the observation image can be an RGB (Red, Green, Blue) image. The specified magnification can be set according to the size of the object to be observed and the observation requirements, and is not specifically limited. For example, the specified magnification can be 20x or 60x.

[0123] Regarding step S102, for each preset light source, the illumination direction and intensity of each pixel in the observed image under the illumination direction of that preset light source can be determined. In one implementation, since the positions of each preset light source are fixed in a scene where an object is observed under a microscope, and the object is often a tiny object, it can be assumed to be a plane. Accordingly, for each preset light source, the illumination direction and intensity of each pixel in the observed image under the illumination direction of that preset light source can be calculated using the first formula and the second formula, respectively, by utilizing the coordinates of the actual position on the surface of the object represented by each pixel position in the world coordinate system.

[0124] The world coordinate system can be set as needed without specific limitations. For example, the first coordinate axis of the world coordinate system can be perpendicular to the plane to which the microscope's platform belongs, and the plane to which the second and third coordinate axes of the world coordinate system belong is the same plane to which the microscope's platform belongs. Thus, the first, second, and third coordinate axes of the world coordinate system can be referred to as the Z-axis, X-axis, and Y-axis of the world coordinate system, respectively.

[0125] The first and second formulas can also be referred to as the point light source model. The first formula is as follows:

[0126]

[0127] K(u,v) represents the illumination direction of the pixel position (u,v); m represents the coordinates of the preset light source in the world coordinate system; p(u,v) represents the coordinates of the actual position on the surface of the object to be observed, represented by the pixel position (u,v), in the world coordinate system.

[0128] The second formula is as follows:

[0129]

[0130] K(u,v) represents the illumination direction at pixel position (u,v); m represents the coordinates of the preset light source in the world coordinate system; p(u,v) represents the coordinates of the actual position on the surface of the object to be observed, represented by pixel position (u,v), in the world coordinate system; s(u,v) represents the illumination intensity at pixel position (u,v); Ψ represents the illumination intensity at the position of the preset light source; and ns represents the principal direction of the preset light source.

[0131] In another implementation, step S102 includes:

[0132] Step 1: For each preset light source, calculate the illumination direction of each pixel in the observed image under the illumination direction of that preset light source according to the first formula.

[0133] Step 2: When the specified magnification is greater than the preset magnification threshold, for each preset light source, calculate the illumination intensity of each pixel position in the observed image under the illumination direction of the preset light source according to the second formula.

[0134] Step 3: Under the condition that the specified magnification is not greater than the preset magnification threshold, for each preset light source, the observation image under the illumination direction of the preset light source is uniformly divided according to the grid matrix to obtain a second number of rectangular regions of equal size.

[0135] Step 4: Measure the light intensity at the actual position on the surface of the object to be observed, represented by the center of each rectangular region, as the light intensity corresponding to that rectangular region.

[0136] Step 5: Based on the illumination intensity corresponding to each rectangular region obtained by division, perform linear interpolation to obtain the illumination intensity of each pixel position in the observed image under the illumination direction of the preset light source.

[0137] In this implementation, for each preset light source, the illumination direction of each pixel in the observed image under the illumination direction of that preset light source can be calculated according to the first formula. The illumination intensity of each pixel in the observed image under the illumination direction of that preset light source can be determined based on a specified magnification. When the specified magnification is not greater than a preset magnification threshold (which can be called a low magnification), for each preset light source, the observed image under the illumination direction of that preset light source can be uniformly divided into a second number of equally sized rectangular regions using a grid matrix. For example, the preset magnification threshold can be 95x or 100x. The number of rows and columns of the grid matrix can be set as needed. For example, the grid matrix can be 7 (rows) × 5 (columns), and the corresponding second number can be 35; or the grid matrix can be 6 × 6, and the corresponding second number can be 36.

[0138] The true position on the surface of the object to be observed, represented by the center of each rectangular region, can be determined. The illumination intensity at this true position on the surface of the object to be observed, represented by the center of each rectangular region, can then be measured to obtain the illumination intensity corresponding to each rectangular region. For example, the illumination intensity corresponding to each rectangular region can be measured using an illumination intensity measuring instrument (also called a light source radiation measuring instrument). Figure 2 As shown, Figure 2 This is a schematic diagram of the light intensity corresponding to each rectangular region measured in the image generation method provided in the embodiments of this application. Figure 2The top left, top right, bottom left, and bottom right in the image represent the positions of each preset light source relative to the object being observed. Taking the top left as an example, it means the preset light source is positioned to the upper left of the object, and correspondingly, the preset light source can illuminate the object from the upper left. For each preset light source, the observation image under that light direction can be uniformly divided into 35 equal-sized rectangular regions using a 7×5 grid matrix. The number in each rectangular region represents the measured illumination intensity of that region, in lux. For example, the rectangular region in the first row and first column of the top left observation image has an illumination intensity of 96364.37 lux.

[0139] Furthermore, linear interpolation can be performed on the illumination intensity corresponding to each of the divided rectangular regions to obtain an illumination intensity map consistent with the image resolution of the observed image. For example, for every two adjacent rectangular regions, the illumination intensity inserted between them is determined based on the illumination intensity corresponding to those two regions. For instance, the inserted illumination intensity can be the average of the illumination intensities corresponding to those two regions. Interpolation is continued between every two adjacent illumination intensities until the number of illumination intensities obtained is the same as the number of pixel locations in the observed image, thus obtaining an illumination intensity map consistent with the image resolution of the observed image. That is, the illumination intensity at each pixel location is obtained.

[0140] When the specified magnification is greater than a preset magnification threshold (which can be called high magnification), the area represented by the observed image on the surface of the object is small due to the high magnification of the microscope observation. Consequently, the number of measurable points within this area is also limited. In this case, the accuracy of determining the illumination intensity at each pixel location by dividing the area as described above is not high. Therefore, the method of calculating the illumination intensity at each pixel location according to the second formula is chosen.

[0141] Based on the above processing, the method for determining the illumination intensity at each pixel location can be selected according to the specified magnification. At low magnification, the illumination intensity at each pixel location can be determined by combining the actual measured illumination intensity with linear interpolation. At high magnification, the illumination intensity at each pixel location can be calculated according to the second formula. Compared to the illumination intensity calculated based on the formula assumptions, the accuracy of the actual measured illumination intensity is higher. Thus, while ensuring the illumination intensity at each pixel location is obtained, the accuracy of the determined illumination intensity at each pixel location is improved, which in turn improves the accuracy of the initial normal vectors obtained subsequently for each pixel location. Correspondingly, in the process of obtaining the target grayscale value of each pixel location in the grayscale image of the object to be observed under a specified observation direction, combining the initial normal vectors of each pixel location improves the effectiveness of the grayscale image, further ensuring that the grayscale image can effectively achieve the observation of the detailed structure of the object to be observed under a specified observation direction.

[0142] Regarding step S103, after obtaining the illumination direction and intensity of each pixel position in the observed image under the illumination direction of each preset light source, for each pixel position, the initial normal vector of that pixel position can be calculated using the pixel value, illumination direction, and illumination intensity in the observed image under the illumination direction of each preset light source. For example, it can be calculated based on the principle of diffuse reflection imaging. By executing steps S101-S103, the initial normal vector of each pixel position can be determined using the observed images under different illumination directions and the imaging principle; that is, the normal vector of the true position on the surface of the object to be observed, represented by each pixel position, can be determined. In other words, the normal vector is determined based on the photometric stereo vision algorithm.

[0143] In one implementation, step S103 includes:

[0144] For each pixel location, based on the pixel value of that pixel location in the observed image under the illumination direction of each preset light source, calculate the brightness value of that pixel location under the illumination direction of each preset light source.

[0145] According to the diffuse reflection formula, combined with the brightness value, illumination direction and illumination intensity of the pixel position under the illumination direction of each preset light source, the product of the diffuse reflection coefficient of the real position on the surface of the object to be observed represented by the pixel position and the unit normal vector is calculated to obtain the initial normal vector of the pixel position.

[0146] The formula for diffuse reflection is as follows:

[0147] I i (u,v)=s i (u,v)·l i(u,v)·N(u,v)

[0148] st

[0149] N(u,v)=ρ(u,v)·n(u,v)

[0150] I i (u,v) represents the brightness value of pixel position (u,v) under the illumination direction of the i-th preset light source; i (u,v), s i (u,v) represent the illumination direction and illumination intensity of pixel position (u,v) in the observed image under the illumination direction of the i-th preset light source, respectively; N(u,v) represents the initial normal vector of pixel position (u,v), ρ(u,v) represents the diffuse reflection coefficient of the true position on the surface of the object to be observed represented by pixel position (u,v), and n(u,v) represents the unit normal vector of the true position on the surface of the object to be observed represented by pixel position (u,v).

[0151] In this implementation, for each pixel location, the luminance value of that pixel location under the illumination direction of each preset light source can be calculated based on the pixel value of that location in the observed image under the illumination direction of each preset light source. For example, the observed image can be an RGB image, which can be converted into a luminance image. The luminance image can be in HSV (a color model) format or YUV (a color encoding method) format. Correspondingly, the V value of that pixel location in the luminance image is the luminance value of that pixel location. Furthermore, according to the diffuse reflection formula, combined with the luminance value, illumination direction, and illumination intensity of that pixel location under the illumination direction of any preset light source, an equation can be obtained for the product of the diffuse reflection coefficient and the unit normal vector (i.e., N(u,v)) representing the true position on the surface of the observed object represented by that pixel location.

[0152] Since at least three preset light sources with different illumination directions are set, at least three equations concerning N(u,v) can be obtained. Furthermore, there are only three unknowns in N(u,v), which are the components of N(u,v) along the three coordinate axes of the world coordinate system. Combining these at least three equations concerning N(u,v), N(u,v) can be solved, thus obtaining the initial normal vector at that pixel position. The magnitude and direction of the initial normal vector at any pixel position represent, respectively, the diffuse reflection coefficient of the true position on the surface of the observed object, and the direction of the normal vector.

[0153] Based on the above processing, the initial normal vector at each pixel position can be calculated, and the magnitude of the initial normal vector at any pixel position can represent the diffuse reflection coefficient of the true position on the surface of the object to be observed, as represented by that pixel position. Subsequently, in the process of obtaining the target grayscale value of each pixel position in the grayscale image of the object to be observed under a specified observation direction, the initial normal vector at each pixel position can be used for calculation. In this way, it is possible to further ensure that the detailed structure of the surface of the object to be observed in the grayscale image can be highlighted, while suppressing noise in the grayscale image. That is, the grayscale image can effectively reflect the detailed structure of the surface of the object to be observed with low noise. Accordingly, based on the grayscale image, it is also possible to effectively observe the detailed structure of the object to be observed.

[0154] Regarding step S104, the user currently needs to observe the detailed structure of the object to be observed from a specified observation direction. The direction vector can represent the specified observation direction; for example, the direction vector can be represented as (H... x H y H z ), H z =0. For each pixel location, the initial grayscale value of that pixel location can be calculated based on the direction vector and the initial normal vector of that pixel location. In one implementation, the initial grayscale value of that pixel location can be obtained by calculating the product of the direction vector and the initial normal vector of that pixel location.

[0155] Due to factors such as microscope installation errors, uneven illumination, and the tilt of the object being observed, the initial normal vectors of each pixel may be biased in a certain direction, which can be called the tilt direction. In this case, if the product of the direction vector and the initial normal vector of the pixel is directly calculated to obtain the initial grayscale value of the pixel, the grayscale values ​​of each pixel in the subsequent grayscale image may be uneven, resulting in one side being bright and the other dark, which can easily obscure the detailed structure of the object being observed. Therefore, in another implementation, the initial normal vector of the pixel can be corrected to remove the tilt direction of the initial normal vector of the pixel, so as to obtain the usable normal vector of the pixel. The specific correction method can be referred to the relevant descriptions of steps S106-S109 and step S1041 in the following embodiments.

[0156] For example, the initial grayscale value at that pixel location can be calculated using the re-rendering formula, which is as follows:

[0157] G c (u,v)=H·N

[0158] Among them, G c(u,v) represents the initial grayscale value of pixel position (u,v); H represents the direction vector; N represents the initial normal vector N(u,v) of pixel position (u,v), or the normal vector N to be used at pixel position (u,v). c (u,v). That is, to perform secondary reshading on the detailed structure of the object to be observed under the specified viewing direction.

[0159] Regarding step S105, for each pixel location, the initial grayscale value of that pixel location may not fall within the grayscale value range (i.e., 0-255). Therefore, the initial grayscale value of that pixel location can be mapped to the grayscale value range to obtain the grayscale value of that pixel location in the grayscale image of the object to be observed under the specified observation direction (i.e., the target grayscale value). That is, a grayscale image of the object to be observed under the specified observation direction can be obtained. Subsequently, the obtained grayscale image can be browsed to observe the detailed structure of the object under the specified observation direction.

[0160] In one implementation, the minimum initial grayscale value of each pixel location can be directly mapped to 0, and the maximum initial grayscale value of each pixel location can be mapped to 255. However, the number of positive and negative values ​​in the initial grayscale values ​​of each pixel location may not be uniform. If there are more positive or negative values ​​in the initial grayscale values ​​of each pixel location, the target grayscale values ​​of each mapped pixel location may be concentrated in a certain range. In this case, the difference between the target grayscale values ​​within this range may be small, and thus it is impossible to effectively reflect the detailed structure of the surface of the object being observed.

[0161] In another implementation, to avoid the above problems, step S105 includes:

[0162] According to the mapping formula, the initial grayscale value of the pixel is mapped to a grayscale range to obtain the target grayscale value of that pixel in the grayscale image of the object to be observed under the specified observation direction; the mapping formula is as follows:

[0163]

[0164] G(u,v) represents the target grayscale value at pixel position (u,v) in the grayscale image of the object to be observed under a specified observation direction; G c (u,v) represents the initial grayscale value at pixel position (u,v); max(abs(G c )) represents the maximum absolute value among the initial grayscale values ​​at each pixel location.

[0165] In this embodiment, the initial grayscale value of a pixel can be mapped to a grayscale range according to a mapping formula to obtain the target grayscale value of that pixel. The initial grayscale value of each pixel is the product of two vectors, and may be negative. When the initial grayscale value is 0, mapping this initial grayscale value according to the mapping formula yields a target grayscale value of 127.5 (i.e., the midpoint of the grayscale range). When the initial grayscale value is negative, mapping this initial grayscale value according to the mapping formula yields a target grayscale value between 0 and 127.5. When the initial grayscale value is positive, mapping this initial grayscale value according to the mapping formula yields a target grayscale value between 127.5 and 255.

[0166] Compared to the method described above, which directly maps the minimum initial grayscale value of each pixel to 0 and the maximum initial grayscale value to 255, this approach maps the initial grayscale values ​​of each pixel in both the positive and negative directions to the two halves of the grayscale range, i.e., uniformly mapping the initial grayscale values ​​of each pixel to 0-255. This ensures the accuracy of the target grayscale values ​​obtained for each pixel. Furthermore, it guarantees that the detailed structure of the object being observed can be effectively captured based on the grayscale image.

[0167] In one embodiment, see Figure 3 , Figure 3 This is a second flowchart illustrating the image generation method provided in this application embodiment. Before step S104, the method further includes:

[0168] Step S106: Based on the preset energy function and the initial normal vector of each pixel position, determine the depth value of each pixel position in the depth image of the object to be observed.

[0169] Step S107: According to the pre-calibrated conversion coefficient between the pixel coordinate system and the world coordinate system, the pixel coordinates and depth values ​​of each pixel position are converted to obtain the world coordinates corresponding to each pixel position.

[0170] The first coordinate axis of the world coordinate system is perpendicular to the plane of the microscope's platform.

[0171] Step S108: Perform plane fitting based on the world coordinates corresponding to each pixel position to obtain the plane to be corrected.

[0172] Step S109: Calculate the rotation matrix between the normal vector of the plane to be corrected and the direction of the first coordinate axis.

[0173] Step S104 includes:

[0174] Step S1041: Calculate the product of the rotation matrix and the initial normal vector of the pixel position to obtain the normal vector to be used at the pixel position.

[0175] Step S1042: Calculate the product of the direction vector representing the specified observation direction and the normal vector to be used at the pixel position to obtain the initial gray value of the pixel position.

[0176] In this embodiment, the depth value of each pixel in the depth image of the object to be observed can be determined based on a preset energy function and the initial normal vector of each pixel position. For example, the function value of the preset energy function can be minimized by combining the initial normal vector of each pixel position to obtain the depth value of each pixel position when the function value of the preset energy function is minimized. Furthermore, the transformation coefficient between the pixel coordinate system and the world coordinate system can be pre-calibrated. For each pixel position, the pixel coordinates and depth value of that pixel position can be transformed using the transformation coefficient to obtain the world coordinates corresponding to that pixel position. For example, the product of the transformation coefficient and the pixel coordinates of that pixel position can be calculated to obtain the components of the world coordinates corresponding to that pixel position on the second and third coordinate axes of the world coordinate system. The depth value of that pixel position is then used as the component of the world coordinates corresponding to that pixel position on the first coordinate axis of the world coordinate system to obtain the world coordinates corresponding to that pixel position.

[0177] After obtaining the world coordinates corresponding to each pixel position, a plane can be fitted based on these world coordinates to obtain the plane to be corrected. For example, the least squares method can be used for plane fitting.

[0178] In one implementation, a plane fitting can be performed using the world coordinates corresponding to all pixel positions to obtain a plane, which can then be used as the plane to be corrected.

[0179] In another implementation, a plane can be fitted using a subset of world coordinates selected from the world coordinates corresponding to each pixel location to obtain a plane, which serves as the plane to be corrected. Alternatively, a subset of world coordinates can be randomly selected multiple times from the world coordinates corresponding to each pixel location, and a plane can be fitted based on each selected world coordinate to obtain multiple planes. The plane to be corrected is then determined based on these multiple planes. For details, please refer to the relevant descriptions of steps 1-4 in the subsequent embodiments.

[0180] After obtaining the plane to be corrected, the rotation matrix between the normal vector of the plane and the direction of the first coordinate axis of the world coordinate system can be calculated. Using the rotation matrix, n can be... totalThe tilt direction is transformed to (0,0,1), where (0,0,1) represents the direction of the first coordinate axis of the world coordinate system. This means the tilt direction can be removed. For each pixel position, the product of the rotation matrix and the initial normal vector of that pixel position can be calculated to remove the tilt direction of the initial normal vector, resulting in the usable normal vector for that pixel position. For example, the usable normal vector for that pixel position can be calculated using the following formula:

[0181] N c (u,v)=R·N(u,v)

[0182] Where, N c (u,v) represents the normal vector to be used at pixel position (u,v); R represents the rotation matrix; N(u,v) represents the initial normal vector at pixel position (u,v).

[0183] Based on the above processing, the initial normal vector of the pixel position can be corrected to remove its tilt direction, thus obtaining the usable normal vector for that pixel position. This avoids the problem of uneven grayscale values ​​at different pixel positions in the subsequent grayscale image due to tilt, which would mask the detailed structure of the object being observed in the grayscale image. In other words, it can ignore the tilt direction and highlight the detailed structure of the object being observed in the grayscale image. This further ensures that the detailed structure of the object being observed under a specified observation direction can be effectively observed based on the grayscale image.

[0184] In this application, the preset energy function can take the following two forms:

[0185] The first type of preset energy function is as follows:

[0186] J(z)=∫∫((z u -p) 2 +(z v -q) 2 )dudv

[0187] Where J(z) represents the energy of the depth image; z u and z v These represent the gradients of the depth image along the two coordinate axes (u-axis and v-axis) of the pixel coordinate system; p = n x / n z q = n y / n z ,(n x n y n z ) represents the unit normal vector at each pixel position.

[0188] Since the object to be observed may have a vertical boundary, |n| at the vertical boundaryz |→0, in this case, p and q will tend to infinity. Correspondingly, in order to minimize the value of the preset energy function, z u and z v It will also tend towards infinity, which may lead to depth images appearing as... Figure 4a The phenomenon shown within the rectangular frame is that irregular black textures appear at the vertical boundary of the object being observed, failing to reflect the true boundary of the object. Figure 4a This is a schematic diagram of a depth image of an object to be observed, provided in an embodiment of this application. To avoid this phenomenon, a second form of preset energy function can be used. The second form of preset energy function is as follows:

[0189] J(z)=∫∫((z u -p) 2 +(z v -q) 2 )dudv+λ(Σ|z u |+|z v |)

[0190] Where J(z) represents the energy of the depth image; z u and z v These represent the gradients of the depth image along the two coordinate axes of the pixel coordinate system; p = n x / n z q = n y / n z ,(n x n y n z ) represents the unit normal vector at each pixel location; λ represents the regularization constraint strength coefficient. For example, the regularization constraint strength coefficient can be any value between 0.01 and 0.05.

[0191] Based on the second form of the preset energy function, when |n z When |→0, due to z u and z v Regularization constraints were added (i.e., λ(∑|z)). u |+|z v Therefore, in order to minimize the value of the preset energy function, z u and z v It also needs to be as small as possible; correspondingly, z u and z v Therefore, it will not tend towards infinity, and thus can avoid |n z |→0 causes the depth image to appear as follows Figure 4a The phenomenon shown within the rectangular box can improve the accuracy of the obtained depth image. The depth image obtained based on the second form of the preset energy function is as follows: Figure 4b As shown, Figure 4b The true boundary of the object being observed can be displayed correctly within the rectangular frame. Figure 4b This is a schematic diagram of another depth image of the object to be observed provided in an embodiment of this application. This improves the accuracy of correcting each initial normal vector, ensuring the accuracy of the subsequently obtained grayscale image, and further ensuring that the detailed structure of the object to be observed can be effectively realized based on the grayscale image.

[0192] In one embodiment, step S108 includes:

[0193] Step 1: Randomly select the first number of world coordinates from the world coordinates corresponding to each pixel position.

[0194] Step 2: Perform plane fitting based on the selected world coordinates to obtain the current candidate plane; then return to Step 1 and continue until a preset number of candidate planes are fitted.

[0195] Step 3: Obtain the number of interior points of each candidate plane, which will be used as the number of points corresponding to each candidate plane.

[0196] Wherein, the interior point of any candidate plane represents the position of the world coordinate representation of the distance between the candidate plane and the world coordinate representation of the distance less than a preset distance threshold.

[0197] Step 4: Determine the plane to be corrected based on the candidate plane with the largest number of corresponding points.

[0198] In this embodiment, a first number of world coordinates can be randomly selected from the world coordinates corresponding to each pixel position. The first number can be determined based on the number of pixel positions and is not specifically limited. For example, if the resolution of the observed image is 640×480, that is, the number of pixel positions is 307200, and the first number can be 800 or 1000. Plane fitting is performed based on the selected world coordinates to obtain candidate planes. The first number of world coordinates can then be randomly selected again from the world coordinates corresponding to each pixel position, and plane fitting is performed based on the newly selected world coordinates to obtain new candidate planes. This process is repeated until a preset number of candidate planes are obtained. For example, the preset number can be 10 or 12.

[0199] For each candidate plane, world coordinates (which can be called interior coordinates) can be determined for distances less than a preset distance threshold from that candidate plane. The position represented by the interior coordinates can be called an interior point of that candidate plane. Accordingly, the number of interior points of that candidate plane can be obtained as the number of points corresponding to that candidate plane. Furthermore, the plane to be corrected can be determined based on the candidate plane with the largest number of corresponding points.

[0200] Based on the above processing, the plane to be corrected is determined through steps 1-4, which can also be called determining the plane to be corrected using the Ransac (random sample consensus) method. Compared to using the world coordinates corresponding to all pixel positions to perform a plane fitting once to obtain a plane as the plane to be corrected, this method reduces the computational load of plane fitting and improves the efficiency of obtaining the plane to be corrected by using only a selected portion of the world coordinates. This, in turn, improves the efficiency of obtaining the grayscale image used for observing the object. Furthermore, determining the plane to be corrected using the Ransac method can, to some extent, reduce the influence of outliers on the fitted plane to be corrected, improving the accuracy of the obtained plane to be corrected. This, in turn, improves the accuracy of the subsequent grayscale image, further ensuring that the detailed structure of the object to be observed can be effectively realized based on the grayscale image.

[0201] In this application, the plane to be corrected can be determined by the following methods one and two.

[0202] In Method 1, the candidate plane with the largest number of corresponding points can be directly selected as the plane to be corrected.

[0203] In Method 2, step 4 includes: performing plane fitting based on the world coordinates of the interior points of the candidate plane with the largest number of corresponding points to obtain the plane to be corrected.

[0204] In this method, plane fitting is performed based on the world coordinates of the interior points of the candidate plane with the largest number of corresponding points to obtain the plane to be corrected, which can also be called the refinement process. In this way, the accuracy of the obtained plane to be corrected can be further improved, which in turn can improve the accuracy of the subsequently obtained grayscale image, and further ensure that the grayscale image can effectively realize the observation of the detailed structure of the object under the specified observation direction.

[0205] In one embodiment, see Figure 5 , Figure 5 This is a schematic diagram of a third process for an image generation method provided in an embodiment of this application. The image generation method includes:

[0206] Step S501: Acquire images from n (n≥3) different lighting directions. That is, step S101 in the above embodiment.

[0207] Step S502: Using the point light source model and a preset light source intensity calibration method, calculate the illumination direction and illumination intensity of each pixel. That is, steps one to five in the above embodiments.

[0208] Step S503: Based on the principle of photometric stereo vision, assuming a diffuse reflection model, estimate the normal vector of each pixel. That is, step S103 in the above embodiment.

[0209] Step S504: Recover the depth map based on the normal vector map obtained from photometric stereo vision. That is, step S106 in the above embodiment.

[0210] Step S505: Using the recovered depth map, perform overall plane fitting using the Ransac method to obtain the overall tilt direction of the object. That is, steps S107, steps 1-4, and step S109 in the above embodiment.

[0211] Step S506: Correct the original normal vector diagram according to the overall tilt direction of the object. That is, step S1041 in the above embodiment.

[0212] Step S507: Based on the corrected normal vector map, perform a second re-render according to the specified direction to obtain the new observation map. That is, steps S1042 and S105 in the above embodiment.

[0213] Based on the above processing, images under various lighting directions can be used, combined with photometric stereo vision algorithms, to recover the normal vector of the object's surface. The normal vector is then used for re-rendering to obtain a grayscale image of the object under the specified viewing direction. Furthermore, the magnitude of the normal vector can represent the diffuse reflectance coefficient of the object's surface, thus highlighting the detailed structure of the object in the grayscale image.

[0214] See Figure 6a and Figure 6b , Figure 6a This is a schematic diagram of an observation image provided in an embodiment of this application. Figure 6b This is a schematic diagram illustrating a grayscale image obtained by an image generation method according to an embodiment of this application. As can be seen, Figure 6a The surface details of the object being observed cannot be observed in the middle. Figure 6b It can be effectively observed, for example, for Figure 6a The surface is concave at the location indicated by the middle circle. Figure 6a It is not obvious in the middle, but in Figure 6b At the location indicated by the circle in the middle, the surface depression can be clearly observed. That is, the grayscale image obtained based on the image generation method provided in this application embodiment can effectively reflect the detailed structure of the surface of the object to be observed under a specified observation direction, and thus can effectively realize the observation of the detailed structure of the object to be observed under a specified observation direction.

[0215] Based on the same inventive concept, this application also provides a microscope device, the device comprising: a microscope and at least three preset light sources, wherein the microscope is provided with an image sensor and a processor;

[0216] The image sensor is used to acquire observation images of the object to be observed under the illumination directions of the at least three preset light sources at a specified magnification, and to send the acquired observation images to the processor.

[0217] The processor is configured to execute any of the image generation methods described above.

[0218] The microscope in this application embodiment can be an optical microscope, and the microscope may also include: objective lens, eyepiece, condenser, mirror, filter, microscope base, microscope arm, stage, microscope tube, objective lens turret and focusing component, etc.

[0219] Based on the microscope apparatus provided in this application embodiment, since the diffuse reflection coefficient of the detailed structure (e.g., scratches) on the surface of the object to be observed is often large, while the diffuse reflection coefficient at the real location where noise is easily generated is often small, the initial normal vector of each pixel position in the grayscale image of the object to be observed under a specified observation direction is combined during the process of obtaining the target grayscale value of each pixel position in the grayscale image of the object to be observed under a specified observation direction. The magnitude of the initial normal vector of any pixel position can represent the diffuse reflection coefficient of the real location on the surface of the object to be observed represented by that pixel position. Therefore, the target grayscale value of the pixel position corresponding to the detailed structure on the surface of the object to be observed in the grayscale image can be increased, while the target grayscale value of the pixel position corresponding to the real location on the surface of the object to be observed that is easily generated in the grayscale image can be decreased. In this way, the detailed structure on the surface of the object to be observed in the grayscale image can be highlighted, and noise in the grayscale image can be suppressed. That is, the grayscale image can effectively reflect the detailed structure on the surface of the object to be observed under a specified observation direction with low noise. Accordingly, based on the grayscale image, the detailed structure of the object to be observed under a specified observation direction can be effectively observed.

[0220] Based on the same inventive concept, this application also provides an image generation apparatus, see [link to relevant documentation]. Figure 7 , Figure 7 This is a schematic diagram of an image generation apparatus provided in an embodiment of this application. The apparatus includes:

[0221] The observation image acquisition module 701 is used to acquire observation images of the object to be observed by the microscope at a specified magnification under the illumination directions of at least three preset light sources.

[0222] The illumination data determination module 702 is used to determine the illumination direction and illumination intensity of each pixel position in the observed image under the illumination direction of each preset light source.

[0223] The initial normal vector calculation module 703 is used to calculate the initial normal vector for each pixel position by combining the pixel value, illumination direction, and illumination intensity of that pixel position in the observed image under the illumination direction of each preset light source; wherein, the magnitude of the initial normal vector of any pixel position represents the diffuse reflection coefficient of the true position on the surface of the object to be observed represented by that pixel position; the direction of the initial normal vector of any pixel position represents the direction of the normal vector of the true position on the surface of the object to be observed represented by that pixel position.

[0224] The grayscale value calculation module 704 is used to calculate the initial grayscale value of the pixel position based on the direction vector representing the specified observation direction and the initial normal vector of the pixel position.

[0225] The grayscale value mapping module 705 is used to map the initial grayscale value of the pixel position to a grayscale value range to obtain the target grayscale value of the pixel position in the grayscale image of the object to be observed under the specified observation direction.

[0226] Based on the image generation apparatus provided in this application embodiment, since the diffuse reflection coefficient of the detailed structure (e.g., scratches) of the surface of the object to be observed is often large, while the diffuse reflection coefficient of the real location that is prone to noise is often small, the initial normal vector of each pixel position in the grayscale image of the object to be observed under a specified observation direction is combined during the process of obtaining the target grayscale value of each pixel position in the grayscale image of the object to be observed. The magnitude of the initial normal vector of any pixel position can represent the diffuse reflection coefficient of the real location on the surface of the object to be observed represented by that pixel position. Therefore, the target grayscale value of the pixel position corresponding to the detailed structure of the surface of the object to be observed in the grayscale image can be increased, and the target grayscale value of the pixel position corresponding to the real location on the surface of the object to be observed that is prone to noise can be decreased. In this way, the detailed structure of the surface of the object to be observed in the grayscale image can be highlighted, and the noise in the grayscale image can be suppressed. That is, the grayscale image can effectively reflect the detailed structure of the surface of the object to be observed under a specified observation direction with low noise. Accordingly, based on the grayscale image, the detailed structure of the object to be observed under a specified observation direction can be effectively observed.

[0227] In one embodiment, the apparatus further includes:

[0228] The depth value determination module is used to determine the depth value of each pixel position in the depth image of the object to be observed, based on a preset energy function and the initial normal vector of each pixel position, before calculating the initial gray value of the pixel position based on the direction vector representing the specified observation direction and the initial normal vector of the pixel position.

[0229] The coordinate transformation module is used to transform the pixel coordinates and depth values ​​of each pixel position according to the pre-calibrated transformation coefficients between the pixel coordinate system and the world coordinate system, so as to obtain the world coordinates corresponding to each pixel position; wherein, the first coordinate axis of the world coordinate system is perpendicular to the plane to which the microscope's platform belongs;

[0230] The plane fitting module is used to perform plane fitting based on the world coordinates corresponding to each pixel position to obtain the plane to be corrected.

[0231] A rotation matrix calculation module is used to calculate the rotation matrix between the normal vector of the plane to be corrected and the direction of the first coordinate axis;

[0232] The grayscale value calculation module 704 is specifically used to calculate the product of the rotation matrix and the initial normal vector of the pixel position to obtain the normal vector to be used at the pixel position; and to calculate the product of the direction vector representing the specified observation direction and the normal vector to be used at the pixel position to obtain the initial grayscale value at the pixel position.

[0233] In one embodiment, the plane fitting module includes:

[0234] The coordinate selection submodule is used to randomly select a first number of world coordinates from the world coordinates corresponding to each pixel position;

[0235] The alternative plane fitting submodule is used to perform plane fitting based on the selected world coordinates to obtain the current alternative plane; and return to execute the step of randomly selecting a first number of world coordinates from the world coordinates corresponding to each pixel position until a preset number of alternative planes are fitted.

[0236] The point count acquisition module is used to acquire the number of interior points of each candidate plane, which is used as the number of points corresponding to each candidate plane; wherein, the interior point of any candidate plane represents the position represented by world coordinates at a distance less than a preset distance threshold from the candidate plane.

[0237] The plane determination submodule is used to determine the plane to be corrected based on the candidate plane with the largest number of corresponding points.

[0238] In one embodiment, the plane determination submodule is specifically used to perform plane fitting based on the world coordinates of the interior points of the candidate plane with the largest number of corresponding points to obtain the plane to be corrected.

[0239] In one embodiment, the preset energy function is as follows:

[0240] J(z)=∫∫((z u -p) 2 +(z v -q) 2 )dudv+λ(∑|z u |+|z v |)

[0241] Where J(z) represents the energy of the depth image; z u and z v These represent the gradients of the depth image along the two coordinate axes of the pixel coordinate system; p = n x / n z q = n y / n z ,(n x n y n z ) represents the unit normal vector at each pixel position; λ represents the regularization constraint strength coefficient.

[0242] In one embodiment, the illumination data determination module 702 is specifically used to calculate, for each preset light source, the illumination direction of each pixel position in the observed image under the illumination direction of the preset light source according to a first formula; wherein, the first formula is as follows:

[0243]

[0244] K(u,v) represents the illumination direction of pixel position (u,v); m represents the coordinates of the preset light source in the world coordinate system; p(u,v) represents the coordinates of the actual position on the surface of the object to be observed, represented by pixel position (u,v), in the world coordinate system.

[0245] When the specified magnification is greater than a preset magnification threshold, for each preset light source, the illumination intensity at each pixel position in the observed image under the illumination direction of that preset light source is calculated according to the second formula; wherein, the second formula is as follows:

[0246]

[0247] K(u,v) represents the illumination direction at pixel position (u,v); m represents the coordinates of the preset light source in the world coordinate system; p(u,v) represents the coordinates of the actual position on the surface of the object to be observed, represented by pixel position (u,v), in the world coordinate system; s(u,v) represents the illumination intensity at pixel position (u,v); Ψ represents the illumination intensity at the position of the preset light source; ns represents the principal direction of the preset light source.

[0248] When the specified magnification is not greater than the preset magnification threshold, for each preset light source, the observed image under the illumination direction of the preset light source is uniformly divided according to the grid matrix to obtain a second number of rectangular regions of equal size;

[0249] The illumination intensity at the actual position on the surface of the object to be observed, represented by the center of each rectangular region, is measured and taken as the illumination intensity corresponding to that rectangular region.

[0250] Based on the illumination intensity corresponding to each rectangular region obtained by division, linear interpolation is performed to obtain the illumination intensity of each pixel position in the observed image under the illumination direction of the preset light source.

[0251] In one embodiment, the grayscale mapping module 705 is specifically used to map the initial grayscale value of the pixel location to a grayscale range according to a mapping formula, thereby obtaining the target grayscale value of the pixel location in the grayscale image of the object to be observed under the specified observation direction; wherein, the mapping formula is as follows:

[0252]

[0253] G(u,v) represents the target grayscale value at pixel position (u,v) in the grayscale image of the object to be observed under the specified observation direction; G c (u,v) represents the initial grayscale value at pixel position (u,v); max(abs(G c )) represents the maximum absolute value among the initial grayscale values ​​at each pixel location.

[0254] In one embodiment, the initial normal vector calculation module 703 is specifically used to calculate the brightness value of each pixel position under the illumination direction of each preset light source, based on the pixel value of that pixel position in the observed image under the illumination direction of each preset light source.

[0255] According to the diffuse reflection formula, and combining the brightness value, illumination direction, and illumination intensity of the pixel location under the illumination direction of each preset light source, the product of the diffuse reflection coefficient and the unit normal vector representing the true position on the surface of the object to be observed at that pixel location is calculated to obtain the initial normal vector of that pixel location; wherein, the diffuse reflection formula is as follows:

[0256] I i (u,v)=s i (u,v)·l i (u,v)·N(u,v)

[0257] st

[0258] N(u,v)=ρ(u,v)·n(u,v)

[0259] I i (u,v) represents the brightness value of pixel position (u,v) under the illumination direction of the i-th preset light source; i (u,v), s i (u,v) represent the illumination direction and illumination intensity of pixel position (u,v) in the observed image under the illumination direction of the i-th preset light source, respectively; N(u,v) represents the initial normal vector of pixel position (u,v), ρ(u,v) represents the diffuse reflection coefficient of the true position on the surface of the object to be observed represented by pixel position (u,v), and n(u,v) represents the unit normal vector of the true position on the surface of the object to be observed represented by pixel position (u,v).

[0260] This application also provides an electronic device, such as... Figure 8 As shown, it includes:

[0261] Memory 801 is used to store computer programs;

[0262] When processor 802 executes a program stored in memory 801, it performs the following steps:

[0263] Acquire observation images of the object to be observed by the microscope at a specified magnification under illumination from at least three preset light sources;

[0264] For each preset light source, determine the illumination direction and intensity of each pixel in the observed image under the illumination direction of that preset light source;

[0265] For each pixel location, the initial normal vector is calculated by combining the pixel value, illumination direction, and illumination intensity of that pixel location in the observed image under the illumination direction of each preset light source. The magnitude of the initial normal vector at any pixel location represents the diffuse reflection coefficient of the true location on the surface of the object being observed, as represented by that pixel location. The direction of the initial normal vector at any pixel location represents the direction of the normal vector at the true location on the surface of the object being observed, as represented by that pixel location.

[0266] The initial grayscale value of the pixel is calculated based on the direction vector representing the specified observation direction and the initial normal vector of the pixel position.

[0267] The initial grayscale value of the pixel location is mapped to a grayscale range to obtain the target grayscale value of the pixel location in the grayscale image of the object to be observed under the specified observation direction.

[0268] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 802, the communication interface, and the memory 801 communicating with each other via the communication bus.

[0269] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0270] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0271] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0272] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0273] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described image generation methods.

[0274] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the image generation methods described above.

[0275] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0276] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0277] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0278] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. An image generation method, characterized in that, The method includes: Acquire observation images of the object to be observed by the microscope at a specified magnification under illumination from at least three preset light sources; For each preset light source, determine the illumination direction and intensity of each pixel in the observed image under the illumination direction of that preset light source; For each pixel location, the initial normal vector is calculated by combining the pixel value, illumination direction, and illumination intensity of that pixel location in the observed image under the illumination direction of each preset light source. The magnitude of the initial normal vector at any pixel location represents the diffuse reflection coefficient of the true location on the surface of the object being observed, as represented by that pixel location. The direction of the initial normal vector at any pixel location represents the direction of the normal vector at the true location on the surface of the object being observed, as represented by that pixel location. The initial grayscale value of the pixel is calculated based on the direction vector representing the specified observation direction and the initial normal vector of the pixel position. The initial grayscale value of the pixel location is mapped to a grayscale range to obtain the target grayscale value of the pixel location in the grayscale image of the object to be observed under the specified observation direction.

2. The method according to claim 1, characterized in that, Before calculating the initial grayscale value of the pixel position based on the direction vector representing the specified observation direction and the initial normal vector of the pixel position, the method further includes: Based on a preset energy function and the initial normal vector of each pixel position, the depth value of each pixel position in the depth image of the object to be observed is determined. According to the pre-defined transformation coefficients between the pixel coordinate system and the world coordinate system, the pixel coordinates and depth values ​​of each pixel position are transformed to obtain the world coordinates corresponding to each pixel position; wherein, the first coordinate axis of the world coordinate system is perpendicular to the plane to which the microscope's platform belongs; The plane to be corrected is obtained by performing plane fitting based on the world coordinates corresponding to each pixel position. Calculate the rotation matrix between the normal vector of the plane to be corrected and the direction of the first coordinate axis; The step of calculating the initial grayscale value of the pixel position based on the direction vector representing the specified observation direction and the initial normal vector of the pixel position includes: Calculate the product of the rotation matrix and the initial normal vector at the pixel position to obtain the normal vector to be used at the pixel position; The initial grayscale value of the pixel is obtained by multiplying the direction vector representing the specified observation direction with the normal vector to be used at the pixel location.

3. The method according to claim 2, characterized in that, The process of performing plane fitting based on the world coordinates corresponding to each pixel position to obtain the plane to be corrected includes: Randomly select the first number of world coordinates corresponding to each pixel position; The selected world coordinates are used to perform plane fitting to obtain the current candidate plane; and the process returns to the step of randomly selecting a first number of world coordinates from the world coordinates corresponding to each pixel position until a preset number of candidate planes are fitted. Obtain the number of interior points of each candidate plane, which is taken as the number of points corresponding to each candidate plane; wherein, the interior point of any candidate plane represents the position represented by world coordinates at a distance less than a preset distance threshold from the candidate plane. The plane to be corrected is determined based on the candidate plane with the largest number of corresponding points.

4. The method according to claim 3, characterized in that, The process of determining the plane to be corrected based on the candidate plane with the largest number of corresponding points includes: The plane to be corrected is obtained by fitting the world coordinates of the interior points of the candidate plane with the largest number of corresponding points.

5. The method according to claim 2, characterized in that, The preset energy function is as follows: J(z)=∫∫((z u -p) 2 +(with v -q) 2 )dudv+λ(∑|z u |+|with v |) Where J(z) represents the energy of the depth image; z u and z v These represent the gradients of the depth image along the two coordinate axes of the pixel coordinate system; p = n x / n z q = n y / n z , (n x n y n z ) represents the unit normal vector at each pixel position; λ represents the regularization constraint strength coefficient.

6. The method according to claim 1, characterized in that, The step of determining the illumination direction and intensity of each pixel in the observed image under the illumination direction of each preset light source includes: For each preset light source, the illumination direction of each pixel in the observed image under the illumination direction of that preset light source is calculated according to the first formula; wherein, the first formula is as follows: K(u,v) represents the illumination direction of pixel position (u,v); m represents the coordinates of the preset light source in the world coordinate system; p(u,v) represents the coordinates of the actual position on the surface of the object to be observed, represented by pixel position (u,v), in the world coordinate system. When the specified magnification is greater than a preset magnification threshold, for each preset light source, the illumination intensity at each pixel position in the observed image under the illumination direction of that preset light source is calculated according to the second formula; wherein, the second formula is as follows: K(u,v) represents the illumination direction at pixel position (u,v); m represents the coordinates of the preset light source in the world coordinate system; p(u,v) represents the coordinates of the actual position on the surface of the object to be observed, represented by pixel position (u,v), in the world coordinate system; s(u,v) represents the illumination intensity at pixel position (u,v); Ψ represents the illumination intensity at the position of the preset light source; n s Indicates the main direction of the preset light source; When the specified magnification is not greater than the preset magnification threshold, for each preset light source, the observed image under the illumination direction of the preset light source is uniformly divided according to the grid matrix to obtain a second number of rectangular regions of equal size; The illumination intensity at the actual position on the surface of the object to be observed, represented by the center of each rectangular region, is measured and taken as the illumination intensity corresponding to that rectangular region. Based on the illumination intensity corresponding to each rectangular region obtained by division, linear interpolation is performed to obtain the illumination intensity of each pixel position in the observed image under the illumination direction of the preset light source.

7. The method according to claim 1, characterized in that, The step of mapping the initial grayscale value of the pixel location to a grayscale range to obtain the target grayscale value of the pixel location in the grayscale image of the object to be observed under the specified observation direction includes: According to the mapping formula, the initial grayscale value of the pixel position is mapped to a grayscale value range to obtain the target grayscale value of the pixel position in the grayscale image of the object to be observed under the specified observation direction; wherein, the mapping formula is as follows: G(u,v) represents the target grayscale value at pixel position (u,v) in the grayscale image of the object to be observed under the specified observation direction; G c (u,v) represents the initial grayscale value at pixel position (u,v); max(abs(G c )) represents the maximum absolute value among the initial grayscale values ​​at each pixel location.

8. The method according to claim 1, characterized in that, For each pixel location, the initial normal vector for that pixel location is calculated by combining the pixel value, illumination direction, and illumination intensity of the observed image under the illumination direction of each preset light source. This includes: For each pixel location, based on the pixel value of that pixel location in the observed image under the illumination direction of each preset light source, calculate the brightness value of that pixel location under the illumination direction of each preset light source. According to the diffuse reflection formula, and combining the brightness value, illumination direction, and illumination intensity of the pixel location under the illumination direction of each preset light source, the product of the diffuse reflection coefficient and the unit normal vector representing the true position on the surface of the object to be observed at that pixel location is calculated to obtain the initial normal vector of that pixel location; wherein, the diffuse reflection formula is as follows: I i (u,v)=s i (u,v)·l i (u,v)·N(u,v) st N(u,v)=ρ(u,v)·n(u,v) I i (u,v) represents the brightness value of pixel position (u,v) under the illumination direction of the i-th preset light source; i (u,v), s i (u,v) represent the illumination direction and illumination intensity of pixel position (u,v) in the observed image under the illumination direction of the i-th preset light source, respectively; N(u,v) represents the initial normal vector of pixel position (u,v), ρ(u,v) represents the diffuse reflection coefficient of the true position on the surface of the object to be observed represented by pixel position (u,v), and n(u,v) represents the unit normal vector of the true position on the surface of the object to be observed represented by pixel position (u,v).

9. A microscope apparatus, characterized in that, The device includes: a microscope and at least three preset light sources, wherein the microscope is equipped with an image sensor and a processor; The image sensor is used to acquire observation images of the object to be observed under the illumination directions of the at least three preset light sources at a specified magnification, and to send the acquired observation images to the processor. The processor is configured to execute the method according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-8.

Images