Computer system based on electron microscope image simulation calculation model

Abstract

The invention relates to the field of computer systems based on specific calculation models, in particular to the field of computer systems based on electron microscope image simulation calculation models. The present disclosure describes a computer system based on an electron microscope image simulation calculation model, the computer system comprising: an electron microscope image simulation calculation model; an analog display unit; the computer system provides an operation mode and an output display which are the same as those of a prototype electron microscope, the prototype electron microscope comprises a prototype control part and a prototype display part, and the simulation control part and the prototype control part are the same in number and type of operable control devices. The operable control device in the simulation control section is operated such that a change generated by the image in the simulation display section is the same as a change generated by operating the operable control device in the prototype control section such that the image in the prototype display section is the same as a change generated by operating the operable control device in the prototype control section.

Description

Technical Field

[0001] The present disclosure relates to the field of computer systems based on specific computing models, and in particular to the field of computer systems based on electron microscope image simulation computing models. The computer system together with the simulation control unit, i.e., the control device, contained therein can also be referred to as or used as an electron microscope simulation training device. The device simulates all operations of a prototype scientific instrument, and gives the same operation results and graphic feedback display as the prototype instrument, while the cost is only about one ten-thousandth of the prototype instrument, and is solid, safe and reliable, unlike the prototype instrument which is delicate and fragile, and provides a reliable tool for training expensive advanced precision instruments through a computer system. Background Art

[0002] Electron microscopes, as cutting-edge tools for microscopic observation and analysis, are an indispensable method in scientific research and theoretical teaching. Through electron microscopes, researchers can explore the microstructure of materials and understand the properties of materials at an unprecedented level of detail. In the process of electron microscope teaching, it is not limited to the imparting of theoretical knowledge, but also focuses on the cultivation of practical operation ability.

[0003] Electron microscope equipment is expensive, and practical training can easily damage the equipment, and it takes up already tight scientific research work time, resulting in high training costs. Therefore, it is necessary to conduct training on the operation of the electron microscope before use. Chinese invention patent CN104537903B discloses an interactive transmission electron microscope virtual simulation teaching system, in which "the user is prompted with operation steps and precautions through text and voice, and the user performs operations on the input module according to the prompt information. The video output module outputs the corresponding operation results of the transmission electron microscope three-dimensional model. When the user makes an error in operation, the video output module and the audio output module respectively give error alarm prompts." Yan Zhen, Yang Jun, Peng Rui, Li Chenghua, Yi Peishan, Lin Honghui, Zhang Dawei, and Liu Huanhuan in "Construction and Application of Virtual Simulation Experiments of Electron Microscopes [J]; Chinese Journal of Cell Biology; Issue 10, 2022", and Wang Kunlun in "Construction and Application of Virtual Simulation Experiment Teaching System of Scanning Electron Microscopes [J]; Science and Technology Style; Issue 30, 2023" also disclosed similar virtual simulation experiment systems, in which "the instrument operation process needs to be strictly followed in steps." The above virtual simulation experiment system mainly relies on the training method of the linear virtual simulation teaching system. This linear virtual simulation teaching system usually includes a user operation interface, an instrument operation guide tutorial, a three-dimensional model of an electron microscope, and a database of fixed operation steps. The system prompts the user with operation steps and precautions through text and voice. The user performs the pre-made operation on the user operation interface according to the prompt. When the user makes an error in operation, the system gives an error alarm prompt. After the user completes all the pre-made steps, the system gives an operation evaluation. In this linear virtual simulation training, the user cannot master all the operation points of the electron microscope operation process, and cannot master the operation experience of quickly handling various problems in the operation process. This virtual simulation training that is separated from the actual operation cannot meet the user's operation training needs for the electron microscope. In addition, these virtual simulation experiments can only perform simple step-by-step process operations through the mouse and keyboard, and do not have the complex free control and adjustment functions in the process of electron microscope image generation and adjustment, and cannot achieve real operation training. In order to achieve the same free operation under multiple degrees of freedom as the real electron microscope operation panel, it is obviously unrealistic to record and save all the changes in electron microscope photos corresponding to different operations in the real electron microscope in the database and retrieve them one by one, because the amount of data required to be saved will increase exponentially. Therefore, it is necessary to develop a specific calculation model, so that starting from a limited number of electron microscope photos, the photo effects that can be obtained by the real electron microscope under other circumstances can be given through model calculation, and then coupled with the corresponding operation and control elements.In this specific computational model, it is necessary to simulate all operations of the electron microscope and give the same operation results and graphic feedback display as the electron microscope. The technical difficulty lies in how to realize the complex operation principle algorithm of the simulated prototype electron microscope system, including: the electron microscope adjusts the brightness, contrast, magnification and focus of the image by precisely controlling the focusing, scanning and signal processing of the electron beam to ensure high-quality imaging. The electron beam is accelerated and focused to scan the sample surface, and the excited secondary electrons and other signals are amplified to modulate the brightness of the picture tube to achieve brightness adjustment, while the contrast is affected by multiple factors such as scanning speed, signal quality, sample characteristics and post-processing software. The magnification is controlled by adjusting the electron beam focusing degree, scanning range and electron optical system parameters, and the correction magnetic field is introduced to eliminate astigmatism, and precise focus is achieved through coarse focusing, fine focusing and destigmatization steps, and finally a clear and high-resolution image is generated. Because the magnification range of the electron microscope is huge, its photos can show the image of the atomic-scale microscopic world, and the amount of data and calculations brought about are also incalculable. Therefore, it is necessary to simplify the algorithm model as much as possible under the condition of acceptable computing power, so as to develop a computer system based on the electron microscope image simulation computing model, in which the simplified algorithm model is combined with a physical controller that completely corresponds to a real electron microscope, so as to simulate the operation process and photo display process of electron microscope photography. This solution is the computer system based on the electron microscope image simulation computing model provided by the present disclosure. Summary of the invention

[0004] In order to solve the above problems, the present invention provides a computer system based on an electron microscope image simulation computing model, which is composed of a new special computer system from two aspects: computing model software and computer system special control hardware, to simulate the actual operation process of an electron microscope. Users can operate the computer system based on the electron microscope image simulation computing model of the present invention in the same way as they operate a real electron microscope, and simplify the algorithm for simulating the electron microscope photography process, while reducing the cost of electron microscope actual operation training and improving the training effect.

[0005] The present disclosure provides a computer system based on an electron microscope image simulation computing model, the computer system comprising: Computational models for electron microscope image simulation; an analog display unit; and Analog control unit, The computer system provides the same operation mode and output display as those of the prototype electron microscope, wherein the prototype electron microscope includes a prototype control unit and a prototype display unit, the simulation control unit and the prototype control unit have the same number and types of operable control devices, and operating the operable control devices in the simulation control unit causes the image in the simulation display unit to produce the same changes as operating the operable control devices in the prototype control unit.

[0006] The computer system of the present disclosure also includes standard hardware and software configurations included in traditional computer systems, such as keyboards, mice, hard disks, transmission lines, etc., because these parts are all conventional technical means in the field, and will not be repeated in this disclosure. In the present disclosure, the parts of the computer system based on the electron microscope image simulation calculation model of the present disclosure that are different from the traditional computer system are mainly described, including the electron microscope image simulation calculation model, the simulation control unit corresponding to the actual electron microscope, and the simulation display unit. In addition to including conventional display device hardware, the simulation display unit also includes the process of processing, rendering and performing other conventional processing on the results calculated by the electron microscope image simulation calculation model, wherein the conventional display device hardware includes displays, especially liquid crystal displays, VR helmets, XR glasses, MR glasses, large screens, ring screens, three-dimensional stereo displays, naked-eye 3D display devices, panoramic CAVE systems, desktop XR or naked-eye 3D display stations and other display output devices.

[0007] In one embodiment, the operable control device in the simulation control unit includes a switching button, which can switch between high-magnification mode and low-magnification mode. When the mode is switched by the switching button, the electron microscope image simulation calculation model generates an image through a multi-level mapping texture algorithm, thereby achieving the same change in the image in the simulation display unit as that in the prototype display unit.

[0008] In one embodiment, the operable control device in the simulation control unit includes a magnification knob. When the magnification knob is rotated, the electron microscope image simulation calculation model calculates the distance between the simulated camera and the simulated sample and generates an image through a multi-level mapping texture algorithm, thereby achieving the same scaling change of the image in the simulation display unit as that in the prototype display unit.

[0009] In one embodiment, the operable control device in the simulation control section includes an X-direction position knob and a Y-direction position knob. When the X-direction position knob and / or the Y-direction position knob are rotated, the electron microscope image simulation calculation model simulates and calculates the coordinate position of the electron beam in the three-dimensional scene through a coordinate conversion formula, thereby achieving the same movement change of the image in the simulation display section and the image in the prototype display section.

[0010] In one embodiment, the operable control device in the simulation control unit includes a trackball and a trackball lock. When the trackball is rotated, the electron microscope image simulation calculation model simulates and calculates the coordinate position of the sample in the three-dimensional scene through a coordinate conversion formula, thereby achieving the same movement change of the image in the simulation display unit as that in the prototype display unit; the trackball lock can lock or unlock the trackball so that rotating the trackball produces no or no operational effect.

[0011] In one embodiment, the operable control device in the simulation control section includes a brightness knob. When the brightness knob is rotated, the electron microscope image simulation calculation model simulates and calculates the signal intensity of the electron beam incident on the sample surface through a light and dark contrast algorithm, thereby achieving the same light and dark changes in the image in the simulation display section as in the image in the prototype display section.

[0012] In one embodiment, the operable control device in the simulation control section includes a contrast knob. When the contrast knob is rotated, the electron microscope image simulation calculation model simulates and calculates the scanning speed of the electron beam, the resolution of the electron beam signal and the signal-to-noise ratio through a light-dark contrast algorithm, thereby achieving the same contrast change between the image in the simulation display section and the image in the prototype display section.

[0013] In one embodiment, the operable control device in the simulation control unit also includes a brightness and contrast automatic adjustment button. When the brightness and contrast are automatically adjusted by the brightness and contrast automatic adjustment button, the electron microscope image simulation calculation model simulates and calculates the signal intensity of the electron beam emitted to the sample surface through a light and dark contrast algorithm, and simulates and calculates the scanning speed of the electron beam, the resolution of the electron beam signal and the signal-to-noise ratio through a light and dark contrast algorithm, thereby achieving the same brightness and contrast changes of the image in the simulation display unit and the image in the prototype display unit.

[0014] In one embodiment, the operable control device in the analog control unit includes an astigmatism and correction mode selection button, an astigmatism mode indicator light and a correction mode indicator light. When the astigmatism mode is selected by the astigmatism and correction mode selection button, the astigmatism mode indicator light turns on and the astigmatism mode is entered. When the correction mode is selected by the astigmatism and correction mode selection button, the correction mode indicator light turns on and the correction mode is entered.

[0015] In one embodiment, the operable control device in the simulation control section includes an X-direction astigmatism and correction knob and a Y-direction astigmatism and correction knob. When the X-direction astigmatism and correction knob and / or the Y-direction astigmatism and correction knob are rotated, the electron microscope image simulation calculation model simulates and calculates the electron beam focusing ability through an astigmatism and focusing algorithm and a noise generation algorithm, thereby achieving the same astigmatism and correction changes in the image in the simulation display section and the image in the prototype display section in the astigmatism mode and the correction mode, respectively.

[0016] In one embodiment, the operable control device in the simulation control unit includes a coarse focusing knob and a fine focusing knob. When the coarse focusing knob and / or the fine focusing knob are rotated, the electron microscope image simulation calculation model simulates the ability of the electromagnetic lens to focus the electron beam through an astigmatism and focusing algorithm and a noise generation algorithm, thereby achieving the same focusing change of the image in the simulation display unit as that in the prototype display unit.

[0017] In one embodiment, the operable control device in the simulation control unit includes a first-gear scanning speed button, a second-gear scanning speed button, a third-gear scanning speed button, a fourth-gear scanning speed button, an image saving button and a photo taking button. When the first-gear scanning speed button, the second-gear scanning speed button, the third-gear scanning speed button or the fourth-gear scanning speed button and the photo taking button and / or the image saving button are pressed, the electron microscope image simulation calculation model simulates and calculates the scanning speed of the electron beam and the image photographing and saving effect in the three-dimensional scene through a scanning and photographing algorithm, thereby realizing scanning and photographing of the image in the simulation display unit that is the same as the image in the prototype display unit.

[0018] In one embodiment, the low magnification mode is 20 to 2000 times, and the high magnification mode is 500 to 1 million times.

[0019] In one embodiment, in the multi-level map texture algorithm, , Among them, N tis the normal vector in tangent space, N is the normal vector in model space, T is the tangent vector in model space, B is the bitangent vector in model space, B=T×N, T, B and N are perpendicular to each other, where: , Among them, T x 、T y and T z are the x, y, and z components of the tangent vector T, respectively, and B x , B y and B z are the x, y and z components of the bitangent vector B, N x 、N y and N z are the x, y and z components of the normal vector N, respectively, where the model space is the coordinate system that defines and stores the geometric data of the sample model, and the bitangent vector is perpendicular to the tangent vector and the normal vector.

[0020] In one embodiment, in the multi-level map texture algorithm, , Among them, I 镜面 is the color intensity of the specular highlight component, I s is the specular light intensity of the light source, K s is the mirror reflection coefficient of the object surface, H is the half-angle vector used to approximate the calculation of the mirror highlight effect, N is the normal vector of the object surface, dot is the dot product, pow is the power function, and max is the function of the maximum value of the elements in the brackets that follow it.

[0021] In one embodiment, the calculation formula of the half-angle vector H is: , Wherein, L is the incident direction vector of the light, V is the observer direction vector, and the normalized (L+V) is the normalized function of (L+V).

[0022] In one embodiment, the noise generation algorithm disturbs the source image through an irregular noise image, and uses the average value of pixel values ​​at the same position of the two images as the pixel information of the corresponding position of the generated image.

[0023] In one embodiment, in the light-dark contrast algorithm, the main texture color is sampled, the brightness is adjusted, the brightness value is calculated, the grayscale is mixed with the original color to adjust the saturation, the grayscale is mixed with the final color to adjust the contrast, and the final color and the original transparency are returned.

[0024] In one embodiment, in the astigmatism and focusing algorithm, the position colors of four different uv coordinates are sampled from the main texture, different weights are applied to these colors for mixing, and the final mixed color is returned as the pixel color output.

[0025] In one embodiment, in the translation and magnification algorithm, , Where x and y are the coordinates before translation, x' and y' are the coordinates after translation, and Δx and Δy are the translation distances in the x and y directions.

[0026] In one embodiment, .

[0027] In one embodiment, in the scanning and photographing algorithm, image data is selectively processed according to the scanning mode, a temporary texture buffer is created, the threshold is adjusted, material parameters are set: diffuse reflection, reflection, specular glossiness, reflection glossiness, bump map, subdivision, transparency, refraction, and the contents of the source buffer are rendered to the target buffer, and finally the buffer reference is updated.

[0028] The present disclosure provides a computer system based on an electron microscope image simulation computing model, and the specific technical solution is as follows: The overall system includes a simulation calculation model based on electron microscope images and a control system; The electron microscope image simulation computing model includes a display unit and an algorithm library; The algorithm library includes a multi-level mapping texture algorithm, a noise generation algorithm, a light and dark contrast algorithm, an astigmatism and focusing algorithm, and a translation magnification algorithm, and a scanning and photographing algorithm; The control unit system includes a communication unit and a control unit; The control unit includes a trackball control unit, a knob control unit, a switch control unit and an indicator light display; The control unit exchanges data with the electron microscope image simulation calculation model through the communication unit and serial port message transmission and reception, realizes the setting of each algorithm parameter in the algorithm library, and displays the image changes in real time in the display unit; The input signal of the button control part of the control part switches the high and low magnification modes, and the input signal of the knob control part of the control part simulates and controls the parameters of the electronic optical system, generates an image through a multi-level mapping texture algorithm, and calculates the distance between the simulated camera and the simulated sample, and the calculation result is transmitted to the display part to realize the image zoom function; The input signal of the knob control part of the control part simulates and controls the coordinate position of the electron beam in the three-dimensional scene, and is calculated through the coordinate conversion formula, and the calculation result is transmitted to the display part to realize the simulated electron beam movement function; The input signal of the control part trackball control part simulates the coordinate position of the sample in the three-dimensional scene, and the coordinate conversion formula is used to calculate, and the calculation result is transmitted to the display part to realize the simulated sample image movement function; The input signal of the knob control part of the control part simulates and controls the signal intensity of the electron beam irradiating the sample surface, and is calculated by the light-dark contrast algorithm, and the calculation result is transmitted to the display part to realize the simulated brightness adjustment function; The input signal of the knob control part of the control part simulates and controls the scanning speed of the electron beam, the resolution and signal-to-noise ratio of the electron beam signal, and is calculated through a light-dark contrast algorithm, and the calculation result is transmitted to the display part to realize the simulated contrast adjustment function; The input signal of the knob control part of the control part simulates and controls the focusing ability of the electron beam, and is calculated through the astigmatism and focusing algorithm and the noise generation algorithm, and the calculation result is transmitted to the display part to realize the simulated astigmatism adjustment function; The input signal of the knob control part of the control part simulates the ability of regulating the electromagnetic lens to focus the electron beam, and is calculated through the astigmatism and focusing algorithm and the noise generation algorithm, and the calculation result is transmitted to the display part to realize the simulated quasi-focus adjustment function; According to the input signal of the button control part of the control part, the scanning speed of the electron beam in the three-dimensional scene is simulated and controlled, and the calculation is performed through the scanning and photographing algorithm, and the calculation result is transmitted to the display part to realize the simulated scanning and photographing function; In the multi-level mapping texture algorithm, , Among them, N t is the normal vector in tangent space, N is the normal vector in model space, T is the tangent vector in model space, B is the bitangent vector in model space, B=T×N, T, B and N are perpendicular to each other, where: , Among them, T x 、T y and T z are the x, y, and z components of the tangent vector T, respectively, and B x , B y and B z are the x, y and z components of the bitangent vector B, N x 、N y and N z are the x, y and z components of the normal vector N, respectively, where the model space is the coordinate system that defines and stores the geometric data of the sample model, and the bitangent vector is perpendicular to the tangent vector and the normal vector; Among them, in the multi-level mapping texture algorithm, , Among them, I 镜面is the color intensity of the specular highlight component, I s is the specular light intensity of the light source, K s is the mirror reflection coefficient of the object surface, H is the half-angle vector used to approximate the calculation of the mirror highlight effect, N is the normal vector of the object surface, dot is for the dot product, pow is for the power function, and max is the function of the maximum value of the elements in the brackets that follow it; The calculation formula of the half-angle vector H is: , Where L is the incident direction vector of the light, V is the observer direction vector, and normalized (L+V) is the normalized function of (L+V); The noise generation algorithm is implemented by disturbing the source image through an irregular noise image. The specific method is: taking the average of the pixel values ​​at the same position of the two images as the pixel information at the corresponding position of the generated image.

[0029] The implementation process of the brightness and darkness contrast algorithm is: sampling the main texture color, adjusting the brightness, calculating the brightness value, mixing grayscale with the original color to adjust the saturation, mixing grayscale with the final color to adjust the contrast, and returning the final color and the original transparency.

[0030] The implementation process of the astigmatism and focusing algorithm in the algorithm library is: sampling the position colors of four different uv coordinates from the main texture, applying different weights to these colors for mixing, and returning the final mixed color as the pixel color output.

[0031] The formula of the translation and amplification algorithm is: Translation formula: , Where x, y, x', y' are the coordinates before and after the translation respectively.

[0032] Magnification formula: , The implementation process of the scanning and photographing algorithm is: selectively process image data according to the scanning mode, create a temporary texture buffer, adjust the threshold, set material parameters: diffuse reflection, reflection, highlight glossiness, reflection glossiness, bump map, subdivision, transparency, refraction, and render the contents of the source buffer to the target buffer, and finally update the buffer reference.

[0033] The control unit exchanges data with the electron microscope image simulation calculation model through the communication unit and serial port message transmission and reception.

[0034] Compared with the practical training of electron microscope equipment and linear virtual simulation training, the computer system based on the electron microscope image simulation calculation model has a simpler simulation algorithm. The scene rendering function of the display unit is used to realize the display of the electron microscope sample image. The simulation effect is closer to the real physical simulation. The simulation data can support the physical simulation in the real environment, and it is repeatable and low-cost. The main algorithms include: multi-level texture algorithm, noise generation algorithm, light and dark contrast algorithm, astigmatism and focusing algorithm, translation magnification algorithm, scanning and photography algorithm. At the same time, by establishing a communication unit based on the electron microscope image simulation calculation model and the control unit system, using UART-USB conversion technology as the communication method for data transmission, and using a self-agreed optimization scheme for the data transmission format, the efficiency and real-time nature of the information interaction between the control unit system and the electron microscope image simulation calculation model are realized. In this way, the practical operation process of the electron microscope is fully simulated in both software and hardware, reducing the cost of practical training for the electron microscope and improving the effect of electron microscope training. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings are only examples of an embodiment and cannot be regarded as all necessary conditions of the present invention. In the accompanying drawings: Figure 1 is a comparison diagram of control devices operable by the simulation control unit and the prototype control unit of the present disclosure; Figure 2 It is a schematic diagram of the overall system architecture of the present disclosure; Figure 3 This is a simulation effect diagram of the multi-level mapping texture algorithm imaging disclosed in the present invention; Figure 4 This is a simulation effect diagram of the noise generation algorithm of the present disclosure; Figure 5 This is a simulation effect diagram of the brightness and darkness contrast algorithm imaging disclosed in the present invention; Figure 6 This is a simulation effect diagram of the astigmatism and focusing algorithm imaging disclosed in the present invention; Figure 7 This is a simulation effect diagram of the imaging of the translation and magnification algorithm disclosed in the present invention.

[0036] Figure 1Middle: 001 is the trackball lock; 002 is the trackball; 003 is the X-direction position knob; 004 is the Y-direction position knob; 005 is the high and low magnification mode switching button; 006 is the magnification switching knob; 007 is the first-gear scanning speed button; 008 is the second-gear scanning speed button; 009 is the third-gear scanning speed button; 010 is the fourth-gear scanning speed button; 011 is the image saving button; 012 is the photo taking button; 013 is the astigmatism mode indicator; 014 is the correction mode indicator; 015 is the astigmatism and correction mode selection button; 016 is the X-direction astigmatism and correction knob; 017 is the Y-direction astigmatism and correction knob; 018 is the brightness and contrast automatic adjustment button; 019 is the brightness knob; 020 is the contrast knob; 021 is the coarse focus knob; 022 is the fine focus knob. DETAILED DESCRIPTION

[0037] The following description clearly and completely describes the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0038] In the description of the embodiments of the present disclosure, it should be noted that the indicated orientation or position relationship is based on the orientation or position relationship shown in the accompanying drawings, or is the orientation or position relationship in which the disclosed product is usually placed when used, or is the orientation or position relationship commonly understood by those skilled in the art, or is the orientation or position relationship in which the disclosed product is usually placed when used. This is only for the convenience of describing the present disclosure and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present disclosure.

[0039] In the description of the embodiments of the present disclosure, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection, a direct connection, or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present disclosure can be understood according to specific circumstances. Example

[0040] [Example 1] Embodiment 1 of the present disclosure discloses a computer system based on an electron microscope image simulation computing model. The electron microscope image simulation computing model is implemented using the Unity engine, and the simulation control unit is implemented using 3D printing technology and an STM32 microcontroller chip. The simulation control unit communicates with the PC-side Unity engine via USB3.0, is compatible with a Type-C data cable, and links the Unity engine to display image changes on a computer monitor.

[0041] like Figure 1 As shown, the number and types of operable control devices in the simulation control unit and the prototype control unit are the same, including: 001, trackball lock; 002, trackball; 003, X-direction position knob; 004, Y-direction position knob; 005, high and low magnification mode switching button; 006, magnification switching knob; 007, first-gear scanning speed button; 008, second-gear scanning speed button; 009, third-gear scanning speed button; 010, fourth-gear scanning speed button; 011, image saving button; 012, photo taking button; 013, astigmatism mode indicator light; 014, correction mode indicator light; 015, astigmatism and correction mode selection button; 016, X-direction astigmatism and correction knob; 017, Y-direction astigmatism and correction knob; 018, brightness and contrast automatic adjustment button; 019, brightness knob; 020, contrast knob; 021, coarse focus knob; 022, fine focus knob.

[0042] like Figure 2 As shown, in the computer system based on the electron microscope image simulation calculation model, the electron microscope image simulation calculation model includes the following algorithms: multi-level mapping texture algorithm, noise generation algorithm, light and dark contrast algorithm, astigmatism and focusing algorithm and translation magnification algorithm, scanning and photography algorithm.

[0043] The simulation control unit exchanges data with the Unity engine through the controller, communication middleware, and serial port message transmission and reception, realizes the parameter setting of the post-rendering processing of the visualization window, and displays the indicator signal sent by the electron microscope image simulation calculation model through the serial port message, so as to realize that the changes of the image in the simulation display unit caused by operating the operable control device in the simulation control unit are the same as the changes of the image in the prototype display unit caused by operating the operable control device in the prototype control unit. The specific process is as follows: the simulation control unit sends the device model information in a loop until it receives the Unity engine connection confirmation message; the serial port manager is turned on, and polls the serial port with a separate thread to find the simulation control unit with a matching model; after the model is matched, the serial port manager starts the waiting thread for receiving messages and suspends the serial port polling thread; the serial port manager sends a connection confirmation message to the matched simulation control unit at regular intervals to maintain the connection; after receiving the connection confirmation message, the simulation control unit reports the status of each of its own controllers; after receiving the status information of the simulation control unit, the serial port manager sets the event processing flag; after the main loop of the electron microscope image simulation calculation model processes the event, it sends a message with a separate thread to update the status of the indicator light on the simulation control unit. In this process: The mathematical formula for calculating the tangent space normal vector in the multi-level mapping texture algorithm is: In the multi-level mapping texture algorithm, , Among them, N t is the normal vector in tangent space, N is the normal vector in model space, T is the tangent vector in model space, B is the bitangent vector in model space, B=T×N, T, B and N are perpendicular to each other, where: , Among them, T x 、T y and T z are the x, y, and z components of the tangent vector T, respectively, and B x , B y and B z are the x, y and z components of the bitangent vector B, N x 、N y and N z are the x, y and z components of the normal vector N, respectively, where the model space is the coordinate system that defines and stores the geometric data of the sample model, and the bitangent vector is perpendicular to the tangent vector and the normal vector; Among them, in the multi-level mapping texture algorithm, , Among them, I 镜面 is the color intensity of the specular highlight component, I s is the specular light intensity of the light source, K sis the mirror reflection coefficient of the object surface, H is the half-angle vector used to approximate the calculation of the mirror highlight effect, N is the normal vector of the object surface, dot is for the dot product, pow is for the power function, and max is the function of the maximum value of the elements in the brackets that follow it; The calculation formula of the half-angle vector H is: , Where L is the incident direction vector of the light, V is the observer direction vector, and normalized (L+V) is the normalized function of (L+V); Figure 3 It is an imaging simulation effect after a multi-level mapping texture algorithm.

[0044] The specific implementation of the noise generation algorithm is: use a noise texture, calculate the mean of each point in the image and this noise map, and realize the noise disturbance effect of the image. In the mean calculation, the larger the weight of the noise map, the greater the noise disturbance of the image, which is used to simulate the imaging quality of the electron microscope at different scanning speeds. In this embodiment, the brightness of a single pixel on the target image is calculated by the following formula: Target image brightness = weight factor × noise image brightness + (1-weight factor) × source image brightness, The weight factor represents the weight of gray, and its value is no greater than 1. The larger the gray weight, the grayer the image.

[0045] In this embodiment, the image is stored in the computer in the form of a brightness matrix, and is accessed through the uv coordinates during post-processing. For example, an image with a resolution of 1080×720 has a white center area, that is, the brightness of the pixels in the 540th column and 360th row of the image is 100%, and the grayscale of the color searched by the uv coordinate (0.5, 0.5) is 1. When post-processing the image, the brightness of the pixels at different uv coordinates is reassigned. The noise map is constantly changing, so the uv coordinates are also changing at any time. This embodiment simulates this change through the following formula: The u coordinate after the change = the u coordinate before the change + a random number between 0 and 1, The v coordinate after the change = the v coordinate before the change + the remainder of time divided by 200 × the remainder of time divided by 200, In the uv coordinate system, the horizontal direction is called u and the vertical direction is called v.

[0046] Figure 4 Simulates the effect of imaging through a noise generation algorithm.

[0047] In this embodiment, the specific implementation process of the light-dark contrast algorithm is: taking the average value of the brightness of each pixel of the source image and the pure gray to adjust the contrast. The more the image is biased toward gray, the lower the contrast. The contrast is controlled by adjusting the weight of gray in the weighted average. The specific implementation formula is as follows: Target image brightness = (1-weight factor) × source image brightness + weight factor × 0.5, The weight factor represents the weight of gray, and its value is no greater than 1. The larger the gray weight, the grayer the image.

[0048] Figure 5 This is a simulation effect of imaging through a light-dark contrast algorithm.

[0049] In this embodiment, astigmatism and focusing are calculated by the following formula: , Among them, η 中心 represents the center weight of the point to be blurred in the source image, η 左 , η 右 , η 上 , η represents the weights of the four points on the left, right, top and bottom. These weights are multiplied by the brightness of the center point and its nearby points and the sum is the average brightness. C(u-Δ,v) represents the brightness of the point slightly to the left of the center point, Δ represents the distance to the left. For the case of defocus, the weights and the distances of the sampling points are symmetrically distributed. When astigmatism occurs, asymmetric weights or unequal sampling distances are used for simulation. When unequal sampling distances are used for simulation, the sum of the distances is a fixed value, and its direction angle 𝜃 rotates with the rotation of the focus knob. At this time, Δ is recalculated. x =r* θ and Δ y =r* θ For the case of misalignment, fix the above (r, θ ) θ , and let r change periodically with time, that is, .

[0050] Figure 6 This is the imaging simulation effect after astigmatism and focusing algorithm.

[0051] In this embodiment, the specific formula for implementing the translation and magnification algorithm is as follows: Translation formula: , Where x, y, x', y' are the coordinates before and after the translation respectively.

[0052] Magnification formula: , The specific implementation process of the scanning and photographing algorithm is: selectively process image data according to the scanning mode, create a temporary texture buffer, adjust the threshold, set material parameters: diffuse reflection, reflection, highlight glossiness, reflection glossiness, bump map, subdivision, transparency, refraction, and render the contents of the source buffer to the target buffer, and finally update the buffer reference.

[0053] Figure 7 This is the imaging simulation effect of the translation and magnification algorithm.

[0054] When the control unit does not receive a connection confirmation message from the electron microscope image simulation computing model for a period of time, the control unit stops sending its own status information and restarts to send model information in a loop to wait for the electron microscope image simulation computing model to be reopened; When the electron microscope image-based simulation computing model captures a "disconnected" related exception when attempting to receive a serial port message, a disconnection event flag is set, and serial port polling is restarted to wait for the control unit to reconnect.

[0055] The present disclosure is not limited to the aforementioned specific embodiments, but extends to any new features or any new combination disclosed in this specification, as well as any new method or process steps or any new combination disclosed.

Claims

1. A computer system based on an electron microscope image simulation computing model, the computer system comprising: Computational models for electron microscope image simulation; an analog display unit; and Analog control unit, The computer system provides the same operation mode and output display as those of the prototype electron microscope, wherein the prototype electron microscope includes a prototype control unit and a prototype display unit, the number and types of operable control devices in the simulation control unit are the same as those in the prototype control unit, and operating the operable control devices in the simulation control unit causes the same changes in the image in the simulation display unit as operating the operable control devices in the prototype control unit causes the same changes in the image in the prototype display unit.

2. The computer system based on the electron microscope image simulation computing model as described in claim 1, wherein the operable control device in the simulation control unit includes a switching button, and the switching button can switch between a high-magnification mode and a low-magnification mode. When the mode is switched by the switching button, the electron microscope image simulation computing model generates an image through a multi-level mapping texture algorithm, thereby achieving the same change in the image in the simulation display unit as that in the prototype display unit.

3. The computer system based on the electron microscope image simulation computing model as described in claim 1, wherein the operable control device in the simulation control unit includes a magnification knob. When the magnification knob is rotated, the electron microscope image simulation computing model calculates the distance between the simulated camera and the simulated sample and generates an image through a multi-level mapping texture algorithm, thereby achieving the same scaling change of the image in the simulation display unit and the image in the prototype display unit.

4. The computer system based on the electron microscope image simulation calculation model as described in claim 1, wherein the operable control device in the simulation control unit includes an X-direction position knob and a Y-direction position knob, and when the X-direction position knob and / or the Y-direction position knob are rotated, the electron microscope image simulation calculation model simulates and calculates the coordinate position of the electron beam in the three-dimensional scene through a coordinate conversion formula, thereby achieving the same movement change of the image in the simulation display unit and the image in the prototype display unit.

5. The computer system based on the electron microscope image simulation calculation model as described in claim 1, wherein the operable control device in the simulation control unit includes a trackball and a trackball lock, and when the trackball is rotated, the electron microscope image simulation calculation model simulates and calculates the coordinate position of the sample in the three-dimensional scene through a coordinate conversion formula, thereby achieving the same movement change of the image in the simulation display unit and the image in the prototype display unit; the trackball lock can lock or unlock the trackball, so that rotating the trackball has no or produces no operation effect.

6. The computer system based on the electron microscope image simulation calculation model as described in claim 1, The operable control device in the simulation control unit includes a brightness knob. When the brightness knob is rotated, the electron microscope image simulation calculation model simulates and calculates the signal intensity of the electron beam irradiating the sample surface through a light-dark contrast algorithm, thereby achieving the same light-dark change of the image in the simulation display unit and the image in the prototype display unit; and / or wherein the operable control device in the simulation control section comprises a contrast knob, and when the contrast knob is rotated, the electron microscope image simulation calculation model simulates and calculates the scanning speed of the electron beam, the resolution of the electron beam signal and the signal-to-noise ratio through a light-dark contrast algorithm, thereby achieving the same contrast change between the image in the simulation display section and the image in the prototype display section; and / or The operable control device in the simulation control unit also includes a brightness and contrast automatic adjustment button. When the brightness and contrast are automatically adjusted by the brightness and contrast automatic adjustment button, the electron microscope image simulation calculation model simulates and calculates the signal intensity of the electron beam emitted to the sample surface through a light and dark contrast algorithm, and simulates and calculates the scanning speed of the electron beam, the resolution of the electron beam signal and the signal-to-noise ratio through a light and dark contrast algorithm, thereby achieving the same brightness and contrast changes of the image in the simulation display unit and the image in the prototype display unit.

7. The computer system based on the electron microscope image simulation calculation model as described in claim 1, wherein the operable control device in the analog control unit comprises an astigmatism and correction mode selection button, an astigmatism mode indicator light and a correction mode indicator light, and when the astigmatism mode is selected by the astigmatism and correction mode selection button, the astigmatism mode indicator light is on and the astigmatism mode is entered; and when the correction mode is selected by the astigmatism and correction mode selection button, the correction mode indicator light is on and the correction mode is entered; and / or The operable control device in the simulation control section includes an X-direction astigmatism and correction knob and a Y-direction astigmatism and correction knob. When the X-direction astigmatism and correction knob and / or the Y-direction astigmatism and correction knob are rotated, the electron microscope image simulation calculation model simulates and calculates the electron beam focusing ability through an astigmatism and focusing algorithm and a noise generation algorithm, thereby achieving the same astigmatism and correction changes in the image in the simulation display section and the image in the prototype display section in the astigmatism mode and the correction mode, respectively.

8. The computer system based on the electron microscope image simulation calculation model as described in claim 1, wherein the operable control device in the simulation control unit includes a coarse focus knob and a fine focus knob, and when the coarse focus knob and / or the fine focus knob are rotated, the electron microscope image simulation calculation model simulates the ability of the electromagnetic lens to focus the electron beam through an astigmatism and focusing algorithm and a noise generation algorithm, thereby achieving the same focusing change of the image in the simulation display unit and the image in the prototype display unit.

9. The computer system based on the electron microscope image simulation calculation model as described in claim 1, wherein the operable control device in the simulation control unit includes a first-gear scanning speed button, a second-gear scanning speed button, a third-gear scanning speed button, a fourth-gear scanning speed button, an image saving button and a photo taking button. When the first-gear scanning speed button, the second-gear scanning speed button, the third-gear scanning speed button or the fourth-gear scanning speed button and the photo taking button and / or the image saving button are pressed, the electron microscope image simulation calculation model simulates and calculates the scanning speed of the electron beam and the image photographing and saving effect in the three-dimensional scene through a scanning and photographing algorithm, thereby realizing scanning and photographing of the image in the simulation display unit that is the same as the image in the prototype display unit.

10. The computer system based on the electron microscope image simulation calculation model according to any one of claims 1 to 9, wherein: The low-magnification mode is 20 to 2000 times, and the high-magnification mode is 500 to 1 million times; In the multi-level mapping texture algorithm, ， Among them, N t is the normal vector in tangent space, N is the normal vector in model space, T is the tangent vector in model space, B is the bitangent vector in model space, B=T×N, T, B and N are perpendicular to each other, where: ， Among them, T x , T y and T z are the x, y, and z components of the tangent vector T, respectively, and B x , B y and B z are the x, y and z components of the bitangent vector B, N x 、N y and N z are the x, y and z components of the normal vector N, respectively, where the model space is the coordinate system that defines and stores the geometric data of the sample model, and the bitangent vector is perpendicular to the tangent vector and the normal vector; Among them, in the multi-level mapping texture algorithm, ， Among them, I 镜面 is the color intensity of the specular highlight component, I s is the specular light intensity of the light source, K s is the mirror reflection coefficient of the object surface, H is the half-angle vector used to approximate the calculation of the mirror highlight effect, N is the normal vector of the object surface, dot is for the dot product, pow is for the power function, and max is the function of the maximum value of the elements in the brackets that follow it; The calculation formula of the half-angle vector H is: ， Where L is the incident direction vector of the light, V is the observer direction vector, and normalized (L+V) is the normalized function of (L+V); The noise generation algorithm disturbs the source image through an irregular noise image, and uses the average value of the pixel values ​​at the same position of the two images as the pixel information of the corresponding position of the generated image; Wherein, in the light and dark contrast algorithm, the main texture color is sampled, the brightness is adjusted, the brightness value is calculated, the grayscale is mixed with the primary color to adjust the saturation, the grayscale is mixed with the final color to adjust the contrast, and the final color and the original transparency are returned; Wherein, in the astigmatism and focusing algorithm, the position colors of four different uv coordinates are sampled from the main texture, different weights are applied to these colors for mixing, and the final mixed color is returned as the pixel color output; Among them, in the translation and magnification algorithm, ， Where x and y are the coordinates before translation, x' and y' are the coordinates after translation, and Δx and Δy are the translation distances in the x and y directions; in, ; Among them, in the scanning and photographing algorithm, the image data is selectively processed according to the scanning mode, a temporary texture buffer is created, the threshold is adjusted, the material parameters are set: diffuse reflection, reflection, highlight glossiness, reflection glossiness, bump map, subdivision, transparency, refraction, and the contents of the source buffer are rendered to the target buffer, and finally the buffer reference is updated.

Images