Method and system for simulating electron microscope imaging based on 3D engine

Abstract

The invention relates to a method and system for simulating electron microscope imaging based on a 3D engine, and the system consists of two parts: a menu module and an imaging module, and aims to provide integrated virtual microscope operation experience. In the menu module part, an intuitive user interface is designed, and a user can browse a menu through a mouse. The system can intelligently recognize the target sample hovered by the mouse cursor, and dynamically display detailed information of the selected sample on the user interface, so that the interaction experience of a user is enhanced. The imaging module is responsible for rendering a corresponding sample image on a screen according to user selection. The module supports a glass slide switching function, and allows a user to flexibly browse microscopic images of different samples. In addition, the system also simulates an objective lens adjustment process of a real microscope, and a user can deeply observe detail features of a sample by switching to different magnifications.

Description

Technical Field

[0001] The present invention relates to the field of microscope imaging, and in particular to a method and system for simulating electron microscope imaging based on a 3D engine. Background Art

[0002] Electron microscopes play a vital role in scientific research and are widely used in many fields such as materials science, biomedicine, and nanotechnology. Traditional electron microscopes scan samples with electron beams and generate high-resolution images by detecting signals generated by the interaction between electrons and samples. However, these devices are usually expensive, complicated to operate, and have strict requirements on the use environment, which limits their wide application in teaching, training, and certain research. Therefore, in the fields of teaching, first of all, in order to solve this problem, the existing technology attempts to use simulation methods to realize teaching devices to meet the needs of teaching and other fields, so that students can understand the usage methods and principles. For example, the electron microscopy and energy spectrum principle experimental teaching device with patent application number 202111445424.5 belongs to the field of experimental teaching, and includes an electron optical path demonstration module, a variable vacuum operation demonstration module, an electron beam channel switch operation demonstration module, an electron beam focusing process demonstration module, and an electron beam scanning imaging process demonstration module; by demonstrating the generation and convergence process of electron signals in the electron microscope barrel, students can understand the generation mechanism of electron light sources, the electron beam convergence method, and the change of the electron beam spot diameter during the convergence process; the electron microscope operation steps can be demonstrated, which is convenient for teaching the types and application characteristics of objective lenses, energy spectra, electron beam contents, types of objective lenses, the relationship between electrons and samples, and energy spectrum contents.

[0003] Although the simulation device disclosed in the above patent can meet the requirements of teaching experiments to a certain extent, it still relies on mechanical hardware and other structures to achieve it, and does not use computer graphics technology to simulate it. With the rapid development of computer technology and graphics processing technology, the application of virtual reality (VR) and computer simulation technology in education and scientific research is increasing. 3D engine, as a powerful game engine and development platform, is widely used in the field of virtual reality and three-dimensional simulation with its excellent graphics processing capabilities and high scalability. By using the 3D engine platform for virtual microscopy, it can not only significantly reduce the equipment cost, but also improve the flexibility and convenience of the system.

[0004] Although there are some simulation microscope systems based on virtual reality technology in the prior art, these systems still have some problems, such as low image resolution, insufficient simulation effect, and poor interactivity, which limit the application effect of the system. In addition, the existing system also needs to be improved in terms of operation process and user experience. Therefore, the present invention is committed to improving the image quality and user experience of the simulation microscope system through advanced 3D engine technology to meet the needs of high-precision and high-realistic imaging. Summary of the invention

[0005] The purpose of the invention is to overcome the shortcomings of the prior art and provide a method and system for simulating electron microscope imaging based on a 3D engine. By utilizing the powerful graphics processing capabilities of the 3D engine and combining the electron microscope imaging principle, the present invention can simulate the observation process of samples under a microscope with high precision. The system not only improves the realism and detail expression of the image, but also has wide application potential in scientific research, education and training.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] The present invention develops a simulated electron microscope imaging system based on a 3D engine, which consists of a menu module and an imaging module. By accurately simulating the operating process of a real microscope, the system allows users to observe and analyze sample images in a virtual environment. In addition, a menu display method based on linear fitting is introduced into the system to achieve more accurate menu presentation and image positioning. The menu module cooperates with the imaging module, and the user can display and adjust the sample image to be observed on the virtual screen as required. This design not only significantly reduces the equipment cost, but also simplifies the use process of the microscope, allowing users to conveniently explore the wonders of the microscopic world in a virtual environment. The present invention aims to provide an efficient and easy-to-use virtual microscope solution, which improves the user's operating experience and expands the application scope of microscope technology in education, scientific research, training and other fields.

[0008] Based on the above purpose, in a first aspect, the present invention provides a menu display method based on linear fitting, comprising the following steps:

[0009] Step 1: determining the current state of the sample box in the simulated electron microscope environment, that is, determining whether the sample box is in an open or closed state;

[0010] Step 2: If the sample box is in an open state, further determine whether the mouse cursor is in contact with the glass slide in the sample box;

[0011] Step 3: If the mouse cursor is detected hovering over the slide, a yellow prompt box will be dynamically displayed in the upper right area of ​​the user interface to display the sample information related to the slide.

[0012] Optionally, in a menu display method based on linear fitting, the step 1 of determining whether the sample box is opened includes:

[0013] By reading transform.localEulerAngles.z of the sample box cover, we can get the rotation angle θ of the box cover about the z axis in the world coordinate system, and compare the obtained rotation angle θ with the preset threshold θ. t Make comparisons;

[0014] If the rotation angle θ is less than or equal to the threshold θ t , the sample box is considered to be in a closed state, otherwise the sample box is considered to be in an open state;

[0015] When the sample box is in the closed state, open the sample box by left-clicking the ring handle;

[0016] When the sample box is open, close it by right-clicking the ring handle.

[0017] Optionally, in a menu display method based on linear fitting, the step 2 of determining whether the mouse cursor is in contact with the glass slide comprises:

[0018] Add a Box Collider property to each slide object in the sample box to enable the slide to have physical collision detection function. A raycast is emitted from the camera position in the virtual environment. The direction of the raycast is calculated based on the coordinates of the slide area when the user clicks on the screen. After the raycast is emitted, the system determines whether the raycast trajectory collides with the slide object's Box Collider. If the raycast collides with the slide's collision body, the system can confirm that the mouse cursor has successfully contacted the slide object.

[0019] Optionally, in a menu display method based on linear fitting, the prompt box in step 3 prompting the slide information includes:

[0020] Slide S i The index is i, and the world coordinate system of the slide in the virtual microscope scene is represented by Where i = 0, 1, 2, 2n-1 represents the index of the slide; when the mouse cursor hovers over a slide, the system will capture the interaction event at that position and confirm that the mouse has hovered over the valid area of ​​the slide; the system will automatically move a yellow rectangular prompt box r to the corresponding position M in the menu coordinate system of the user interface. i to display the slide name information;

[0021] In order to complete the movement of the above prompt box r, this system adopts the least squares fitting method;

[0022] Assume that the slide S is known i World coordinates The slide S was calculated by the least square method. i The world coordinates are mapped to the corresponding positions in the menu coordinate system where u i and v i is the value of the two-dimensional coordinate in the menu coordinate system, corresponding to the position of the slide on the user interface; Determine the slide and determine its corresponding value from the slide according to Fitting to get in According to c s There are two possible values ​​for , corresponding to the different situations in the left and right columns of the menu, namely:

[0023]

[0024] For u', it is Sure;

[0025] Through the above steps, when the mouse cursor hovers over the slide, the coordinates of the prompt box r (u' i ,v' i ) is determined so that the name information of the touched slide is correctly displayed on the user interface.

[0026] In a second aspect, after obtaining the slide information in the first aspect and placing the slide, the system provides a simulated electron microscope imaging system based on a 3D engine, including:

[0027] Step 4: Generate and present a corresponding high-quality sample image on the display screen according to the glass slide sample placed on the stage;

[0028] Step 5: Adjust the screen brightness to simulate the observation effect under different light source intensities under the microscope;

[0029] Step 6: Dynamically adjust the clarity of the image according to the change of focal length;

[0030] Step 7: Calculate and update the position of the sample image on the display screen in real time according to the movement offset of the glass slide on the stage;

[0031] Step 8: Automatically change to the sample image at the corresponding magnification according to the magnification of the objective lens, and realize accurate positioning of the image according to the offset calculated previously;

[0032] Optionally, in a simulated electron microscope imaging system based on a 3D engine, the step 4 of presenting the sample image comprises:

[0033] Dynamically load sample image resources stored in the project resource directory through code;

[0034] Convert the acquired sample image from the original image format into the Sprite type and configure the relevant properties of the Sprite object; finally, apply the configured Sprite to the simulated electron microscope model to ensure that the sample image can be correctly presented on the user interface.

[0035] Optionally, in a simulated electron microscope imaging system based on a 3D engine, the step 5 of adjusting the screen brightness includes:

[0036] Step 1: Adjust the microscope frame and reflector to obtain the desired horizontal deflection angle θ in the microscope coordinate system h With vertical deflection angle θ v , calculate the corresponding gray level Where μ is the grayscale adjustment factor, which can be set according to actual conditions;

[0037] Step 2: Calculate the current gray level g m With the preset gray level g e The difference between △g=g e -g m ;

[0038] Step 3: Pass Normalize the difference value to a predetermined range to control the change of material color, where a is the adjustment coefficient used to adjust the rate of grayscale change;

[0039] Step 4: Finally, change the brightness of the image material through the color setting method in the engine, where the values ​​of the three RGB channels are uniformly set to g n ;

[0040] Optionally, in a simulated electron microscope imaging system based on a 3D engine, the step 6 of adjusting the image clarity comprises:

[0041] Assume that the displacement of the objective lens is △y, calculate the zoom error l=k·△y, where k is the zoom factor, and change the clarity of the image material by setting parameters, and change the clarity of the image material by adjusting l in the code.

[0042] Optionally, in a simulated electron microscope imaging system based on a 3D engine, the step 7 of calculating the image position includes:

[0043] Set the initial position of the image in the menu coordinate system to p I =(u I ,v I ), the initial position of the stage in the microscope coordinate system is p t =(xt ,y t ,z t ), then after adjusting the stage position, its position p' t =(x t +△x,y t ,z t +△z), where △x and △z represent the distances it moves left and right and front and back, respectively.

[0044] Then the position of the image after the shift can be expressed as p' I =(a(x t +△x)+b0,a(z t +△z)+b1), where a represents the scaling ratio, b0 and b1 represent the offsets in the x and z directions respectively.

[0045] Optionally, in a simulated electron microscope imaging system based on a 3D engine, the step eight of implementing image positioning includes:

[0046] After calculating the image position in step 7, the image position p' after moving in the menu coordinate system I =(a(x t +△x)+b0,a(z t +△z)+b1), let u I '=a(x t +△x)+b0,v I '=a(z t +△z)+b1,

[0047] The image width before the objective lens is switched is w0 and the height is h0. The image width after the switch is w1 and the height is h1. The image position in the menu coordinate system after the objective lens is switched is p" I =(u I 'w1 / w0,v I 'h1 / h0).

[0048] A system for simulating electron microscope imaging based on a 3D engine includes a menu module and an imaging module. The menu module is triggered by a user to adjust and display a corresponding image.

[0049] The advantages of the present invention are:

[0050] (1) The 3D engine-based simulated electron microscope imaging system provided by the present invention simulates the operation process of an electron microscope in a virtual environment, allowing users to intuitively observe and analyze the microstructure of a sample.

[0051] (2) Compared with traditional electron microscopes, this system effectively reduces equipment and maintenance costs, making it a more affordable choice, thereby improving cost-effectiveness.

[0052] (3) This system simplifies the operation process of the microscope through virtual reality technology. Users can complete sample observation without making complex equipment adjustments, which can significantly improve the convenience and efficiency of operation. It also reduces the time and space costs required by traditional microscopes and provides users with a more flexible operating environment.

[0053] (4) The system based on the 3D engine platform has good scalability and can be quickly updated and adjusted according to the latest scientific research progress and teaching needs, ensuring that the system can adapt to the ever-changing technology and application requirements.

[0054] (5) The present invention lowers the threshold for the use of microscopy technology, enabling scientific researchers and educational institutions to apply this technology more widely, promoting the popularization of scientific research and education, and providing strong support for research and teaching in related fields.

[0055] (6) The simulated electron microscope imaging system of the present invention provides users with a highly simulated virtual microscopic observation platform that is easy to operate, greatly improving the efficiency and experience of scientific research and educational activities, while reducing dependence on physical microscopes and saving equipment costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The following is a brief description of the contents expressed in the drawings of the present invention and the symbols in the drawings:

[0057] Figure 1 A schematic diagram of a menu display method based on linear fitting is shown;

[0058] Figure 2 A schematic diagram of a simulated electron microscope imaging system based on a 3D engine is shown;

[0059] Figure 3 A framework diagram of a simulated electron microscope imaging system based on a 3D engine is shown. DETAILED DESCRIPTION

[0060] The specific implementation of the present invention will be further explained in detail below by describing the optimal embodiment with reference to the accompanying drawings.

[0061] The present invention relates to the field of microscope imaging technology, and in particular to a simulated electron microscope imaging system based on a 3D engine platform. Specifically, the present invention provides a system and method for simulating the electron microscope image generation and display process by combining computer graphics technology and virtual reality technology. By giving full play to the powerful ability of the 3D engine in image processing. The present invention utilizes the powerful graphics processing capability of the 3D engine and combines it with the electron microscope imaging principle. The present invention can simulate the observation process of samples under a microscope with high precision. The system not only improves the realism and detail expression of the image, but also has broad application potential in scientific research, education and training.

[0062] The present invention develops a simulated electron microscope imaging system based on a 3D engine, which consists of a menu module and an imaging module. By accurately simulating the operating process of a real microscope, the system allows users to observe and analyze sample images in a virtual environment. In addition, a menu display method based on linear fitting is introduced into the system to achieve more accurate menu presentation and image positioning. The menu module cooperates with the imaging module, and the user can display and adjust the sample image to be observed on the virtual screen as required. This design not only significantly reduces the equipment cost, but also simplifies the use process of the microscope, allowing users to conveniently explore the wonders of the microscopic world in a virtual environment. The present invention aims to provide an efficient and easy-to-use virtual microscope solution, which improves the user's operating experience and expands the application scope of microscope technology in education, scientific research, training and other fields.

[0063] like Figure 3 As shown, the present embodiment provides a system for simulating electron microscope imaging based on a 3D engine, including a menu module and an imaging module, and triggers the imaging module to adjust and display the corresponding image according to the user's operation of the menu module. The function of the imaging module is triggered by the user adjusting the operation menu module to adjust the display effect of the corresponding microscope image and display the microscope image with the corresponding adjustment effect. The virtual electron microscope imaging system is developed and implemented in a software virtual environment. The present invention relates to a virtual electron microscope imaging system based on a software virtual environment, which provides a realistic virtual microscopic operation experience by simulating the hardware structure and operation process of a real electron microscope.

[0064] This system creates a total of 4 models in the user interface:

[0065] 1. Creation of a virtual simulation electron microscope: Use the 3D modeling tool Unity to build the structure and components of the microscope, including key components such as simulated lens barrels, objective lenses, eyepieces, stages, light sources, focusing knobs, mirror frames, and reflectors. It should be noted here that the main position and direction changes of the virtual simulation electron microscope are completed in the world coordinate system, and the rotation, scaling, and other operations of the microscope will change relative to the world coordinate system. In Unity, each object has its own local coordinate system, which represents the position information, rotation angle, and scaling ratio of the object relative to its parent object. For the four model objects created by this system, they each have an independent local coordinate system to facilitate precise positioning and control in the scene.

[0066] 2. Creation of virtual simulation sample box: Use 3D modeling tools in Unity to create a sample box model, including the box body and lid, and add a round handle to the lid. Add a Box Collider component to the round handle on the lid of the sample box. Box Collider is a physical component in Unity that is used to add an invisible, cube-shaped collision detection area to a 3D object. This collision area can detect collisions or interactions with other objects. Collision detection is used to control the switch of the sample box.

[0067] Create slides and create multiple slide objects inside the sample box, such as Figure 1 As shown in the left figure, there are 2n-1 slides in the sample box, arranged in 2 rows. Each slide is S i It means that 1≤i≤2n-1, and each slide also needs to add a BoxCollider component for collision detection.

[0068] When the user clicks an area of ​​the slide on the screen, Unity detects this input event, typically using Input.GetMouseButtonDown(0) to detect a left-click.

[0069] The virtual simulation sample box needs a state variable to record its current state (open or closed). Use a Boolean variable to indicate whether the sample box is currently open or closed. First, set it to false as the initial state of the sample box. Use Unity's Input class to detect the user's mouse click events: left click (Input.GetMouseButtonDown(0)); right click (Input.GetMouseButtonDown(1)). These events will be detected according to the different buttons the user clicks on the mouse. In order to ensure that the user clicks on the handle of the sample box, a ray (Raycast) needs to be emitted to detect the object clicked by the mouse. Through Physics.Raycast, it can be detected whether the object clicked by the user is the handle of the sample box and respond accordingly. The logic of opening the sample box is to simulate the opening and closing actions by rotating the lid. Use transform.localEulerAngles in Unity to represent the rotation angle of the sample box handle relative to the sample box itself. transform.localEulerAngles returns a three-dimensional vector containing X, Y, and Z axis rotations, but this article only needs to set the Z-axis rotation angle (transform.localEulerAngles.z), that is, the sample box handle relative to the Z axis of the coordinate system where the sample box is located. The rotation angle of the object around the Z axis is 0 degrees, which represents the action of closing the sample box. Setting the rotation angle of the object around the Z axis to 90 degrees represents the action of opening the sample box.

[0070] 3. Create a slide menu bar interface to display all slide name information in the sample box. The menu bar interface consists of 2 columns, each column has n slide information, a total of 2n slide information, use M i Indicates that the position of the slide list information in the menu bar is based on the local coordinate system of the menu bar. Figure 1 As shown in the right figure. Click the slide in the sample box with the mouse to display the corresponding slide information in the prompt box on the menu interface. The specific display in the menu bar is as follows Figure 2 As shown, Figure 2 The yellow box in the middle represents a prompt box. When the user selects a slide in the virtual sample box, the yellow box will move to the corresponding slide name information on the menu interface.

[0071] 4. Create a slide display screen and USB device for interaction with the eyepiece of the electron microscope. By presenting the slide information on the display screen, the slide can be observed. The user can drag the USB with the mouse and place it on the eyepiece. The connection status of the USB (connected or not connected) is managed by code, and collision detection is used to determine whether it can be connected. According to the user's mouse operation, the position and rotation of the USB are updated in real time to ensure that it is correctly aligned with the eyepiece. By adjusting the focal length of the electron microscope and the objective lens magnification, the slide cell information is displayed on the display screen, which is convenient for users to observe the slide cell information.

[0072] Light adjustment: Add a Light component to the microscope light source, adjust the light intensity through the UI slider to simulate the light intensity in the real environment, and write the slider value to control the intensity of the Light to achieve different lighting environments.

[0073] Focus adjustment: Use the left and right mouse buttons to control the coarse and fine focusing spiral adjustment wheels of the electron microscope model to simulate the change of focal length, thereby achieving the effect of focal length change. The user clicks the coarse and fine focusing spiral adjustment wheel with the left mouse button to gradually increase the focal length, simulating the coarse or fine adjustment of the focal length in reality. The user clicks the coarse and fine focusing spiral adjustment wheel with the left mouse button to reduce the focal length, simulating the coarse or fine adjustment of the focal length in reality.

[0074] Magnification switching: When the user presses the left or right button of the mouse on the objective lens of the electron microscope, the rotation of the objective lens is controlled by detecting the movement direction of the mouse on the horizontal x-axis in the world coordinate system to simulate the switching of the magnification.

[0075] The implementation of the present invention involves two main modules: a menu display method based on linear fitting and a simulated electron microscope imaging system based on a 3D engine. The present invention is further described in detail below in conjunction with specific embodiments.

[0076] like Figure 3 As shown, the embodiment of this specification provides a menu display module based on linear fitting, including the following steps: S1, judging the current state of the virtual simulation sample box, that is, whether it is in an open or closed state;

[0077] The rotation angle θ of the box cover on the z axis in the world coordinate system is obtained by reading transform.localEulerAngles.z of the virtual simulation sample box cover, and the obtained rotation angle is compared with the preset threshold θ. t Compare; if the rotation angle θ is less than or equal to the threshold θ t , the sample box is considered to be in a closed state, otherwise the sample box is considered to be in an open state;

[0078] When the sample box is in the closed state, open the sample box by left-clicking the ring handle;

[0079] When the sample box is open, right-click the ring handle to close the sample box. The opening and closing of the virtual sample box is controlled by triggering the left and right handles of the ring.

[0080] S2, if the sample box is in the open state, further determine whether the mouse is in contact with the glass slide;

[0081] Step 2 of detecting whether the mouse cursor is in contact with the slide includes:

[0082] Add a Box Collider property to each slide object in the sample box to give the slide a physical collision detection function. A ray (Raycast) is emitted from the camera position in the virtual environment. The direction of the ray is calculated based on the coordinates of the slide area when the user clicks on the screen. After the ray is emitted, the system determines whether the ray trajectory collides with the Box Collider of the slide object. If the ray collides with the slide's collision body, the system can confirm that the mouse cursor has successfully contacted the slide object. S3. If the mouse is detected to be in contact with the slide, the sample information related to the slide touched by the user will be dynamically displayed in a yellow prompt box in the upper right area of ​​the user interface, such as Figure 1 As shown, the specific steps include:

[0083] S31. When the mouse cursor touches the slide s in this box i , in the local coordinate system where the user menu bar is located, a yellow rectangular prompt box r is automatically moved to a certain position M of the corresponding menu bar. i , used to remind the user of the name of the currently selected slide. i The slide name information M of the local coordinate system where the menu bar is located i One-to-one correspondence. The name of the target slide touched by the mouse is prompted. In order to complete the movement of the above prompt box r, this system adopts the least squares fitting method to achieve it.

[0084] S32, the slide number in the local coordinate system of the menu bar is Location is

[0085] Indicates that u0 and u1 represent the column index of the menu bar slide name information, and v j Indicates the row index of a column of slide name information. The position coordinates of the left column slides in the menu bar local coordinate system are Indicates, that is, M0, M1…M n-1 ; The position coordinates of the slide in the right column in the local coordinate system of the menu bar are Indicates that Mn , M n+1 …M 2n-1 ;like Figure 2 As shown in the right figure, v' and u' are the position coordinates of each slide in the menu bar in the local coordinate system of the menu bar. The coordinates of the yellow moving prompt box in the local coordinate system of the menu bar are obtained by fitting. in

[0086] S33, according to There are two possible values ​​for , corresponding to the different situations in the left and right columns of the menu, namely:

[0087]

[0088] For u', it is Sure;

[0089] S34, through the above steps, after the mouse cursor hovers over the slide, the coordinates of the prompt box r (u' i ,v' i ) is determined so that the touched slide name information is correctly displayed on the user interface. The menu display bar has 2n slide name information. i Indicates that the prompt box r is used to highlight the name information of a slide in the menu display bar, and by hovering the mouse over a slide in the sample box, the prompt box is positioned to the corresponding slide information in the menu display bar to highlight it, so that users can understand the name information of the selected slide. Get the slide information, and after placing the slide, such as Figure 3 As shown, the present invention provides a method for simulating electron microscope imaging based on a 3D engine, comprising:

[0090] S4. Generate and present the corresponding high-quality sample image on the display screen according to the glass slide sample placed on the stage; specifically, dynamically locate and access the sample image file stored in the project resource directory through the code, and load the sample image resource into the memory using the interface provided by the 3D engine, wherein the sample and the sample image file are in a one-to-one correspondence. After obtaining the sample information, the corresponding sample image file can be loaded to prepare for the subsequent display.

[0091] The obtained sample image is converted from the original image format to the Sprite type and the relevant properties of the Sprite object are configured; finally, the configured Sprite is applied to the simulated electron microscope model to ensure that the sample image can be correctly presented on the display. S5. The user slides the slider on the interface to trigger the adjustment of the screen brightness to simulate the observation effect under different light source intensities under the microscope; when building a microscope in a virtual environment, add a Light component to the microscope light source, adjust the light intensity through the UI slider to simulate the light intensity in the real environment, and write the slider value to control the intensity of the Light to achieve different lighting environments.

[0092] S51, obtaining the horizontal and vertical deflection angles θ in the world coordinate system by adjusting the reflective mirror frame h ,θ v , then the gray level Wherein μ is the grayscale adjustment factor, which can be set according to actual conditions. According to the above steps, the deflection angle adjustment based on the reflector in the virtual electron microscope can be realized. This interactive method allows the user to intuitively control the observation angle of the microscope.

[0093] S52, calculate the current gray level g m With the reference gray level g e The difference between △g=g e -g m ;

[0094] S53. Normalize the difference value into a range to control the change of material color: Where a is the adjustment coefficient, which is used to adjust the rate of change of gray levels;

[0095] S54. Finally, the brightness of the image material is changed by setting the color in the engine, where the values ​​of the three RGB channels are all set to g n ; The color change is achieved by setting the brightness of the image material, and the values ​​of the three RGB channels are uniformly set to the standardized grayscale g n , to adjust the brightness of the material color;

[0096] S6. Dynamically adjust the clarity of the image according to the change of focal length; the microscope in the pre-built virtual environment has the same principle as the real microscope. By using the left and right buttons of the mouse to control the coarse and fine focusing spiral adjustment wheel of the electron microscope model, the change of focal length is simulated, thereby achieving the effect of focal length change. The user clicks the coarse and fine focusing spiral adjustment wheel with the left button of the mouse to gradually increase the focal length, simulating the increase of the coarse or fine adjustment focal length in reality. The user clicks the coarse and fine focusing spiral adjustment wheel with the left button of the mouse to reduce the focal length, simulating the decrease of the coarse or fine adjustment focal length in reality.

[0097] Assume that the displacement of the objective lens is △y, and calculate the zoom error l = k·△y, where k is the zoom factor. This error is used to dynamically adjust the clarity of the image. Depending on the size of the zoom error, different image resolutions are selected to achieve different clarity. For example, when the focal length changes greatly, a higher resolution can be selected to retain more details; when the focal length changes slightly or is blurred, the resolution is reduced to reduce the computational burden.

[0098] S7. According to the movement offset of the glass slide on the stage, the position of the sample image on the display screen is calculated and updated in real time; the display screen here is a computer display screen, and the entire microscope is developed in the computer's unity software, so the corresponding position is displayed on the display screen.

[0099] Assume that the initial position of the slide image on the stage in the world coordinate system is p t =(x t ,y t ,z t ), the initial position of the slide image in the corresponding slide display screen is p I =(u I ,v I ), then after adjusting the stage position it is p' t =(x t +△x,y t ,z t +△z), where △x and △z represent the distances it moves left and right and front and back respectively, that is, the stage moves along the x-axis and z-axis in the world coordinate system.

[0100] Correspondingly, the position of the slide on the stage changes, and the position of the slide slice displayed on the display screen also changes. Then the position of the image after the movement on the display screen can be expressed as p' I =(a(x t +△x)+b0,a(z t +△z)+b1), where a represents the scaling ratio, b0 and b1 represent the offsets in the x-axis direction and the z-axis direction respectively. The image movement operation can be completed by moving the image from the initial position to the corresponding position according to the position of the image after the movement, such as Figure 2 shown.

[0101] exist Figure 2 In the figure, the left picture is an example of the movement of the electron microscope slide, and the right picture is an example of the movement of the slide on the display screen. By adjusting the position of the slide on the stage, the position of the slide information in the display screen used to display the slide slice information also changes accordingly. However, the movement of the slide display screen is determined by the movement of the slide on the stage, which is equivalent to mapping.

[0102] S8. Adjust the image zoom, position and switch microscope images of different magnifications according to the objective lens magnification, and control the objective lens rotation and switch magnification according to the movement of the mouse. When the mouse is pressed and the horizontal movement exceeds the set threshold, the objective lens magnification will automatically increase or decrease. And according to the offset calculated previously, the image is accurately positioned; the magnification is switched through software configuration in the simulation environment: when the user presses the left or right button of the mouse on the objective lens of the electron microscope, the rotation of the objective lens is controlled by detecting the movement direction of the mouse on the horizontal x-axis in the world coordinate system, simulating the switching of magnification.

[0103] After the image position is calculated at S7, the image position p' on the display screen after the move I =(a(x t +△x)+b0,a(z t +△z)+b1), let u I '=a(x t +△x)+b0,v I '=a(z t +△z)+b1,

[0104] The image width before the objective lens is switched is w0 and the height is h0. The image width after the objective lens is switched is w1 and the height is h1. Then the image position on the display screen after switching the objective lens is P" I , according to the formula p" I =(u I 'w1 / w0,v I 'h1 / h0).

[0105] where u I ' and v I ' are respectively the adjusted image positions. Through the above steps, the system can dynamically adjust the position of the image on the display screen according to the change of the objective lens.

[0106] Obviously, the specific implementation of the present invention is not limited to the above-mentioned methods. As long as various non-substantial improvements are made using the method concept and technical solution of the present invention, they are all within the protection scope of the present invention.

Claims

1. A method for simulating electron microscope imaging based on a 3D engine, characterized in that: The steps include: Step 1: simulate a sample box in a software environment, and determine the open and closed states of the simulated sample box; Step 2: If the sample box is in an open state, determine whether the sample box is triggered by the mouse cursor; if it is detected that the mouse cursor triggers the sample box, trigger the user interface to display a reminder box, and the reminder box displays the sample information corresponding to the slide in the sample box to the user.

2. The method for simulating electron microscope imaging based on a 3D engine as claimed in claim 1, characterized in that: The method further comprises: Step 3: Generate and display a corresponding high-quality sample image on the display screen according to the glass slide sample placed on the electron microscope stage; Step 4: Adjust the screen brightness to simulate the observation effect under different light source intensities under the microscope; Step 5: Dynamically adjust the image clarity according to the change of focal length to simulate the focusing effect of a microscope; Step 6: Calculate and update the position of the sample image on the display screen in real time according to the movement offset of the slide on the stage; Step 7: According to the magnification of the objective lens, automatically switch to the sample image of the corresponding magnification.

3. A method for simulating electron microscope imaging based on a 3D engine as claimed in claim 1 or 2, characterized in that: In step 1, the rotation angle θ of the sample box cover on the z-axis in the world coordinate system is obtained, and the angle is compared with the preset threshold θ t Compare; if the rotation angle θ is less than or equal to the threshold θ t , the sample box is considered to be in a closed state; otherwise, the sample box is considered to be in an open state.

4. A method for simulating electron microscope imaging based on a 3D engine as claimed in claim 1 or 2, characterized in that: In step 2, when simulating a sample box in the software environment, a BoxCollider property is added to each glass slide object in the sample box so that the glass slide has a physical collision detection function, and a ray is emitted from the camera position in the virtual environment. The emission direction of the ray is calculated based on the coordinates when the user clicks the glass slide area on the screen; after the ray is emitted, the system determines whether the ray trajectory collides with the Box Collider of the glass slide object. If the ray collides with the collision body of the glass slide, the system can confirm that the mouse cursor has successfully contacted the glass slide object.

5. A method for simulating electron microscope imaging based on a 3D engine as claimed in claim 1 or 2, characterized in that: The slide information in the prompt box in step 2 includes: Slide S i The index is i, and the world coordinate system of the slide in the virtual microscope scene is represented by Where i = 0, 1, 2, 2n-1 represents the position of the slide; when the mouse cursor hovers over a slide, the system will capture the interaction event at that position and confirm that the mouse has hovered over the valid area of ​​the slide; the system will automatically move a yellow rectangular prompt box r to the corresponding position M in the menu coordinate system of the user interface. i to display the slide name information.

6. The method for simulating electron microscope imaging based on a 3D engine as claimed in claim 2, characterized in that: In step 3, the sample image file stored in the project resource directory is dynamically located and accessed through software code, and the sample image resource is loaded into the memory using the interface provided by the 3D engine; Convert the acquired sample image from the original image format into the Sprite type and configure the relevant properties of the Sprite object; finally, apply the configured Sprite to the simulated electron microscope model to ensure that the sample image can be correctly presented on the user interface.

7. The method for simulating electron microscope imaging based on a 3D engine as claimed in claim 2, characterized in that: In step 4, adjusting the screen brightness includes: Step 1: Adjust the microscope frame and mirrors to obtain the desired horizontal deflection angle θ in the microscope coordinate system h With vertical deflection angle θ v , thereby calculating the corresponding gray level g m , the calculation formula is: Where μ is the grayscale adjustment factor, which can be set according to actual conditions and is used to adjust the grayscale amplitude; Step 2: Calculate the current gray level g m With the preset gray level g e The difference between them is calculated as follows: △g=g e -g m Step 3: Normalize the difference value to a predetermined range by the following formula to control the variation of material color: Where a is the adjustment coefficient, which is used to control the rate of gray level change; the standardized gray level g n Make sure the brightness is adjusted within a suitable range so that the final color changes meet expectations; Step 4: Finally, the color change is achieved by setting the brightness of the image material, and the values ​​of the three RGB channels are uniformly set to the standardized grayscale g n , in order to adjust the brightness of the material color.

8. The method for simulating electron microscope imaging based on a 3D engine as claimed in claim 2, characterized in that: In step 5, adjusting the image clarity includes: Assuming the displacement of the objective lens is △y, the formula for calculating the zoom error l is as follows: l=k·△y Where k is the scaling factor and l is adjusted in the code to change the clarity of the image material.

9. The method for simulating electron microscope imaging based on a 3D engine as claimed in claim 2, characterized in that: Calculating the image position in step 6 includes: Set the initial position of the image in the menu coordinate system to p I =(u I ,v I ), and the initial position p of the microscope stage in the world coordinate system t =(x t ,y t ,z t ), when the user adjusts the position of the stage through the operation interface, the position of the stage is updated to p' t =(x t +△x,y t ,z t +△z), where △x and △z represent the distances of left-right movement and front-back movement respectively, then the image position after movement is P' I It can be expressed as: p' I =(a(x t +△x)+b0,a(z t +△z)+b1) Where a represents the scaling ratio, b0 and b1 represent the offset in the x and z directions respectively.

10. A system for simulating electron microscope imaging based on a 3D engine, characterized in that: It includes a menu module and an imaging module. The menu module is triggered by the user to adjust and display the corresponding image.

Images