Microscopic experiment teaching interaction method and device, computer equipment and storage medium

Abstract

The invention provides a microscopic experiment teaching interaction method, which is applied to a student end of an interactive teaching system, and comprises the steps: obtaining a target digital slice; in response to a course start instruction, synchronizing a target operation of the user on the target digital slice to the teacher end; in response to an after-class homework start instruction, displaying a slice discrimination homework to the student; in the process of completing the slice distinguishing operation, in response to an auxiliary question and answer opening instruction, calling the target large model to provide a question answer; and in response to an after-class homework submission instruction, synchronizing the slice distinguishing homework to the teacher end. According to the scheme, teaching interactivity is improved, teachers can timely understand student operation and give guidance, the students can timely answer questions when encountering the questions, information communication between the students and the teachers is enhanced, learning initiative and efficiency of the students are improved, the teachers are helped to carry out teaching and homework correction work more efficiently, and the teaching efficiency is improved. And the method is of great significance to improvement of microscopic experiment teaching quality.

Description

Technical Field

[0001] The present application relates to the technical field of microscopic experiment teaching, and in particular to a microscopic experiment teaching interactive method, device, computer equipment and storage medium. Background Art

[0002] In the current field of microscopic experimental teaching, traditional teaching methods have many limitations and are difficult to meet the needs of efficient teaching and personalized learning for students. For students, obtaining appropriate experimental materials and performing effective operations is a key issue when conducting microscopic experimental learning. In the existing teaching model, it is difficult for students to directly obtain high-quality target digital slices and cannot conveniently perform experimental operations. Moreover, during the course, it is difficult for students to synchronize their operations on the slices to the teacher in real time, so that teachers cannot understand the students' operations in time and give targeted guidance, which affects the teaching effect. In terms of homework, students often lack a systematic and intuitive way to display slice identification homework, which is not conducive to students' consolidation and application of the knowledge they have learned. When students encounter problems in the process of completing slice identification homework, there is no effective instant answer mechanism, and they cannot get help in time, resulting in low learning efficiency. Therefore, there is an urgent need for a new interactive method for microscopic experimental teaching that can effectively solve the above problems, improve the quality and efficiency of microscopic experimental teaching, and meet the actual needs of students and teachers in the teaching process. Summary of the invention

[0003] The purpose of this application is to solve at least one of the above-mentioned technical defects, especially the problems existing in the prior art such as lack of real-time interactive capability and fixed teaching samples.

[0004] In a first aspect, the present application provides a microscopic experiment teaching interactive method, which is applied to a student end of an interactive teaching system. The microscopic experiment teaching interactive method includes:

[0005] Obtain target digital slices;

[0006] In response to a course start instruction, synchronizing the user's target operation on the target digital slice to the teacher's end;

[0007] In response to an assignment start instruction, presenting the slice identification assignment to the student;

[0008] In the process of completing the slice identification operation, in response to the auxiliary question and answer start instruction, the target large model is called to provide question answers;

[0009] In response to the after-class homework submission instruction, the slice identification homework is synchronized to the teacher's side.

[0010] In one embodiment, obtaining a target digital slice includes:

[0011] Acquire a bright-field image of the target slice;

[0012] Determine the target modeling strategy;

[0013] If the target modeling strategy is bright field segmentation, the bright field image is input into the segmentation model to obtain a first segmentation mask, the first segmentation mask is range-indented to obtain a first indentation mask, and then the modeling point distribution is determined according to the first indentation mask;

[0014] If the target modeling strategy is dark field segmentation, multiple dark field sub-images corresponding to the bright field image are obtained, and the dark field sub-images are combined to obtain a dark field image; the dark field sub-images are obtained by sequentially scanning the slice areas corresponding to the bright field image using a higher magnification objective lens;

[0015] Inputting the dark field image into the segmentation model to obtain a second segmentation mask, and determining the distribution of modeling points according to the second segmentation mask;

[0016] Modeling is performed according to the distribution of modeling points to obtain the target digital slice.

[0017] In one embodiment, obtaining a plurality of dark-field sub-images corresponding to a bright-field image and combining the dark-field sub-images to obtain a dark-field image includes:

[0018] Determine the scanning area based on the bright field image;

[0019] Controlling the microscope to switch to a higher magnification objective lens, and scanning line by line according to the scanning area to obtain multiple dark field sub-images;

[0020] According to the coordinate mapping of each dark field sub-image in the scanning area, each dark field sub-image is filled into the scanning area to obtain a dark field image.

[0021] In one embodiment, determining the distribution of modeling points according to the first indentation mask includes:

[0022] Determine a concentrated distribution area according to the center of the first indentation mask;

[0023] Modeling points with a higher density than other areas are evenly distributed in the concentrated distribution area of ​​the first indentation mask to obtain a modeling point distribution.

[0024] In one embodiment, the target operation includes a question operation and an answer operation. In response to a course start instruction, the target operation of the user on the target digital slice is synchronized to the teacher's end, including:

[0025] In response to the user's question operation on the target digital slice, determine the question area on the target digital slice, and synchronize the question content and the screenshot corresponding to the question area to the teacher's end;

[0026] Receive the first recognition target sent by the teacher end, and in response to the user's question and answer operation on the target digital slice, determine the question and answer area on the target digital slice, and synchronize the screenshot corresponding to the question and answer area to the teacher end.

[0027] In one embodiment, in response to the auxiliary question-answering start instruction, calling the target big model to provide a question answer includes:

[0028] Generate restriction prompt words according to the user's teaching objectives;

[0029] Inputting restriction prompt words into the target large model to limit the question and answer scope of the target large model;

[0030] In response to a user's annotation selection operation on a target digital slice, determining a corresponding target screenshot;

[0031] Use the target big model to conduct Q&A based on the target screenshots and user questions.

[0032] In one embodiment, generating a restriction prompt word according to the user's teaching goal includes:

[0033] Determine the experimental field based on the course, chapter and subject corresponding to the slice identification assignment;

[0034] Generate restriction cue words based on the experimental domain.

[0035] In one embodiment, the process of using the target macro model to conduct question-answering based on the target screenshot and the user's question also includes:

[0036] Determine the corresponding target knowledge graph according to the experimental field; the target knowledge graph includes preset knowledge points corresponding to the experimental field;

[0037] Before inputting the latest question into the target large model, the latest question is subjected to keyword extraction, and the extracted target keywords are matched with each preset knowledge point in the target knowledge graph to determine the target knowledge point;

[0038] For any target knowledge point, the mastery level of the target knowledge point is updated based on the user's question and answer performance.

[0039] In one embodiment, the process of completing the slice identification operation also includes:

[0040] In response to the auxiliary recognition start instruction, determining a region to be recognized on the target digital slice;

[0041] Inputting the screenshot corresponding to the area to be identified into the area detection model, and taking the identification results with the highest confidence among the multiple identification results as the optional results; the area detection model is used to identify the category of the input image to obtain multiple identification results and corresponding confidences;

[0042] Output optional results to the user for the user to choose;

[0043] The result selected by the user is determined as the first result, and the result with the highest confidence among the optional results is determined as the second result;

[0044] If the first result is the same as the second result, it is determined that the user's selection is correct.

[0045] In one embodiment, if the first result is the same as the second result, then it is determined that the user's selection is correct, further comprising:

[0046] If the first result is different from the second result, and the maximum deviation value of the confidence between the optional results is less than the first threshold, outputting all optional results to the user;

[0047] If the first result is different from the second result, and the confidence deviation value between the optional results is greater than or equal to the first threshold, the optional result with the highest confidence is output to the user.

[0048] In one embodiment, if the first result is the same as the second result, then it is determined that the user's selection is correct, further comprising:

[0049] If the confidence level corresponding to the target result is lower than the second threshold, the optional results, the target result and the area to be identified are sent to the teacher terminal;

[0050] Re-judge whether the user's selection is correct based on the teacher's feedback.

[0051] In one embodiment, in response to the auxiliary recognition start instruction, determining the area to be recognized on the target digital slice further includes:

[0052] Record the number of times the auxiliary identification start command is triggered during this slice identification operation;

[0053] Output optional results to the user for selection, including:

[0054] If the triggering number is greater than the third threshold, the corresponding first test question is determined according to the slice identification operation and displayed;

[0055] When the correct rate of the first test question is greater than a fourth threshold, outputting optional results to the user for the user to select;

[0056] Otherwise, the knowledge point analysis corresponding to the slice identification task is output to the user.

[0057] In one embodiment, outputting the knowledge point analysis corresponding to the slice identification operation to the user includes:

[0058] Display knowledge point analysis through interactive pop-up windows and activate the test start control after the first set time;

[0059] In response to an operation on a test start control, displaying a second test question corresponding to the knowledge point analysis to the user;

[0060] If the user answers the second number of second test questions correctly, close the interactive pop-up window; otherwise, return to the step of displaying the knowledge point analysis through the interactive pop-up window and activating the test start control after the first set time.

[0061] In one embodiment, the microscopy experiment teaching interaction further includes:

[0062] If the first result does not match the second result, and the difference between the confidence level corresponding to the first result and the second result is lower than a fifth threshold, the first result is used as the target category, and the screenshot corresponding to the area to be identified is used as a reference image to be sent to the image generation model to obtain a confusing image; the image generation model is used to generate a confusing image whose recognition result is the target category according to the reference image;

[0063] The first result and the easily confused image are presented to the user as one group, and the second result and the screenshot corresponding to the area to be identified are presented as another group.

[0064] In one embodiment, before displaying the first result and the easily confused image as one group and the second result and the screenshot corresponding to the area to be identified as another group to the user, the method further includes:

[0065] Determine a marking distinguishing structure according to the first result, the second result and the first mapping relationship;

[0066] The landmark distinguishing structures and the confusable images are input into the annotation model to annotate the landmark distinguishing structures in the confusable images.

[0067] In a second aspect, the present application provides a microscopic experiment teaching interactive device, which is applied to a student end of an interactive teaching system. The microscopic experiment teaching interactive method includes:

[0068] A digital slice acquisition module, used for acquiring target digital slices;

[0069] A course interaction module, for synchronizing the user's target operation on the target digital slice to the teacher's end in response to a course start instruction;

[0070] The homework module is used to respond to the homework start instruction and display the slice identification homework to the students. In the process of completing the slice identification homework, it responds to the auxiliary question and answer start instruction to call the target large model to provide answers to the questions. In response to the homework submission instruction, the slice identification homework is synchronized to the teacher's end.

[0071] In a third aspect, the present application provides a computer device comprising one or more processors and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the steps of the microscopic experiment teaching interactive method in any of the above-mentioned embodiments are executed.

[0072] In a fourth aspect, the present application provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the microscopic experiment teaching interactive method in any of the above-mentioned embodiments.

[0073] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0074] The interactive method for microscopic experiment teaching provided in this scheme is applied to the student end of the interactive teaching system. First, the target digital slice is obtained, which can be obtained from existing resources or scanned and generated through the cooperation of the digital slice module and the scanning imaging module; at the beginning of the course, the course start instruction is responded to, and the target operation of the user on the target digital slice is synchronized to the teacher end through the classroom interactive module, so that the teacher can grasp the student operation in real time; in the after-school homework link, the after-school homework start instruction is responded to, and the slice identification homework is displayed to the students; in the process of completing the slice identification homework, the auxiliary question and answer start instruction is responded to, and the question answer is provided by calling the target large model to solve the doubts for the students; finally, the after-school homework submission instruction is responded to, and the slice identification homework is synchronized to the teacher end. Through the coordination of various steps, this scheme realizes the functions of digital slice acquisition, operation synchronization, homework display and submission, and question answering in the teaching process, which improves the interactivity of teaching. The teacher can understand the student operation in time and give guidance, and the student can get answers in time when encountering problems. At the same time, it enhances the information exchange between students and teachers, improves the initiative and efficiency of students' learning, and also helps teachers to carry out teaching and homework correction more efficiently, which is of great significance to the improvement of the quality of microscopic experiment teaching. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0076] Figure 1 A schematic diagram of the process flow of the interactive method for microscopic experiment teaching provided in the embodiment of the present application;

[0077] Figure 2 This is a schematic diagram of a process for obtaining a target digital slice in one embodiment of the present application;

[0078] Figure 3 A schematic diagram of a process for generating a dark field image in one embodiment of the present application;

[0079] Figure 4 This is a comparison diagram of the output results of the segmentation model and the target detection model in one embodiment of the present application;

[0080] Figure 5 A comparison diagram of a bright field image and a dark field image in one embodiment of the present application;

[0081] Figure 6 This is a flow chart of using a target large model to assist question answering in one embodiment of the present application;

[0082] Figure 7 A schematic diagram of a process for determining a user's mastery of a target knowledge point in one embodiment of the present application;

[0083] Figure 8 This is a schematic diagram of the auxiliary identification process in one embodiment of the present application;

[0084] Fig. 9 A schematic diagram of a process for preventing abuse of auxiliary teaching in one embodiment of the present application;

[0085] Fig.10 A schematic diagram of a process for preventing abuse of auxiliary teaching in one embodiment of the present application;

[0086] Fig.11 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0087] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0088] The present application provides an interactive method for microscopic experiment teaching, which is applied to the student end of an interactive teaching system. The interactive teaching system includes a student end and a teacher end, and the functional modules included therein include a fluorescence analysis module, an industrial analysis module, a scanning imaging module, a classroom interaction module, an online discussion module, a user management module, an examination module, a question-and-answer dialogue module, a digital slice module, an image recognition module, and a data management module, etc. The devices that can be linked include scanners, microscopes, etc.

[0089] Among them, the fluorescence analysis module is mainly used to analyze samples involving fluorescence characteristics in microscopic experiments. Its functional principle is to detect the fluorescence signal emitted by the fluorescent substance in the sample, analyze the intensity, wavelength, distribution and other characteristics of the fluorescence, and then obtain relevant information of the sample. It can also be used to judge the quality of the film. The industrial analysis module is used to perform microstructural analysis on industrial materials, parts, etc. to evaluate their quality, performance, etc. The principle is to obtain the microscopic image of the industrial sample through a microscope and other equipment, and then use the image analysis algorithm to process and analyze the image to extract relevant feature parameters. The scanning imaging module is used to scan and image the sample to obtain the digital image of the sample. Its principle is to scan the sample point by point or line by line through a scanning device (such as a scanner, microscope, etc.), convert the optical information of the sample into a digital signal, and then generate a digital image.

[0090] The classroom interaction module is mainly used to realize the interaction between the student end and the teacher end in the classroom. Its functions include student operation synchronization, teacher instruction reception, real-time feedback, etc. Specifically, the operation information of the student end can be transmitted to the teacher end in real time through network communication technology, and the instructions of the teacher end can be transmitted to the student end at the same time, so as to realize two-way interaction. The online discussion module provides an online discussion platform where students and teachers can have academic discussions and exchange questions. Specifically, it can store and manage the discussion content through the network server, and users can access the server through the client to discuss. The user management module is responsible for managing the users of the interactive teaching system, including user registration, login, permission setting, information management and other functions. Specifically, it can store user information through the database, and the system performs corresponding operation control according to the user's identity and authority. The examination module is used to organize and manage examinations, and its specific functions include test paper generation, examination arrangement, candidate answering, automatic scoring, etc. The test questions can be stored through the question bank management system, and the test papers can be generated according to the test requirements. The candidates answer the questions through the client, and the system automatically scores according to the preset scoring standards. The question-and-answer dialogue module provides students with real-time dialogue services to help them answer questions related to experiments. Specifically, it can understand and analyze the questions input by users through a large language model, and then organize the answers and return them to the users. The digital slice module is mainly responsible for managing and providing digital slice resources. Its functions include storage, retrieval, upload, and download of digital slices. Specifically, the relevant information of digital slices can be stored in the database, and users can operate through the client. The image recognition module is mainly used to identify and analyze microscopic images. Its functions include image classification, target detection, feature extraction, etc. The principle is to train a large number of microscopic images through machine learning algorithms (such as convolutional neural networks), learn the characteristics and patterns of images, and then identify and analyze the input images. The data management module is mainly responsible for managing various data in the interactive teaching system, including user data, experimental data, teaching data, etc. Its functions include data storage, backup, recovery, query, statistics, etc. The principle is to manage data through a database management system to ensure the security, integrity, and availability of data. Both the student end and the teacher end can use these modules in the system to achieve teaching interaction.

[0091] See also Figure 1 The microscopic experiment teaching interactive method in this embodiment includes steps S102 to S110.

[0092] S102, obtaining a target digital slice.

[0093] It can be understood that the target digital slice refers to a specific digital slice resource determined according to the teaching objectives and student needs in microscopic experimental teaching. These digital slices can be digital images formed by students making their own slices according to the teacher's requirements and uploading them to the system for students to perform experimental operations and learn. Specifically, the required target digital slices can be obtained through the cooperation of the digital slice module and the scanning imaging module. The digital slice module is responsible for managing and storing digital slice resources, and the scanning imaging module is used to scan and image the samples to generate digital slices.

[0094] S104, in response to the course start instruction, synchronizing the user's target operation on the target digital slice to the teacher's end.

[0095] It can be understood that the course start instruction refers to the operation instruction issued by the teacher on the teacher side to indicate the start of the course, which is used to start the teaching process. The target operation refers to the specific operation performed by the student on the target digital slice that needs to be synchronized to the teacher side, such as asking questions, answering, etc. Specifically, during the experimental class, students can learn based on the teacher's explanation and the target digital slice displayed on this side. When the teacher asks questions and requires students to find the corresponding target on the digital slice. The student side can receive the first recognition target sent by the teacher side, and in response to the user's question and answer operation on the target digital slice, determine the question and answer area on the target digital slice, and synchronize the screenshot corresponding to the question and answer area to the teacher side. The teacher side can review whether the answer results of each student are correct to ensure that the students' cognition is correct. The teacher finds that typical cases can be extracted through the teacher side and synchronized to all student sides for explanation in real time. For typical error cases, the teacher side can also temporarily publish in-class questions to consolidate the teaching objectives. In addition, when students encounter something they do not understand during the learning process, they can also ask questions on the student side. The student side responds to the user's question operation on the target digital slice and determines the question area on the target digital slice. That is, the student circles the difficult-to-identify area to obtain the question area. The student side then synchronizes the question content and the screenshot corresponding to the question area to the teacher side.

[0096] S106, in response to the after-class homework start instruction, showing the slice identification homework to the students.

[0097] It can be understood that the after-school homework start instruction refers to the operation instruction issued by the teacher on the teacher side to indicate the start of the after-school homework, which is used to start the after-school homework link. Slice identification homework refers to the homework task assigned by the teacher to students to identify and analyze specific digital slices based on the teaching content and objectives, such as identifying cell types and structural features in the slices. This step is to obtain the content of the slice identification homework from the relevant homework database or the questions written by the teacher after receiving the after-school homework start instruction through the system's course management and homework management mechanism, and display it on the student side interface. The system sends the homework information to the student side according to the homework parameters set by the teacher on the teacher side (such as homework title, requirements, time limit, etc.).

[0098] S108, in the process of completing the slice identification operation, in response to the auxiliary question and answer start instruction, the target large model is called to provide question answers.

[0099] It can be understood that the auxiliary question and answer activation instruction refers to the operation instruction issued by the student to activate the auxiliary question and answer function in order to obtain help in the process of completing the slice identification homework. The target large model refers to an artificial intelligence model that has been trained with a large amount of data, which can understand and answer various questions related to microscopic experiments. In this system, it is used to provide students with answers to questions about slice identification homework. It can be integrated with the question and answer dialogue module and the target large model. When the student end receives the auxiliary question and answer activation instruction, the question raised by the student is sent to the question and answer dialogue module. The question and answer dialogue module then forwards the question to the target large model, and the target large model analyzes and answers the question based on its trained knowledge and algorithm, and returns the answer result to the question and answer dialogue module. The question and answer dialogue module finally sends the answer result to the student end for display to the student. Specifically, the question and answer dialogue module can use the network interface to communicate with the target large model. The student enters the question on the student end interface and clicks the auxiliary question and answer activation instruction button. The system sends the question to the question and answer dialogue module, and the question and answer dialogue module obtains the answer result by calling the API of the target large model and displays it on the student end interface.

[0100] S110, in response to the after-class homework submission instruction, the slice identification homework is synchronized to the teacher's end.

[0101] It can be understood that the after-class homework submission instruction refers to the operation instruction issued by the student to submit the homework to the teacher after completing the slice identification homework. After receiving the after-class homework submission instruction, the student end specifically organizes and encapsulates the slice identification homework information completed by the student, and then sends it to the teacher end through network communication technology.

[0102] The interactive method for microscopic experiment teaching provided in this scheme is applied to the student end of the interactive teaching system. First, the target digital slice is obtained, which can be obtained from existing resources or scanned and generated through the cooperation of the digital slice module and the scanning imaging module; at the beginning of the course, the course start instruction is responded to, and the target operation of the user on the target digital slice is synchronized to the teacher end through the classroom interactive module, so that the teacher can grasp the student operation in real time; in the after-school homework link, the after-school homework start instruction is responded to, and the slice identification homework is displayed to the students; in the process of completing the slice identification homework, the auxiliary question and answer start instruction is responded to, and the question answer is provided by calling the target large model to solve the doubts for the students; finally, the after-school homework submission instruction is responded to, and the slice identification homework is synchronized to the teacher end. Through the coordination of various steps, this scheme realizes the functions of digital slice acquisition, operation synchronization, homework display and submission, and question answering in the teaching process, which improves the interactivity of teaching. The teacher can understand the student operation in time and give guidance, and the student can get answers in time when encountering problems. At the same time, it enhances the information exchange between students and teachers, improves the initiative and efficiency of students' learning, and also helps teachers to carry out teaching and homework correction more efficiently, which is of great significance to the improvement of the quality of microscopic experiment teaching.

[0103] In one embodiment, see Figure 2 , obtaining a target digital slice, including steps S202 to S212.

[0104] S202, acquiring a bright field image of the target slice.

[0105] It can be understood that the target slice refers to the specific slice sample targeted in the present slice modeling method. It is a sample that has undergone specific processing (such as fixation, slicing, staining, etc.) and is used for subsequent image acquisition and modeling processing. Brightfield images are images acquired through brightfield microscope imaging technology. In brightfield imaging, light directly passes through the sample, and different structures in the sample absorb and scatter light to different degrees, thereby presenting different grayscale or color differences on the image to reflect the structural information of the sample. Specifically, conventional microscope equipment can be used to place the prepared target slice on the stage of the microscope, and adjust the parameters such as the focal length and illumination intensity of the microscope to make the target slice clearly imaged. Then, the image acquisition device equipped with the microscope (such as a CCD camera or a CMOS camera) is used to acquire the image of the target slice, and the optical image is converted into a digital image and stored in the computer.

[0106] S204, determining a target modeling strategy.

[0107] It can be understood that the target modeling strategy refers to the specific way and method used when modeling the target slice, including different strategy selections such as bright field segmentation and dark field segmentation, which determines the subsequent processing method and process of the bright field image. After acquiring the bright field image, it is necessary to select the appropriate modeling strategy based on the specific needs and sample characteristics. Specifically, it can be manually selected by the operator based on the preliminary observation of the target slice image and the research purpose, or it can be configured in advance before use.

[0108] S206, if the target modeling strategy is bright field segmentation, the bright field image is input into the segmentation model to obtain a first segmentation mask, the first segmentation mask is range-indented to obtain a first indentation mask, and then the modeling point distribution is determined according to the first indentation mask.

[0109] It can be understood that the microscope uses a low-power objective lens when collecting bright field images, which has a larger field of view, can complete the collection without scanning, has high imaging efficiency, and the bright field segmentation strategy can directly segment the image. When the target modeling strategy is bright field segmentation, the bright field image is input into the segmentation model. The segmentation model is a neural network model trained to segment the stained slice tissue area, which can output a corresponding mask for the input image, but when the input image is a bright field image, the first segmentation mask it outputs corresponds to the slice tissue area marked by a marker (such as a marker) in the image. The first segmentation mask is essentially an image data of the same size as the bright field image, usually presented in the form of a binary image or a multi-channel image. In the case of a binary image, the value of each pixel (usually 0 or 1) represents whether the point belongs to a specific structure or object; in a multi-channel image, the combination of pixel values ​​of different channels can represent different categories. Based on this, after the bright field image of the cell slice is segmented by the segmentation model, the first segmentation mask can clearly identify the cell area and the background area.

[0110] Although the acquisition of bright field images is very efficient, the results of segmentation based on bright field images will contain many areas that are not related to the slice tissue. When laying out the modeling points, some invalid modeling points will be produced, which increases the time consumption of modeling. For this reason, it is necessary to indent the range of the first segmentation mask, which means that at the boundaries of each area identified by the first segmentation mask, the area is contracted inward according to certain rules and degrees, so that the original area range becomes smaller. The purpose of this operation is to further refine the segmented area to highlight the core part of the area. When modeling the slice, all modeling points are distributed in the first indentation mask. Therefore, the distribution of modeling points can be determined according to the first indentation mask and the set distribution rules. Since the first indentation mask is more accurate than the result range obtained by the traditional bright field segmentation modeling method, the number of invalid modeling points can be reduced, and the overall modeling efficiency of bright field segmentation modeling is improved.

[0111] S208, if the target modeling strategy is dark field segmentation, obtain multiple dark field sub-images corresponding to the bright field image, and combine the dark field sub-images to obtain a dark field image. The dark field sub-images are obtained by sequentially scanning the slice areas corresponding to the bright field image using a higher magnification objective lens.

[0112] S210: Input the dark field image into the segmentation model to obtain a second segmentation mask, and determine the distribution of modeling points according to the second segmentation mask.

[0113] It can be understood that the dark field image is formed by combining multiple dark field sub-images. The dark field sub-image uses a strong and narrow light beam to illuminate the specimen without allowing the light beam to directly enter the objective lens. However, the particles in the specimen can scatter light, and some of these scattered light rays enter the objective lens, so on a dark background, the particles in the specimen can also be seen at the flash point, and each dark field sub-image is collected using a higher magnification objective lens than when collecting bright field images, which can provide more detailed structural information than bright field images. The dark field image is composed of a combination of multiple dark field sub-images, and the dark field sub-image needs to be scanned and collected multiple times, so its generation takes a certain amount of time, but the dark field segmentation strategy can directly segment the actual position of the sample in the fluorescence imaging, greatly increasing the proportion of effective modeling points, reducing the time-consuming modeling, thereby neutralizing the time cost required to construct the dark field image, thereby improving the overall efficiency. Therefore, the two modeling strategies in this application can take into account both efficiency and quality at the same time, and users can choose bright field or dark field strategies as needed.

[0114] Specifically, after scanning to obtain multiple dark field sub-images, the coordinate mapping relationship between them and the bright field image is combined into a dark field image. The dark field image will be input into the segmentation model for region division. The segmentation model is trained using bright field images and dark field images respectively, and it has the ability to perform segmentation under these two strategies. The difference is that during training, the annotation of the bright field image only annotates its tissue slices.

[0115] S212, modeling is performed according to the distribution of modeling points to obtain a target digital slice.

[0116] It can be understood that after the modeling distribution points are determined, feature points are collected according to the corresponding positions of the arranged modeling points, and a model is constructed according to the collected features, so that the structural features of the target slice can be expressed in the form of a mathematical model, a two-dimensional model, or a three-dimensional model. By reasonably utilizing the distribution of modeling points, a model that accurately reflects the sample structure can be constructed to achieve modeling analysis of the target slice, and the final modeling result is the target digital slice.

[0117] In one embodiment, determining the distribution of modeling points according to the first segmentation mask includes determining a concentrated distribution area according to the center of the first indentation mask, and uniformly distributing modeling points with a higher density than other areas in the concentrated distribution area of ​​the first indentation mask to obtain the distribution of modeling points.

[0118] It can be understood that the concentrated distribution area refers to a specific area determined according to the center of the first indentation mask. Considering that the marks drawn by the bright field image using a marker pen are larger than the actual boundary of the slice tissue, and the slice tissue is closer to the center of the mark. Therefore, by setting the center of the concentrated distribution area according to the center of the first indentation mask (such as setting them to be concentric), it can be ensured that more modeling points can fall in the core area of ​​the slice tissue, and the effective proportion of modeling points can be increased, thereby improving accuracy and efficiency. For each area in the first indentation mask, the traditional geometric center calculation method can be used. For example, for a simple rectangular area, its center can be determined by calculating the intersection of the diagonal lines of the rectangle; for an irregularly shaped area, the pixel coordinate information of the image can be used to calculate the average value of the coordinates of all pixel points to obtain the center of the area. In specific implementation, a programming language (such as Python) can be used in combination with an image processing library (such as OpenCV) to achieve it.

[0119] The area where the first indentation mask is located is divided into concentrated distribution areas and other areas. The higher density of modeling points in the concentrated distribution area means that there are more modeling points per unit area in the area. Since the concentrated distribution area has been determined as an area containing important features of the target structure, increasing the density of modeling points in this area can capture and express these features in more detail. Setting relatively few modeling points in other areas can not only ensure basic coverage of the entire target slice structure, but also avoid excessive redundancy of modeling points, thereby improving the efficiency and accuracy of modeling. This distribution of modeling points can make the constructed model have higher accuracy in key parts, while maintaining a reasonable complexity as a whole, and better reflect the structural characteristics of the target slice.

[0120] In one embodiment, performing range indentation on the first segmentation mask to obtain the first indentation mask includes: determining the center point coordinates of the first segmentation mask, determining the coordinates of the vertices of a rectangle according to the center point coordinates and a preset offset, the coordinates of the vertices of the rectangle being located inside the first segmentation mask, and obtaining the first indentation mask according to the coordinates of the vertices of the rectangle.

[0121] It can be understood that determining the coordinates of the center point of the first segmentation mask is a basic step for performing the range indentation operation. The center point has certain representativeness and symmetry in the entire mask area, and using the center point as a reference can more conveniently and evenly perform range indentation on the mask.

[0122] The preset offset is a preset value used to control the size of the rectangular area constructed with the center point as the reference. It represents the distance offset from the center point in all directions (horizontally and vertically). By adjusting the size of the preset offset, the size of the rectangular area can be changed, thereby achieving different degrees of range indentation of the first segmentation mask. When the center point coordinates and the preset offset are determined, the position coordinates of the four vertices of the rectangle in the image coordinate system can be determined based on the preset offset, with the center point of the first segmentation mask as the reference, usually expressed in the form of (x1, y1), (x2, y2), (x3, y3), (x4, y4). These vertex coordinates determine a rectangular area located inside the first segmentation mask, which will be used to obtain the first indentation mask later. This rectangular area will serve as the boundary of the mask after the range indentation, so that the original first segmentation mask is retained within the rectangular area, and the part beyond the area is removed, achieving the effect of range indentation. In the entire range indentation operation, this step plays a key role in area definition. Reasonable determination of the coordinates of the rectangle vertices can ensure that the mask after range indentation accurately reflects the core part of the target structure while avoiding excessive removal of useful information.

[0123] In one embodiment, determining the concentrated distribution area according to the center of the first indentation mask includes: determining a target size of the concentrated distribution area according to the diagonal length of the first indentation mask and the first ratio, and determining the concentrated distribution area according to the target size and a set shape.

[0124] It can be understood that the first indentation mask is a rectangle, and the diagonal length can reflect the size and range of the area. It is an important geometric feature quantity, which is used for the subsequent calculation of the target size of the concentrated distribution area. The first ratio is a preset value, ranging from 0 to 1. It is used to control the degree of scaling of the target size of the concentrated distribution area based on the diagonal length of the first indentation mask. By adjusting the value of the first ratio, the size of the concentrated distribution area can be changed to adapt to different modeling requirements and characteristics of the target slice. The target size refers to the size measurement of the concentrated distribution area calculated based on the diagonal length of the first indentation mask and the first ratio.

[0125] The first indentation mask is already the result of optimizing the original segmentation mask, and its diagonal length reflects the range of the core part of the target structure to a certain extent. By multiplying the diagonal length by the first ratio, a relatively reasonable size value can be obtained as the target size of the concentrated distribution area. The reason for doing this is that the size of the concentrated distribution area is determined by using the inherent characteristics of the first indentation mask, which can make the concentrated distribution area closely related to the core part of the target structure, ensuring that the modeling points set in the area can more accurately reflect the key features of the target structure.

[0126] The set shape is a pre-specified geometric shape of the concentrated distribution area. Common shapes include square, rectangle, regular hexagon, regular octagon, circle, etc. The set shape determines the outline and boundary form of the concentrated distribution area. Different shapes are suitable for different target structure characteristics and modeling requirements. Choosing a suitable set shape helps to more accurately include the key parts of the target structure so that the modeling points can be reasonably distributed in the area later.

[0127] On the basis of determining the target size, the specific position and outline of the concentrated distribution area are clarified in combination with the set shape. The target size provides information about the size of the area, while the set shape specifies the geometric form of the area. By applying the target size to the set shape, a specific area, namely the concentrated distribution area, can be determined in the first indentation mask image. For example, if the set shape is a square and the target size determines the value of the side length, then a square with a side length of the target size can be drawn in the image as a concentrated distribution area based on the center of the first indentation mask. The concentrated distribution area determined in this way can reasonably delineate the range that needs to be focused on and set the modeling points according to the characteristics of the target structure and the modeling requirements.

[0128] In one embodiment, uniformly distributing modeling points with a higher density than other areas in the concentrated distribution area of ​​the first indentation mask to obtain a distribution of modeling points includes: determining a second ratio according to a ratio of an area of ​​the concentrated distribution area to an area of ​​the other areas, and determining the number of modeling points in the concentrated distribution area and the other areas respectively according to the second ratio.

[0129] It can be understood that the second ratio is a value calculated based on the ratio of the area of ​​the concentrated distribution area to the area of ​​other areas. It is used to reflect the relative relationship between the concentrated distribution area and other areas in terms of area. The concentrated distribution area is set as the part containing the key features of the target structure, and more modeling points are required to accurately describe its features; while other areas contribute less to the key features of the target structure, and the number of modeling points required is relatively small. By calculating the ratio of the area of ​​the concentrated distribution area to the area of ​​other areas to obtain the second ratio, this relative relationship can be quantified. Since the density of modeling points in the concentrated distribution area needs to be greater, the number of modeling points in the concentrated distribution area should be more than that in other areas under the same area. After the second ratio is determined, the number of modeling points in the two parts at the same density can be determined based on the total number of modeling points and the second ratio. At this time, the number of modeling points in the concentrated distribution area obtained is its lower limit value, and more modeling points than this lower limit value need to be allocated to it during allocation.

[0130] In one embodiment, see Figure 3, obtaining multiple dark-field sub-images corresponding to the bright-field image, and combining the dark-field sub-images to obtain a dark-field image, including steps S302 to S306.

[0131] S302, determining a scanning area according to the bright field image.

[0132] It can be understood that the scanning area is determined based on the boundary of the bright field image, and the dark field image is obtained to obtain more detailed sample information than the bright field image. Determining the scanning area is a key step in the early stage of obtaining the dark field image. The image boundaries of the bright field image and the dark field image are the same, and each dark field sub-image is smaller than the bright field image, so it is necessary to determine the boundary when scanning and collecting the dark field sub-image based on the bright field image, that is, the scanning area.

[0133] S304, controlling the microscope to switch to a higher magnification objective lens, and performing line-by-line scanning according to the scanning area to obtain a plurality of dark field sub-images.

[0134] It can be understood that the microscope here can be the same device as that used to obtain the bright field image, or it can be another one. However, it needs to have a dark field scanning and acquisition function. Compared with the objective lens used for bright field imaging, the objective lens used for collecting dark field sub-images has a higher magnification. Using a higher magnification objective lens can capture more subtle structural features in the sample, thereby obtaining a dark field sub-image that is more refined than the bright field image. However, the field of view of a higher magnification objective lens is relatively small and cannot cover the entire scanning area at one time, so the scanning area needs to be scanned line by line. During the line-by-line scanning process, each scan will acquire a small sub-image until the entire scanning area is covered. The scanning area can be divided into multiple small sub-areas by line-by-line scanning, and the image corresponding to each sub-area is the dark field sub-image. When scanning, it is best to ensure that the collected dark field sub-images have a certain degree of overlap to facilitate subsequent image stitching.

[0135] The switching of the objective lens can be done by the operator manually operating the objective lens switching button of the microscope to switch the objective lens to a higher magnification objective lens. Then, the stage control knob of the microscope is used to move the stage in sequence in the order of line-by-line scanning, so that different sub-areas of the scanning area are in the field of view of the objective lens in turn, and an image acquisition device (such as a CCD camera) is used to capture the image of each sub-area to obtain a dark field sub-image. However, this method requires the operator to have certain operating experience and is relatively inefficient, so it is best to use an interactive teaching system to link with the automatic control software of the microscope, and realize automatic switching of the objective lens and line-by-line scanning operations by writing scripts or setting parameters. The scanning step size is determined by the size of the scanning area and the dark field sub-image, as long as the obtained dark field sub-image can cover the entire scanning area.

[0136] S306 , filling each dark field sub-image into the scanning area according to the coordinate mapping of each dark field sub-image in the scanning area to obtain a dark field image.

[0137] It can be understood that the coordinate mapping reflects the position information of each dark field sub-image in the scanning area, usually expressed in the form of pixel coordinates. During the line-by-line scanning process, the coordinates of a certain point in the scanning area corresponding to each dark field sub-image are recorded, such as the center point, vertex, etc. This coordinate information is the coordinate mapping. The exact position of each dark field sub-image in the final dark field image can be determined by coordinate mapping. After obtaining multiple dark field sub-images through line-by-line scanning, these sub-images need to be combined into a complete dark field image. The coordinate mapping provides each dark field sub-image with position information in the scanning area. Based on this information, each dark field sub-image can be accurately placed at the corresponding position in the scanning area to achieve image splicing and filling. During the filling process, it is generally necessary to process the overlapping parts between the sub-images to ensure that the spliced ​​dark field image is seamless and continuous. The dark field image obtained in this way can reflect the complete fine structure of the sample in the scanning area.

[0138] Specifically, a blank image of the same size as the scanning area can be created as the basis of the dark field image. Each dark field sub-image is traversed, and the pixel values ​​of the dark field sub-image are copied to the corresponding positions of the dark field image according to its coordinate mapping information. For the overlapping parts between sub-images, a simple covering or averaging processing method can be used. Feature extraction and matching algorithms (such as SIFT, SURF, etc.) can also be used to process the overlapping parts between sub-images. First, feature points are extracted in each dark field sub-image, and then feature matching is performed between adjacent sub-images to find the corresponding relationship between them. Based on the results of feature matching, the transformation matrix (such as affine transformation matrix) between sub-images is calculated, the sub-images are transformed and aligned, and then spliced. This method can more accurately process the overlapping parts between sub-images and improve the accuracy of splicing.

[0139] Before filling each dark field sub-image into the scanning area according to the coordinate mapping of each dark field sub-image in the scanning area to obtain the dark field image, the method further includes: determining the brightness difference between adjacent dark field sub-images. If the brightness difference is greater than a first threshold, brightness compensation is performed on adjacent dark field sub-images respectively so that the brightness difference is less than the first threshold.

[0140] It can be understood that in the process of acquiring multiple dark field sub-images by scanning line by line according to the scanning area, two dark field sub-images adjacent in position. For example, when scanning line by line, adjacent sub-images in the same row or the same column belong to adjacent dark field sub-images. It is used to measure the degree of brightness difference between adjacent dark field sub-images. In an image, brightness can usually be represented by the grayscale value of a pixel (for a grayscale image), or reflected by a numerical combination of color channels (such as each channel in the RGB color space) (for a color image). The brightness difference can be determined by calculating the difference in brightness values ​​of pixels at corresponding positions of adjacent sub-images. Common calculation methods include average brightness difference, mean square error, etc.

[0141] In the process of acquiring dark field images, due to the influence of various factors that may exist during the scanning process (such as the stability of the microscope light source, the difference in light transmittance at different positions of the sample, etc.), there may be inconsistent brightness between adjacent dark field sub-images. If this brightness difference is not handled, when the dark field sub-images are combined into a dark field image, it will cause obvious seams or uneven brightness in the spliced ​​image, affecting the quality of the dark field image and the subsequent analysis and processing of the image. Therefore, before performing image filling and splicing, first determine the brightness difference between adjacent dark field sub-images so that corresponding measures can be taken to make adjustments based on the difference.

[0142] The first threshold is a pre-set value, which is used as a standard to determine whether the brightness difference between adjacent dark field sub-images needs to be adjusted. When the calculated brightness difference is greater than the first threshold, it means that the brightness difference between adjacent sub-images is large and brightness compensation is required; when the brightness difference is less than or equal to the first threshold, it is considered that the brightness difference is within an acceptable range and no compensation is required. Brightness compensation refers to adjusting adjacent dark field sub-images with inconsistent brightness to make their brightness closer. Common brightness compensation methods include linear transformation, histogram equalization, etc., which change the brightness of the image by adjusting the pixel value of the image, thereby reducing the brightness difference between adjacent sub-images. It can be to reduce the brightness of the high-brightness image alone, or to increase the brightness of the low-brightness image alone, or to adjust both at the same time.

[0143] In one embodiment, the segmentation model is built based on the U-Net architecture. The U-Net architecture is a semantic segmentation model architecture based on a convolutional neural network (CNN). It consists of a contraction path (downsampling path) and an expansion path (upsampling path). In the contraction path, the image resolution is gradually reduced through continuous convolution and pooling operations to extract the semantic information of the image; in the expansion path, the image resolution is restored through upsampling and feature fusion operations of the corresponding layers of the contraction path, so that the segmentation result can be accurate to the pixel level. The U-Net architecture is widely used in fields such as medical image segmentation and can effectively segment and classify different areas in an image. Please refer to Figure 4 , the left side of the figure shows the detection result obtained by the target detection model, and the right side shows the result obtained by the segmentation model in this embodiment. Compared with the traditional target detection model, the mask output by the Unet architecture can be irregular, further cropping the area not related to the mark.

[0144] In one embodiment, there is a mark on the target slice for highlighting the area where the tissue to be analyzed is located. For details, please refer to Figure 4 The black handwriting part in the image delimits the area where the sliced ​​tissue is located. The training process of the segmentation model includes: obtaining a bright field training image and a dark field training image respectively to obtain a segmentation training set. The annotation of the bright field training image is used to mark the position of the mark, and the annotation of the dark field training image is used to mark the position of the tissue to be analyzed. The initial model is trained using the segmentation training set to obtain a segmentation model.

[0145] It can be understood that the bright field training image is obtained by further annotating the slice image obtained by bright field microscope imaging technology, while the dark field training image is obtained by further annotating the slice image obtained by dark field microscope imaging technology. Figure 5 The difference between the two lies in the different annotation methods. In the bright field image, the marks on the slices are clearly visible, but the difference between the slice tissue and other positions on the slide is not so obvious, so the marks on the slices are used as the basis for segmentation. In the dark field image, the contrast between the slice tissue and the background is higher, and the slice tissue is more prominent. The tissue contour can be used directly as the basis for segmentation. Based on this, the annotation of the bright field training image will identify the location of the above marks, while the annotation of the dark field training image will focus on outlining the contours of the slice tissue. Through this differentiated annotation, the segmentation model can more accurately identify and distinguish slice features under different imaging conditions, thereby learning the ability to segment dark field images and bright field images respectively.

[0146] The initial model is trained with the segmentation training set, which allows the initial model to adjust its own parameters by learning the images and annotation information in the training set, so that it has the ability to accurately segment the target slice image. During the training process, the initial model takes the bright field training image and the dark field training image as input, and extracts the image features through operations such as convolution, pooling, and upsampling within the model. Then, the output of the model is compared with the annotation information, and the loss function (such as the cross entropy loss function, etc.) is calculated to measure the difference between the model prediction result and the true annotation. According to the value of the loss function, the optimization algorithm (such as stochastic gradient descent, Adam optimizer, etc.) is used to adjust the parameters of the model so that the loss function gradually decreases and the model prediction result is closer and closer to the true annotation. Through continuous iterative training, the initial model gradually learns the marked features in the bright field image and the features of the tissue to be analyzed in the dark field image, and finally obtains a segmentation model that can accurately segment the target slice image.

[0147] In one embodiment, in response to the auxiliary question and answer start instruction, the target large model is called to provide the question answer, see Figure 6 , including steps S602 to S608.

[0148] S602, generating restriction prompt words according to the user's teaching objectives.

[0149] It can be understood that the teaching goal is the learning outcome or knowledge acquisition direction that users expect to achieve when conducting microscopic experimental teaching, such as mastering the morphological characteristics of cells in specific courses, chapters, and subjects, the components of microstructures, etc. Restrictive prompt words are generated based on teaching goals. Teaching goals can be compared with slice identification tasks. They are used to constrain the keywords or phrases of the target large model's answer range. It can guide the large model to focus on the problem areas that users are concerned about and avoid answers that deviate from the topic. This step is the key link in connecting user teaching needs with the target large model. By analyzing and understanding the user's teaching goals, extracting key information and converting it into restrictive prompt words, these prompt words as input information can enable the target large model to generate answers in a targeted manner in the subsequent question-and-answer process, effectively solving the problem that the large model's answers are broad and lack specificity, and improving the accuracy and effectiveness of questions and answers, which is an important guarantee for improving teaching efficiency and quality.

[0150] S604: inputting restriction prompt words into the target macro model to limit the question and answer scope of the target macro model.

[0151] It can be understood that the target large model here refers to a large language model that has been trained with a large amount of data and has strong language understanding and generation capabilities. In this solution, the target large model is used to perform knowledge retrieval and answer generation based on the input information, provide users with question answers, and the input accepted by the target large model includes at least two modes: text and image. Before the user starts learning, inputting the restricted prompt words into the target large model can change the attention distribution and knowledge retrieval scope of the large model when generating answers. After receiving the restricted prompt words, the large model will filter and match the knowledge stored in itself based on these prompt words, and only generate answers related to the prompt words, thereby effectively limiting the scope of the large model's questions and answers, avoiding it from generating irrelevant or overly broad answers, improving the accuracy and relevance of the answers, and meeting the needs for accurate knowledge acquisition in teaching scenarios. The target large model here can be completely built and trained by the developer himself. However, in order to improve efficiency and reduce costs, it can also be fine-tuned using an open source pre-trained large language model.

[0152] S606 , in response to the user's annotation selection operation on the target digital slice, determining a corresponding target screenshot.

[0153] It can be understood that the annotation selection operation refers to the behavior of the user marking the area or object of interest on the slice image through an interactive device such as a mouse or touch screen when observing the displayed target digital slice. In addition, the user can also annotate the structural recognition results of the target screenshot through the annotation selection operation, such as when the user determines that a certain tissue structure is an interlobular bile duct. The user's newly created annotations can form a record, and the previous annotations can be retrieved from the historical records when the same slice is studied later. The target screenshot is the image portion containing the content that the user is concerned about, which is captured from the target digital slice image based on the user's annotation selection operation. It is an important basis for the subsequent large model to conduct question and answer analysis.

[0154] This step is to refine the user's focus from the overall target digital slice to a specific local area. Through the user's annotation selection operation, the system can determine the location of the microstructure or feature that the user is interested in, and then extract the corresponding target screenshot to provide accurate data support for the subsequent large model based on specific microscopic areas. Specifically, when the user performs an annotation selection operation on the teaching terminal screen, the system records the coordinate information of the user's operation, and based on this coordinate information, the corresponding area is intercepted from the digital image of the target digital slice to generate a target screenshot.

[0155] S608, using the target large model to conduct question and answer based on the target screenshot and the user's question.

[0156] It can be understood that user questions refer to questions about microstructure, characteristics, principles, etc. raised by users to the system based on their observation of the target digital slices and their own learning needs. It can also be the user's type discrimination and abnormality recognition of the tissue structure corresponding to the target screenshot. Question and answer refers to the process in which the target large model performs knowledge retrieval, reasoning and answer generation based on the input target screenshot and user questions, and interacts with the user to answer questions. The target large model combines the microscopic image information contained in the target screenshot and the text information of the user's question to perform multimodal information fusion processing. By extracting and understanding image features and analyzing text semantics, the large model retrieves relevant information in its knowledge system, performs reasoning and logical judgment, and finally generates accurate and targeted answers to solve users' doubts in microscopic experiment teaching and achieve the purpose of teaching assistance. Since the target large model has been restricted by the restricted prompt words before starting the question and answer, the knowledge that needs to be paid attention to when reasoning and text generation is greatly reduced, which can reduce the computing power and time required by the model and improve the accuracy of question answering.

[0157] In one embodiment, the limiting prompt words are generated according to the teaching objectives of the user, including identifying the course, chapter and subject corresponding to the assignment according to the slices, and determining the experimental field. The limiting prompt words are generated according to the experimental field.

[0158] It can be understood that courses are the largest classification in the learning process, and chapters are smaller knowledge modules divided according to the knowledge system within the course. For example, the anatomy chapter elaborates on the knowledge of human anatomical structure; the material performance chapter of the material science course mainly explains the various performance-related contents of materials. In the medical field, disciplines represent different professional directions and business areas. For example, cardiology is mainly aimed at the diagnosis and treatment of heart-related diseases; neurosurgery focuses on surgical treatment of nervous system diseases. Combining these three choices from coarse to fine can determine the specific direction of this study, and thus obtain the experimental field. For example, in the hematology chapter of the medical course, combined with the needs of the hematology department, the experimental field of blood cell morphology and function research is determined.

[0159] Each slice identification assignment has its corresponding field, which can be specifically divided into courses, chapters and departments. The system can automatically find the corresponding course, chapter and department according to the label of the slice identification assignment, or the user can choose one from multiple set courses as the goal to be learned when starting experimental teaching. The course clarifies the macro-category of knowledge, the chapters are refined into specific knowledge modules, and the departments further clarify the application scenarios and professional directions in the medical field. Through the comprehensive analysis of information from these three dimensions, it is possible to accurately locate the specific knowledge category and determine the corresponding experimental field.

[0160] After determining the experimental field, based on the professional knowledge system, common research problems and experimental focus in this field, generate restriction prompts that can guide the target large model to accurately answer questions. These prompts serve as a constraint to guide the large model to extract information only from the knowledge related to the experimental field during knowledge retrieval and answer generation, avoiding the generation of broad and irrelevant answers, thereby improving the accuracy and pertinence of the large model's answers to questions and meeting the demand for accurate knowledge in teaching scenarios. Further, in some embodiments, the restriction prompts may also include the detection target restrictions selected by the user. The detection target restriction is the condition or range set by the user for a specific detection object in a medical experiment or teaching scenario. For example, if the user is concerned about the detection of a specific disease marker, such as alpha-fetoprotein (AFP) for early screening of liver cancer, then "alpha-fetoprotein detection" is a detection target restriction; or the user focuses on identifying a specific structure in the selected department, such as interlobular bile ducts, interlobular arteries, hepatocytes, etc. in the liver department, these are detection target restrictions.

[0161] In one of the embodiments, the target large model is obtained by fine-tuning the pre-trained large model. The pre-trained large model is a model that is pre-trained on large-scale general data, such as training on massive text data, so that it has strong language understanding and generation capabilities. GPT-4, BERT, etc. are typical pre-trained large models, which have shown excellent performance in multiple tasks of natural language processing. Ordinary pre-trained large models do not focus on the medical field. This application can make the target large model better adapt to specific tasks and needs in the medical field by fine-tuning the target large model.

[0162] In one embodiment, the process of fine-tuning the pre-trained large model includes: fine-tuning the pre-trained large model using a question-and-answer dataset. The question-and-answer dataset includes teaching questions and answers corresponding to different types of tissues in multiple organ systems, and the teaching questions and answers include easy-to-error point prompts corresponding to easy-to-error annotations. It can be understood that it includes teaching questions and answers corresponding to different types of tissues in multiple organ systems, and these questions and answers come from real teaching scenarios, medical research literature, and clinical practice cases. For example, according to the organ system classification, the respiratory system, digestive system, urinary system, etc., taking the respiratory system as an example, the different types of tissues included therein can be trachea, lungs, etc. The teaching questions and answers also include easy-to-error point prompts corresponding to easy-to-error annotations. For example, in the teaching questions and answers for the liver, students often misjudge the interlobular bile duct as an interlobular artery because they lack consideration of the surrounding cytoplasm and nucleus. The easy-to-error point prompts include: Please pay attention to the cytoplasm and nucleus in this structure.

[0163] Specifically, when using LORA technology to fine-tune the pre-trained large model, by inserting a low-rank matrix into the model, only the parameters of these newly added low-rank matrices are trained, while most of the parameters of the pre-trained model remain unchanged. In this way, when fine-tuning using a question-and-answer dataset containing teaching questions and answers corresponding to different types of tissues in multiple organ systems and prompts for easy errors, the model can quickly learn professional knowledge and common easy errors in the medical field, so that after receiving restricted prompt words, the target large model can more accurately retrieve and generate answers from knowledge related to the detection target and experimental field, and can also promptly point out easy-to-error items in the question and answer, improving the user's learning effect.

[0164] In one embodiment, see Figure 7 In the process of using the target large model to conduct question and answer based on the target screenshot and the user's question, it also includes steps S702 to S706.

[0165] S702, determining a corresponding target knowledge graph according to the experimental field. The target knowledge graph includes preset knowledge points corresponding to the experimental field.

[0166] It can be understood that the knowledge graph is a structured knowledge representation form based on a semantic network, which graphically displays the knowledge elements (such as concepts, entities, events, etc.) in the experimental field and their relationships. In this embodiment, the target knowledge graph contains preset knowledge points in the corresponding experimental field, and these knowledge points construct an organic knowledge network in the form of nodes and edges. For example, in the knowledge graph in the field of oncology experiments, knowledge points such as "tumor cells", "gene mutations", and "targeted therapy" are used as nodes, and their associations such as "the relationship between gene mutations in tumor cells and targeted therapy" are used as edges to reflect. Different experimental fields have unique knowledge systems and internal connections. By constructing corresponding target knowledge graphs, complex knowledge can be structured. Determining the target knowledge graph according to the experimental field provides a knowledge framework for accurately matching knowledge points and understanding problems in the subsequent question-and-answer process, so that the system can analyze and process within a specific knowledge range, improving the accuracy and professionalism of questions and answers.

[0167] Specifically, knowledge graph construction tools, such as Neo4j and other graph database platforms, can be used to extract information and semantically annotate a large number of experimental field-related literature, teaching materials, clinical data, etc. to build a target knowledge graph. After the experimental field is determined, the system retrieves the corresponding target knowledge graph from the database storing the knowledge graph according to the pre-set mapping relationship. For example, a unique identifier is assigned to each experimental field in the database, and the target knowledge graph is found from the database through the query instruction match.

[0168] S704, extract keywords from the latest question before inputting it into the target large model, and match the extracted target keywords with each preset knowledge point in the target knowledge graph to determine the target knowledge point.

[0169] It can be understood that the latest question refers to the latest question input by the user in the process of question-answering using the target large model. The target keyword is a word extracted from the latest question that can represent the key information of the question. The target knowledge point is the preset knowledge point in the target knowledge graph that successfully matches the target keyword, and it is the key knowledge to answer the user's question. Through keyword extraction, the core of the user's question can be quickly grasped, and the target keyword can be matched with the preset knowledge point in the target knowledge graph, and the knowledge area related to the question can be located in the knowledge graph. In this way, before the question is input into the target large model, the question is preliminarily located and screened, so that the target large model can determine the specific direction of the user's attention when answering the question. Specifically, the keyword extraction algorithm in natural language processing, such as the TextRank algorithm, can be used to extract the target keyword from the latest question. Then, the target keyword is matched with the node (preset knowledge point) in the target knowledge graph using a string matching algorithm or a semantic similarity calculation method. For example, by calculating the cosine similarity between the target keyword and the name of the knowledge graph node, when the similarity exceeds a certain threshold, it is considered that the match is successful and the target knowledge point is determined.

[0170] S706, for any target knowledge point, update the mastery level of the target knowledge point according to the user's question and answer performance.

[0171] It can be understood that the question-answering performance refers to the performance of the user's answers to the questions raised by the target large model, the tasks issued, etc. in the dialogue triggered by the latest question. The user's question-answering performance can be used to determine the user's familiarity and understanding of the target knowledge point, and the mastery level can be used to quantify it. By continuously updating the mastery level of the target knowledge point, the system can track the user's learning progress and knowledge weaknesses, provide a basis for subsequent personalized teaching and targeted tutoring, and achieve dynamic optimization of the teaching process.

[0172] Specifically, the current mastery of the target knowledge point can be determined based on at least one of the user's answer accuracy, the number of follow-up questions, and the annotation modification record. The answer accuracy refers to the accuracy of the user's answer to the target question, which is obtained by comparing the answer content with the standard answer or authoritative knowledge source. The number of follow-up questions refers to the number of times the user further asks related questions after receiving the answer from the target large model. The annotation modification record is a record of the user's modification of the existing annotations in operations related to the target knowledge point, such as image annotation. For example, in medical image analysis, the modification of the tumor area annotation reflects the change in the user's understanding of the relevant knowledge. The answer accuracy directly reflects the fit between the user's answer and the correct knowledge, and can reflect the user's current cognitive accuracy of the knowledge point; the number of follow-up questions indicates the user's satisfaction with the answer and the further demand for knowledge depth. The more follow-up questions, the less the user's mastery of the current knowledge point; the annotation modification record reflects the dynamic changes in the user's understanding of the knowledge point from the actual operation level. By comprehensively considering these factors, the user's current mastery of the target knowledge point can be more comprehensively and accurately evaluated, providing a reliable basis for subsequent updates to the mastery level. After these evaluation factors are quantified, they can be weighted and summed to obtain the user's mastery of the target knowledge point in the current conversation.

[0173] Since the user's mastery of knowledge is a gradual and dynamic process, the user's past mastery of the target knowledge point is recorded in the historical mastery index. After obtaining the current mastery, by weighting the current mastery and the historical mastery, it is possible to comprehensively consider the changes in the user's mastery of knowledge points at different stages and more accurately describe the user's learning trajectory and knowledge mastery level. The more recent mastery reflects the user's current learning status, and giving it a higher weight can timely reflect the user's learning progress; while the historical mastery reflects the user's learning foundation and knowledge accumulation process. The combination of the two can make the updated mastery more stable and reliable, providing a more scientific basis for personalized teaching and tutoring.

[0174] In one embodiment, in the process of using the target macro model to conduct question-answering based on the target screenshot and the user's question, the process further includes: if the mastery level of any target knowledge point is lower than a first threshold, determining the knowledge blind spot corresponding to the target knowledge point. Adding guiding prompt words for each knowledge blind spot to the latest question. The guiding prompt words are used to instruct the target macro model to guide the user to ask questions about the knowledge blind spot when answering.

[0175] It can be understood that the first threshold is a pre-set measurement standard used to determine whether the user's mastery of the target knowledge point meets the basic requirements. When the mastery of the target knowledge point is lower than the first threshold, the user has not yet understood or has not thoroughly understood the knowledge area corresponding to the knowledge point. For example, in the knowledge point of "Structure and Function of Heart Valve", if the user has a low mastery, then the specific opening and closing mechanism of the heart valve, the physiological effects of different valve lesions, and other contents may be knowledge blind spots. After the user completes a question and answer session, the system automatically compares the mastery of the target knowledge point with the first threshold, which can be implemented using conditional judgment statements. When determining to directly use the first threshold to determine the knowledge blind spot, the sub-knowledge points related to the target knowledge point but not mastered by the user can also be extracted from the target knowledge graph as the target knowledge blind spot. For different preset knowledge points, a dynamic threshold setting method can be used to adaptively adjust the first threshold according to factors such as the user's learning ability, learning progress, and difficulty of the experimental field. For example, as the user's learning time increases, the size of the first threshold gradually increases the knowledge mastery requirements for the user.

[0176] Guiding prompts are keywords or phrases specially designed to guide users to pay attention to knowledge blind spots and ask questions. For example, when the knowledge blind spot is "the relationship between the liver lobule structure and the portal area", the guiding prompt can be "Do you need to know the relationship between the liver lobule structure and the portal area" and so on. In addition, further divisions can be made for target knowledge points whose mastery level is lower than the first threshold. For example, for target knowledge points whose mastery level is lower than the first threshold but higher than the second threshold, the first level of detail restrictor can be added to the latest question sentence to instruct the large language model to answer the question in the first mode, and the first mode needs to include basic definitions and text descriptions of misunderstandings. Users have a certain degree of mastery of such target knowledge points, and can instruct the large language model to simplify the answer. For target knowledge points whose mastery level is lower than the second threshold, the second level of detail restrictor can be added to the latest question sentence to instruct the large language model to answer the question in the second mode, and the second mode needs to include basic definitions, graphic descriptions of misunderstandings, and typical error cases. Users have a poor grasp of such target knowledge points, and need to instruct the large language model to answer in a more detailed mode to help students improve their mastery of the target knowledge points as soon as possible.

[0177] In one of the embodiments, the process of completing the slice identification operation further includes steps S802 to S810.

[0178] S802: In response to the auxiliary recognition start instruction, determine a region to be recognized on the target digital slice.

[0179] It can be understood that the area to be identified is a specific area determined on the target digital slice that needs to be further identified and analyzed. It is generally the area selected by the user when completing the assignment based on the tissue structure required to be found in the assignment. Specifically, an auxiliary identification start button can be set in the teaching software or system, and the user can issue an auxiliary identification start instruction by clicking the button. After receiving the instruction, the system provides a graphical interface. When the user encounters a difficult-to-identify tissue area, he can use the mouse or touch operation to select the area to be identified on the image of the target digital slice. The system will record the coordinate information of the user's selection, etc., to determine the area to be identified.

[0180] S804: Input the screenshot corresponding to the area to be identified into the area detection model, and select the identification results with the highest confidence among the multiple identification results as optional results. The area detection model is used to identify the category of the input image to obtain multiple identification results and corresponding confidence levels.

[0181] It can be understood that the screenshot corresponding to the area to be identified refers to the image portion captured from the overall image of the target digital slice according to the area to be identified determined in step S702, which contains the key information to be identified. The regional detection model is a model based on artificial intelligence technology. After training, it can analyze the input image, identify the category of the image, and give a variety of possible recognition results and the confidence corresponding to each result. The recognition result is the judgment of which tissue category the image belongs to after the regional detection model analyzes the input image. Confidence is a numerical indicator to measure the reliability of the recognition result. The higher the value, the higher the degree of certainty of the model for the recognition result. The first number is a pre-set parameter used to determine a certain number of results with higher confidence from multiple recognition results as optional results.

[0182] After determining the screenshot corresponding to the area to be identified, it is input into the region detection model to use the powerful image recognition ability of the model to analyze the area to be identified. The region detection model can identify the features of different images and match them with known categories by learning and training a large amount of known image data. After obtaining multiple recognition results and corresponding confidence levels, the recognition results with the first number of confidence levels are selected as optional results in order to provide users with some options with higher probability so that users can choose among these options while avoiding providing too many results that make it difficult for users to make a decision. For example, when the region detection model analyzes a tissue slice image of an area to be identified, it may identify that the area may be an interlobular bile duct, interlobular artery, interlobular vein, etc. The region detection model will also obtain the confidence level of each identified category.

[0183] The region detection model can be specifically obtained by training based on the UNET network architecture, which is a convolutional neural network architecture widely used in the field of medical image segmentation and has a symmetrical encoder-decoder structure. The encoder part is used to downsample the input image and extract the high-level semantic features of the image. The decoder part performs upsampling and restores the feature map to the same size as the input image to achieve pixel-level classification. In this region detection model, it is used to classify the region to be identified in the medical tissue slice image. The training data can be obtained by annotating a large number of tissue slices for training. In order to enable the model to output multiple classification results and confidence levels, a corresponding number of neurons are set in the output layer according to the specific number of categories. For example, if there are 10 categories of tissue structures to be identified, a total of 10 neurons are set, corresponding to these categories one by one. During the training process, the output of these neurons is supervised by a suitable loss function (such as a cross entropy loss function) so that they can learn the distinguishing features between different categories. During inference, the output values ​​of these neurons are processed by an activation function (such as the softmax function, which is often used in multi-classification tasks to convert the output values ​​into a probability form, i.e., confidence) to obtain the confidence corresponding to each category, thereby outputting recognition results of multiple categories and their corresponding confidences.

[0184] S806: Output optional results to the user for the user to select.

[0185] It can be understood that the output of optional results to the user is to allow the user to participate in the slice identification teaching process, and to achieve the interactivity of teaching and the user's active learning through the user's selection of optional results. In traditional teaching, the teacher explains in a one-way manner, and the students lack participation and initiative. In this step, the user can analyze and judge the optional results based on his or her observation of the target digital slice and existing knowledge, and choose the result he or she thinks is correct. This method can not only test the user's mastery of knowledge, but also allow the user to further think and learn during the selection process, thereby improving the teaching effect. For example, when the user is faced with multiple optional tissue type identification results, it is necessary to compare the actual characteristics of the target digital slice and make a choice based on his or her own knowledge, so as to deepen the understanding of the characteristics of different tissue types. Specifically, the optional results can be displayed through the user interface of the teaching software or system. For example, the optional results are presented in the form of a list or tab on the interface, and the user can make a selection by clicking the corresponding option. At the same time, some brief descriptions or prompts can be provided next to the optional results to help users better understand the meaning of each result.

[0186] S808: Determine the result selected by the user as the first result, and the result with the highest confidence among the optional results as the second result.

[0187] It can be understood that the first result refers to the result selected by the user from the optional results in step S806, which represents the user's judgment on the target digital slice area to be identified. The second result is the recognition result with the highest confidence among the optional results, which is the most likely result obtained by the area detection model based on its algorithm and training data. Specifically, after the user makes a selection, the system will record the first result selected by the user. At the same time, the system will automatically obtain the result with the highest confidence from the optional results as the second result.

[0188] S810: If the first result is the same as the second result, it is determined that the user's selection is correct.

[0189] It can be understood that if the first result is the same as the second result, the user's choice is judged to be correct. This step is the key link for evaluating and giving feedback on the user's choice in the entire microscopic experiment teaching interaction. Through this comparison and judgment method, the user can be provided with information about whether his choice is correct in a timely manner, helping the user to understand his learning situation and the degree of mastery of knowledge. In the teaching of medical morphology, timely and accurate feedback is very important for the user's learning and improvement. If the user's choice is correct, it means that the user's understanding and application of the relevant knowledge is correct, and he can continue to study more deeply and continue to complete the slice identification homework. In addition, before the student submits the homework to the teacher, the screenshot corresponding to the answer area circled by the student can be input into the regional detection model, and the answer with the highest confidence output by the regional detection model can be compared with the user's answer, and the comparison results can be packaged together and uploaded into the homework. In this way, the teacher can refer to the comparison results to make corrections faster.

[0190] In one embodiment, if the first result is the same as the second result, it is determined that the user's selection is correct, and it also includes: if the first result is different from the second result and the maximum deviation value of the confidence between the optional results is less than a first threshold, all optional results are output to the user.

[0191] It can be understood that the maximum deviation value refers to the difference between the maximum and minimum values ​​in the confidence of all optional results. The first threshold is a pre-set numerical standard for judgment. The first threshold can be adjusted with the user's level label. The setting of the level label can be based on the user's actual operation ability and theoretical knowledge mastery level. For beginners, lower the first threshold so that they can be exposed to more results for learning and comparison. For experienced users, increase the first threshold so that they can face more challenging situations and exercise their judgment ability. When the first result is different from the second result, it means that there is a difference between the user's judgment and the most likely result considered by the model. At this time, the maximum deviation value of the confidence between the optional results is further considered. If the value is less than the first threshold, it means that the confidence of these optional results is relatively close, and no result has an absolute advantage. In this case, outputting all optional results to the user is to enable the user to fully understand the results with higher probability given by the model, encourage the user to think and analyze again, and make more accurate judgments based on their own observations and knowledge, so as to avoid ignoring other potential possibilities because only seeing the results with the highest confidence, thereby improving the user's understanding and mastery of slice identification knowledge, and also improving the comprehensiveness and accuracy of teaching.

[0192] If the first result is different from the second result, and the confidence deviation value between the optional results is greater than or equal to the first threshold, the optional result with the highest confidence is output to the user.

[0193] It can be understood that when the first result is different from the second result, and the confidence deviation value between the optional results is greater than or equal to the first threshold, it indicates that among these optional results, the result with the highest confidence has a greater advantage over other results, and the model is relatively certain about the result. At this time, the optional result with the highest confidence is output to the user based on the judgment of the model, which believes that this result is most likely to be correct. Doing so can avoid confusion when users are faced with multiple results, especially when the model already has a clearer tendency, guiding users to focus on the most likely correct answer, helping users correct wrong judgments, and quickly mastering the correct knowledge of slice identification, thereby improving the efficiency and pertinence of teaching.

[0194] In one embodiment, if the first result is the same as the second result, it is determined that the user's selection is correct, and it also includes: if the confidence corresponding to the target result is lower than the second threshold, the optional result, the target result and the area to be identified are sent to the teacher's end. Re-judge whether the user's selection is correct based on the feedback from the teacher's end. It can be understood that the second threshold is a numerical standard for measuring confidence. The teacher's end is a terminal device or platform used by the teacher to receive information and perform feedback operations, such as teaching management software on the computer end. When the first result is the same as the second result, it is preliminarily determined from the perspective of the model that the user's selection is correct. This is based on the model's recognition of high-confidence results, and it is believed that the user's selection is consistent with the most likely result considered by the model. However, when the confidence corresponding to the target result (i.e., the result selected by the user) is lower than the second threshold, it means that although the user has selected the highest confidence result considered by the model, the model's certainty of the result is not high. In this case, the optional results, the target result and the area to be identified are sent to the teacher's end because the teacher has more professional knowledge and experience and can conduct a more in-depth analysis and judgment of this information. The teacher can give more accurate feedback on whether the user's selection is correct based on his or her professional knowledge and actual situation. Re-judging whether the user's choice is correct based on the feedback from the teacher side can make up for the limitations of the model in some cases, improve the accuracy and reliability of the judgment, make the teaching evaluation more objective and comprehensive, and help users understand their learning situation and knowledge mastery more accurately. After receiving this information on the teacher side, the teacher enters the feedback information on the teacher side interface through viewing and analysis, such as judging whether the user's choice is correct or wrong, and can attach relevant explanations and instructions. After receiving the feedback information from the teacher side, the system re-judgments whether the user's choice is correct based on the feedback, and records the results in the system. At the same time, relevant prompts can be displayed on the user side interface to inform the user of the final judgment result and the teacher's feedback information.

[0195] In one embodiment, after the slice identification job is issued, in response to the auxiliary identification start instruction, the area to be identified on the target digital slice is determined, and the process also includes: recording the number of times the auxiliary identification start instruction is triggered during this slice identification job. During the slice identification job, recording the number of times the auxiliary identification start instruction is triggered can reflect the degree of students' reliance on auxiliary help when completing the job. If the number of triggers is large, it may mean that students have difficulty understanding and identifying slices and need more auxiliary support.

[0196] Output optional results to the user for selection, see Fig. 9 , also includes steps S902 to S906.

[0197] S902: If the number of triggering times is greater than the third threshold, the corresponding first test question is determined according to the slice identification operation and displayed.

[0198] It can be understood that the third threshold is a numerical standard for comparison with the number of triggers to determine whether subsequent special processing is required. The first test question is a related test question automatically generated according to the current slice identification homework content or retrieved from the question bank, which is used to examine the student's mastery of the knowledge involved in the homework. The correct rate is the proportion of students answering the first test question correctly, that is, the ratio of the number of correct answers to the total number of questions. The fourth threshold is a numerical standard for measuring the correct rate of the first test question to determine whether to output optional results to the user. When the number of triggers of the auxiliary recognition start instruction is greater than the third threshold, it means that the user frequently seeks auxiliary help during the homework process, which may imply that the user is not proficient in the knowledge or skills involved in the homework and has difficulty in understanding. In order to more accurately understand the user's knowledge mastery, the corresponding first test question is determined and displayed according to the specific content of the slice identification homework. Through the test questions, the user's understanding and application ability of key knowledge points can be targeted, so that more appropriate teaching guidance and support can be provided according to the test results later. For example, if the user frequently triggers assisted teaching in multiple slice identification tasks, it may indicate that he or she has problems identifying the basic features of tissue slices. In this case, the first test question can be designed around the identification and judgment of these basic features.

[0199] S904: When the accuracy rate of the first test question is greater than a fourth threshold, output optional results to the user for the user to select.

[0200] It can be understood that the fourth threshold is a reference value for judging whether the accuracy of the first test question reaches a certain level. When the user completes the first test question, the accuracy is calculated. If the accuracy is greater than the fourth threshold, it means that the user has a good understanding and mastery of the relevant knowledge involved in the slice identification operation, and has the basis for the next step (i.e., making a selection based on the optional results). At this time, outputting optional results to the user and allowing the user to choose from these results with higher probability can further test the user's ability to apply knowledge. It also conforms to the principle of gradual progress in teaching. After the user has a certain knowledge base, he is given the opportunity to practice and apply. For example, if the user has a high accuracy rate in the first test question about a certain pathological tissue slice, it means that he has a good grasp of the knowledge of the characteristics of the pathological tissue, so outputting optional results for him to make a selection will help consolidate and deepen his understanding of this part of knowledge.

[0201] S906: Otherwise, output the knowledge point analysis corresponding to the slice identification operation to the user.

[0202] It can be understood that the knowledge point analysis is a detailed explanation and description of the relevant knowledge content involved in the slice identification operation, including knowledge about the characteristics of tissue slices, classification methods, and basis for lesion judgment, etc., which is intended to help users understand and master these knowledge points. When the user's correct rate in the first test question is not greater than the fourth threshold, it means that the user's knowledge of the slice identification operation is not solid enough, and there are deficiencies or errors in understanding. At this time, the knowledge point analysis corresponding to the slice identification operation is output to the user to help the user fill in the knowledge gaps and relearn and understand the relevant knowledge. Through detailed knowledge point analysis, users can have a deeper understanding of the key content and key points of the slice identification operation, laying a solid foundation for subsequent slice identification or related tests. For example, if the user has a low correct rate in the first test question about a certain normal tissue and pathological tissue slice, the output knowledge point analysis can explain in detail the characteristic differences and identification methods of the two tissues, helping users correct their misunderstandings and improve their identification capabilities.

[0203] In one embodiment, the knowledge point analysis corresponding to the slice identification operation is output to the user. Fig.10 , including steps S1002 to S1006.

[0204] S1002, displaying the knowledge point analysis through an interactive pop-up window, and activating the test start control after a first set time.

[0205] It can be understood that the interactive pop-up window is a window that pops up on the user operation interface. It is interactive and the user can perform operations in the window, such as viewing information, inputting content, etc. In this scenario, it can at least be used to show the user the knowledge point analysis corresponding to the slice identification job. The first set time is the length of time that forces the user to read the knowledge point analysis. Only when the user stays on the interface that currently displays the knowledge point analysis for a long enough time to ensure that the user has read the knowledge point analysis, can the test start control be activated, and the user can start the test of the degree of understanding of the knowledge point analysis through the test start control. The first set time corresponding to different knowledge point analyses can be flexibly changed according to the actual reading time. For example, for the same knowledge point analysis, the time from the interactive pop-up window popping up to the actual click to activate the test start control for all users is counted, and the average of these collected times is calculated to obtain the corresponding first set time.

[0206] S1004, in response to the operation on the test start control, displaying a second test question corresponding to the knowledge point analysis to the user.

[0207] It can be understood that the operation of the test start control refers to the triggering action performed by the user on the test start control, such as clicking a button, etc., indicating that the user wants to start a test related to the knowledge point analysis. The second test question is a test question specially designed according to the content of the knowledge point analysis, which is used to test the user's understanding and mastery of the knowledge point, and to check whether the user can correctly apply the knowledge after learning the knowledge point analysis. This step is to start the test link by operating the test start control after the user completes the learning of the knowledge point analysis. By displaying the second test question, it can timely feedback the user's mastery of the knowledge point, allowing the user to find out the problems he has in the learning process, and also provide a basis for subsequent learning adjustments. This step closely combines learning and testing to promote the user's consolidation and application of knowledge. The second test question can be a second test question corresponding to each knowledge point analysis that is pre-stored in the system's question bank. The system selects the corresponding second test question from the question bank based on the currently displayed knowledge point analysis content, and displays it to the user in an appropriate manner (such as a list form, a question and answer form, etc.).

[0208] S1006, when the user answers the second number of second test questions correctly, close the interactive pop-up window, otherwise, return to the step of displaying the knowledge point analysis through the interactive pop-up window and activating the test start control after the first set time.

[0209] It can be understood that the second number is a pre-set standard for the number of correct answers to questions, which is used to determine whether the user has passed the test of this knowledge point analysis. The principle of this step is to determine whether the user has mastered the content of the knowledge point analysis based on the test results. If the number of questions answered correctly by the user reaches the second number, it means that the user has a good understanding and mastery of the knowledge point, and the interactive pop-up window can be closed to end the learning and testing of this knowledge point. Otherwise, it means that the user has not fully mastered the knowledge point and needs to re-learn the knowledge point analysis. By returning to step S1002, the user is allowed to read and understand the knowledge point again, and then tested again until the standard is reached, ensuring that the user truly masters the relevant knowledge. Specifically, after the user completes the answer to the second test question, the system will automatically correct the user's answer, compare the user's answer with the pre-set correct answer, and count the number of questions answered correctly. If the number of questions answered correctly reaches the second number, the system will call the function of closing the pop-up window, remove the interactive pop-up window from the user interface, and further display the optional results to the user, and provide the auxiliary teaching function for the user again. If the second number is not reached, the system will call the function of step S902 again, display the knowledge point analysis through the interactive pop-up window again, and restart the timing to activate the test start control after the first set time.

[0210] In one embodiment, inputting a screenshot corresponding to the area to be identified into the area detection model includes: determining a center point of the area to be identified, and forming a screenshot area larger than and including the area to be identified according to the center point. Taking a screenshot on the target digital slice according to the screenshot area, and inputting the screenshot result into the area detection model.

[0211] It can be understood that the center point is a point at the center of the area to be identified obtained by a specific calculation method, and it serves as a key reference element for the subsequent construction of the screenshot area. If the area to be identified is a rectangular frame, the horizontal coordinate of the center point is the midpoint of the horizontal coordinates of the left and right vertices, and the vertical coordinate is the midpoint of the vertical coordinates of the upper and lower vertices. The screenshot area is a new area range determined based on the center point of the area to be identified. This area is larger than the area to be identified and completely covers the area to be identified. Its function is to intercept the image part containing the information related to the area to be identified and its surroundings from the target digital slice. After determining the area to be identified, finding its center point is to more accurately construct a screenshot area containing the area to be identified. As a stable reference point, the center point can ensure that the constructed screenshot area covers the area to be identified and its surroundings in a reasonable way. The screenshot area larger than the area to be identified is formed because in the analysis of medical tissue slices, the tissue information around the area to be identified may play an auxiliary role in accurate identification. For example, the edge features of some lesion tissues may be related to the transition of the surrounding normal tissues. Including more surrounding information can provide richer context features for the regional detection model, helping the model to more accurately determine the category of the area to be identified. The screenshot area may be a rectangular area with the center point as the center, which is longer and wider than the original area to be identified.

[0212] In one embodiment, the interactive method for microscopic experiment teaching further includes: if the first result does not match the second result, and the difference between the confidence level corresponding to the first result and the second result is lower than a fifth threshold, the first result is used as the target category and the screenshot corresponding to the area to be identified is used as a reference image to send to the image generation model to obtain a confusing image. The image generation model is used to generate a confusing image whose recognition result is the target category based on the reference image. The first result and the confusing image are presented to the user as a group, and the second result and the screenshot corresponding to the area to be identified are presented as another group.

[0213] It can be understood that the fifth threshold is a reference value for judging the confidence difference. When the confidence difference between the first result and the second result is less than the threshold, the subsequent operation will be triggered. The target category refers to the category represented by the first result as a specific target, which is used as the basis for the subsequent image generation model to generate images. The image generation model is an artificial intelligence-based model that can generate confusable images with specific recognition results (target categories in this scenario) based on the input reference image after training. A confusable image refers to a generated image that has a certain similarity with the reference image in visual features or recognition results, and is easily confused with the recognition results corresponding to the reference image.

[0214] When the first result does not match the second result and the confidence difference between the two is small, it means that there is a difference between the user's judgment and the most likely result considered by the model, but the model's certainty of the two results is not much different. In this case, the first result is sent to the image generation model as the target category and the screenshot corresponding to the area to be identified as the reference image in order to generate a confusing image that is similar to the reference image but the recognition result is the target category. The purpose of this is to allow users to more intuitively observe and understand the subtle differences between different categories by comparing the confusing image and the original screenshot of the area to be identified, thereby improving the user's knowledge of slice identification.

[0215] The training set used in the image generation model training includes multiple first images of all tissue categories, and each image is matched with a corresponding second image, which is an image of a tissue structure that is often confused with the corresponding first image. Then, a generative adversarial network (GAN)-based architecture is selected, such as DCGAN (deep convolutional generative adversarial network), WGAN (Wasserstein generative adversarial network), etc. The categories of the first image and the second image are used as input, and the loss function is obtained according to the similarity between the image generated by the model and the second image. The parameters of the model are adjusted to reduce the direction of the loss function, thereby obtaining an image generation model that can generate easily confused images.

[0216] By combining the first result with the easily confused image, and the second result with the screenshot corresponding to the area to be identified, and displaying them together to the user, an intuitive comparison environment is provided for the user. By observing and comparing these two sets of information, the user can more clearly see the difference between the result they selected (the first result) and the most likely result considered by the model (the second result), as well as the difference in visual features between the easily confused image and the original screenshot of the area to be identified. This comparative display method helps users deeply understand the subtle differences between different categories, analyze the reasons for their own misjudgments, and thus improve their ability to identify slices.

[0217] In one embodiment, before presenting the first result and the easily confused image as one group and the second result and the screenshot corresponding to the area to be identified as another group to the user, the method further includes: determining a distinguishing mark structure according to the first result, the second result and the first mapping relationship. The distinguishing mark structure and the easily confused image are input into the annotation model to annotate the distinguishing mark structure in the easily confused image.

[0218] It can be understood that the first mapping relationship is a pre-established correspondence relationship, which associates different recognition results (such as tissue categories, etc.) with specific marker distinguishing structures. The marker distinguishing structure refers to the key structure or feature that can reflect the difference between different tissue categories in the medical tissue slice image, such as specific cell morphology, tissue structure arrangement, etc. When the first result is different from the second result, the marker distinguishing structure corresponding to the two results can be determined through the first mapping relationship. These marker distinguishing structures are an important basis for distinguishing different tissue categories. Learning how to find them helps users understand the differences between different results more clearly, thereby improving the accuracy of slice identification.

[0219] Specifically, it can be a database that pre-establishes and stores the first mapping relationship in the teaching software system. The database can exist in the form of a table, and each row of records is used to record two recognition results and the mark distinguishing structural information used to distinguish the two recognition results. After obtaining the first result and the second result, the system queries the mark distinguishing structure corresponding to the two results from the database. For example, if the first result is "normal liver cell tissue" and the second result is "early liver cancer cell tissue", the system searches the database for the mark distinguishing structure corresponding to the two tissue categories, such as the regular arrangement structure of normal liver cells and the structural characteristics of early liver cancer cells such as nuclear enlargement and irregular morphology.

[0220] The annotation model is a model based on artificial intelligence technology. After training, it can accurately mark these structures on the image according to the input image and related structural information. Using the image understanding and annotation capabilities of the annotation model, the landmark distinguishing structures determined in the above steps are marked in the easily confused image. Through annotation, users can more intuitively see the key structural features corresponding to the first result in the easily confused image, which helps users compare the easily confused image and the original screenshot of the area to be identified, and analyze the differences between different results. By learning a large amount of image and annotation data, the annotation model has mastered the characteristics and position information of different structures in the image, and can accurately mark the target structure in the input image. This step cooperates with the previous steps of determining the landmark distinguishing structure and generating the easily confused image to form a complete information processing and display process, providing users with clearer and more intuitive learning and identification materials.

[0221] The annotation model can be a deep learning-based target detection model architecture, such as Faster R-CNN, YOLO, etc., as the basis of the annotation model. These models perform well in target detection tasks and can quickly and accurately detect target objects in images (in this scenario, landmarks that distinguish structures). The dataset for training the annotation model can be a large number of medical tissue slice images collected and various landmarks that distinguish structures in these images are annotated. The annotation information includes the location of the structure (such as bounding box coordinates) and the category (such as cell nucleus, intercellular matrix, etc.). The preprocessed image and annotation information are input into the annotation model for training. During the training process, the model calculates the loss value between the predicted result and the true annotation based on the input image and annotation information, and updates the model parameters through the back propagation algorithm to minimize the loss value. The performance of the model is regularly evaluated on the validation set, and the changes in indicators such as loss value and detection accuracy are observed, and the training parameters are adjusted according to the evaluation results.

[0222] The present application provides a microscopic experiment teaching interactive device, which is applied to the student end of an interactive teaching system. The microscopic experiment teaching interactive device includes a digital slice acquisition module, a course interaction module and an after-class homework module. The digital slice acquisition module is used to acquire the target digital slice. The course interaction module is used to synchronize the user's target operation on the target digital slice to the teacher end in response to the course start instruction. The after-class homework module is used to respond to the after-class homework start instruction to display the slice identification homework to the students. In the process of completing the slice identification homework, in response to the auxiliary question and answer start instruction, the target large model is called to provide answers to the questions, and in response to the after-class homework submission instruction, the slice identification homework is synchronized to the teacher end.

[0223] For the specific definition of the interactive device for microscopic experiment teaching, please refer to the definition of the interactive method for microscopic experiment teaching above, which will not be repeated here. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0224] The present application provides a computer device, including one or more processors and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the microscopic experiment teaching interactive method in any of the above embodiments are executed.

[0225] Indicatively, Fig.11 As shown, Fig.11 A schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. Fig.11The computer device 1100 includes a processing component 1102, which further includes one or more processors, and a memory resource represented by a memory 1101, for storing instructions that can be executed by the processing component 1102, such as an application. The application stored in the memory 1101 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1102 is configured to execute instructions to perform the microscopic experiment teaching interactive method of any of the above embodiments.

[0226] The computer device 1100 may further include a power supply component 1103 configured to perform power management of the computer device 1100, a wired or wireless network interface 1104 configured to connect the computer device 1100 to a network, and an input / output (I / O) interface 1105. The computer device 1100 may operate based on an operating system stored in the memory 1101, such as Windows Server TM, Mac OS X TM, Unix TM, Linux TM, Free BSD TM, or the like.

[0227] Those skilled in the art will understand that Fig.11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0228] The present application provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the microscopic experiment teaching interactive method in any of the above-mentioned embodiments.

[0229] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0230] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

[0231] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A microscopic experiment teaching interactive method, characterized in that: Applied to the student end of the interactive teaching system, the microscopic experiment teaching interactive method includes: Obtain target digital slices; In response to a course start instruction, synchronizing the user's target operation on the target digital slice to the teacher's end; In response to an assignment start instruction, presenting the slice identification assignment to the student; In the process of completing the slice identification operation, in response to the auxiliary question and answer start instruction, the target large model is called to provide question answers; In response to the after-class homework submission instruction, the slice identification homework is synchronized to the teacher's end.

2. The interactive method for microscopic experiment teaching according to claim 1, characterized in that: The step of obtaining a target digital slice comprises: Acquire a bright-field image of the target slice; Determine the target modeling strategy; If the target modeling strategy is bright field segmentation, the bright field image is input into a segmentation model to obtain a first segmentation mask, the first segmentation mask is range-indented to obtain a first indentation mask, and then the modeling point distribution is determined according to the first indentation mask; If the target modeling strategy is dark field segmentation, a plurality of dark field sub-images corresponding to the bright field image are obtained, and the dark field sub-images are combined to obtain a dark field image; the dark field sub-images are obtained by sequentially scanning the slice areas corresponding to the bright field image using a higher magnification objective lens; Inputting the dark field image into the segmentation model to obtain a second segmentation mask, and determining the modeling point distribution according to the second segmentation mask; Modeling is performed according to the distribution of the modeling points to obtain the target digital slice.

3. The interactive method for microscopic experiment teaching according to claim 2, characterized in that: The step of acquiring a plurality of dark-field sub-images corresponding to the bright-field image and combining the dark-field sub-images to obtain a dark-field image comprises: determining a scanning area according to the bright field image; Controlling the microscope to switch to a higher magnification objective lens, and performing line-by-line scanning according to the scanning area to obtain a plurality of the dark field sub-images; According to the coordinate mapping of each dark field sub-image in the scanning area, each dark field sub-image is filled into the scanning area to obtain the dark field image.

4. The interactive method for microscopic experiment teaching according to claim 2, characterized in that: The determining of the distribution of modeling points according to the first indentation mask comprises: Determining a concentrated distribution area according to the center of the first indentation mask; Modeling points with a higher density than other areas are evenly distributed in the concentrated distribution area of ​​the first indentation mask to obtain the modeling point distribution.

5. The interactive method for microscopic experiment teaching according to claim 1, characterized in that: The target operation includes a questioning operation and an answering operation, and in response to the course start instruction, synchronizing the user's target operation on the target digital slice to the teacher's end includes: In response to a user's question operation on the target digital slice, determining a question area on the target digital slice, and synchronizing the question content and a screenshot corresponding to the question area to the teacher terminal; Receive the first recognition target sent by the teacher end, and in response to the user's question and answer operation on the target digital slice, determine the question and answer area on the target digital slice, and synchronize the screenshot corresponding to the question and answer area to the teacher end.

6. The interactive method for microscopic experiment teaching according to claim 1, characterized in that: In response to the auxiliary question and answer start instruction, calling the target large model to provide question answers includes: Generate restriction prompt words according to the user's teaching objectives; Inputting the restriction prompt words into the target macro model to limit the question and answer scope of the target macro model; In response to a user's annotation selection operation on the target digital slice, determining a corresponding target screenshot; The target large model is used to conduct question and answer based on the target screenshot and the user's question.

7. The interactive method for microscopic experiment teaching according to claim 6, characterized in that: The step of generating restriction prompt words according to the user's teaching objectives includes: Determine the experimental field based on the course, chapter and subject corresponding to the slice identification assignment; The restriction prompt words are generated according to the experimental field.

8. The interactive method for microscopic experiment teaching according to claim 7, characterized in that: In the process of using the target macro model to conduct question-answering according to the target screenshot and the user's question, it also includes: Determine a corresponding target knowledge graph according to the experimental field; the target knowledge graph includes preset knowledge points corresponding to the experimental field; Extract keywords from the latest question before inputting it into the target macro model, and match the extracted target keywords with each of the preset knowledge points in the target knowledge graph to determine the target knowledge points; For any of the target knowledge points, the mastery level of the target knowledge point is updated according to the question-answering performance of the user.

9. The interactive method for microscopic experiment teaching according to claim 6, characterized in that: The process of completing the slice identification operation also includes: In response to the auxiliary recognition start instruction, determining a to-be-recognized area on the target digital slice; Inputting the screenshot corresponding to the area to be identified into the area detection model, and taking the identification results with the first number of confidences among the multiple identification results as optional results; the area detection model is used to identify the category of the input image to obtain multiple identification results and corresponding confidences; Outputting the optional results to the user for the user to select; Determine the result selected by the user as the first result, and the one with the highest confidence among the optional results as the second result; If the first result is the same as the second result, it is determined that the user's selection is correct.

10. The interactive method for microscopic experiment teaching according to claim 9, characterized in that: If the first result is the same as the second result, determining that the user's selection is correct further includes: If the first result is different from the second result, and the maximum deviation value of the confidence between the optional results is less than a first threshold, outputting all the optional results to the user; If the first result is different from the second result, and the confidence deviation value between the optional results is greater than or equal to the first threshold, the optional result with the highest confidence is output to the user.

11. The interactive method for microscopic experiment teaching according to claim 9, characterized in that: If the first result is the same as the second result, determining that the user's selection is correct further includes: If the confidence level corresponding to the target result is lower than a second threshold, the optional result, the target result and the area to be identified are sent to the teacher terminal; Re-determine whether the user's selection is correct based on the feedback from the teacher.

12. The interactive method for microscopic experiment teaching according to claim 9, characterized in that: The step of determining the area to be identified on the target digital slice in response to the auxiliary identification start instruction further includes: Recording the number of times the auxiliary identification start instruction is triggered during the slice identification operation; The outputting the optional results to the user for selection by the user further includes: If the triggering number is greater than a third threshold, determining and displaying a corresponding first test question according to the slice identification operation; When the correct rate of the first test question is greater than a fourth threshold, outputting the optional result to the user for the user to select; Otherwise, the knowledge point analysis corresponding to the slice identification operation is output to the user.

13. The interactive method for microscopic experiment teaching according to claim 12, characterized in that: The step of outputting the knowledge point analysis corresponding to the slice identification operation to the user includes: Display the knowledge point analysis through an interactive pop-up window, and activate the test start control after a first set time; In response to an operation on the test start control, displaying a second test question corresponding to the knowledge point analysis to the user; If the user answers the second number of the second test questions correctly, close the interactive pop-up window; otherwise, return to the step of displaying the knowledge point analysis through the interactive pop-up window and activating the test start control after the first set time.

14. The interactive method for microscopic experiment teaching according to claim 9, characterized in that: Also includes: If the first result does not match the second result, and the difference between the confidence level corresponding to the first result and the second result is lower than a fifth threshold, sending the first result as the target category and the screenshot corresponding to the area to be identified as a reference image to the image generation model to obtain an easily confusing image; The image generation model is used to generate the easily confusing image whose recognition result is the target category according to the reference image; The first result and the easily-confusable image are taken as a group, and the second result and the screenshot corresponding to the to-be-identified area are taken as another group and displayed to the user together.

15. The interactive method for microscopic experiment teaching according to claim 14, characterized in that: Before presenting the first result and the easily-confusable image as a group and the second result and the screenshot corresponding to the to-be-identified area as another group to the user, the method further includes: Determine a marking distinguishing structure according to the first result, the second result and the first mapping relationship; The landmark distinguishing structure and the confusable image are input into a labeling model to label the landmark distinguishing structure in the confusable image.

16. A microscopic experiment teaching interactive device, characterized in that: Applied to the student end of the interactive teaching system, the microscopic experiment teaching interactive method includes: A digital slice acquisition module, used for acquiring target digital slices; A course interaction module, for synchronizing the user's target operation on the target digital slice to the teacher's end in response to a course start instruction; The homework module is used to respond to the homework start instruction to display the slice identification homework to the students. In the process of completing the slice identification homework, it responds to the auxiliary question and answer start instruction to call the target large model to provide answers to the questions. In response to the homework submission instruction, the slice identification homework is synchronized to the teacher's end.

17. A computer device, characterized in that: It includes one or more processors and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the microscopic experiment teaching interactive method as described in any one of claims 1 to 15 are executed.

18. A storage medium, characterized in that: The storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the microscopic experiment teaching interactive method as described in any one of claims 1 to 15.

Images