Endoscope scene simulation teaching assessment automatic scoring method and system based on artificial intelligence

Abstract

The invention relates to an automatic scoring method and system for endoscope scene simulation teaching assessment based on artificial intelligence, and belongs to the field of medical assessment. The method comprises the steps that an endoscope simulation scene is obtained, standard focus information in the endoscope simulation scene is obtained, and the standard focus information comprises the standard focus position, the standard focus size and the standard focus type; performing simulation teaching assessment in the endoscope simulation scene through the virtual reality equipment, wherein the simulation teaching assessment comprises endoscope holding posture assessment, endoscope operation assessment and focus judgment assessment; teaching assessment data are obtained, the teaching assessment data comprise lens-holding posture similarity, endoscope operation assessment data and focus judgment goodness of fit, and the endoscope operation assessment data comprise lens-holding force moderation, lens-holding stability and patient simulated pain degree; and calculating a teaching assessment score according to the teaching assessment data. According to the invention, comprehensive, comprehensive, rapid and automatic scoring of simulation endoscope examination is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical assessment, and in particular relates to an automatic scoring method and system for endoscopic scenario simulation teaching assessment based on artificial intelligence. Background Art

[0002] Endoscopic simulation teaching is a way to teach medical students or doctors endoscopy-related knowledge and skills with the help of a simulation system. Traditional endoscopy teaching directly practices on patients, which is risky. Simulation teaching allows students to repeatedly practice endoscopy operation skills in a safe environment, such as basic operations such as insertion, movement, and observation of the endoscope. For example, in urological endoscopy simulation teaching, students can practice the use of rigid cystoscopes, rigid ureteroscopes, etc., and become familiar with the anatomical structure and operation feel of the urinary system. Students can more intuitively understand the structure and principle of endoscopes, as well as theoretical knowledge such as indications and contraindications of endoscopic examinations through the simulation system. For example, in the digestive endoscopy simulation teaching system, students can better understand the application principles of endoscopy in the diagnosis and treatment of digestive tract diseases and other related knowledge. Before actually operating the endoscope in clinical practice, students practice a lot through the simulation system to reduce possible mistakes when operating on real patients, thereby ensuring patient safety and improving medical quality.

[0003] At present, simulation teaching assessment has been included in the assessment system of many hospitals, but its scoring still mostly adopts manual scoring method, which is inefficient and relatively subjective. It also requires more human resources, and the definition of assessment content is relatively simple. Summary of the invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides an automatic scoring method and system for endoscopic scenario simulation teaching assessment based on artificial intelligence.

[0005] The purpose of the present invention can be achieved through the following technical solutions:

[0006] An automatic scoring method for endoscopy scenario simulation teaching assessment based on artificial intelligence, the implementation of the automatic scoring method for endoscopy scenario simulation teaching assessment includes the following steps:

[0007] Acquire an endoscopic simulation scene, wherein the endoscopic simulation scene is a simulated scene inside a diseased organ containing a lesion, and acquire standard lesion information in the endoscopic simulation scene, wherein the standard lesion information includes a standard lesion position, a standard lesion size, and a standard lesion type;

[0008] The examinee uses virtual reality equipment to conduct a simulated teaching assessment in the endoscopy simulation scenario, and the simulated teaching assessment includes an endoscope holding posture assessment, an endoscope operation assessment, and a lesion judgment assessment;

[0009] Acquiring teaching assessment data, wherein the teaching assessment data includes similarity of endoscope holding posture, endoscope operation assessment data, and consistency of lesion judgment, wherein the endoscope operation assessment data includes moderate endoscope holding force, endoscope holding stability, and patient simulated pain degree;

[0010] A teaching assessment score is calculated based on the teaching assessment data.

[0011] Preferably, the obtaining of teaching assessment data includes:

[0012] Acquire the similarity of the mirror holding posture by constructing a three-dimensional hand skeleton model;

[0013] Acquiring the endoscopic operation assessment data according to the sensor;

[0014] The degree of consistency of the lesion is determined based on the standard lesion information.

[0015] Preferably, the constructing of a three-dimensional hand skeleton model comprises:

[0016] The hand gesture video is captured by a camera and processed in frames. The camera follows the hand movement of the examinee to ensure that the examinee's hand is always within the camera capture screen. Hand feature points are randomly selected to obtain the camera coordinates of the hand feature points. The hand feature point camera coordinates include the first hand feature point camera coordinates, the second hand feature point camera coordinates, ..., the nth hand feature point camera coordinates, where n is the total number of frames of the hand gesture video. The first hand feature point camera coordinates are the imaging plane coordinates of the hand feature point on the camera in the first frame, the second hand feature point camera coordinates are the imaging plane coordinates of the hand feature point on the camera in the second frame, and the nth hand feature point camera coordinates are the imaging plane coordinates of the hand feature point on the camera in the nth frame.

[0017] Acquire a gesture capture position, wherein the gesture capture position includes a first gesture capture position, a second gesture capture position, ..., and an nth gesture capture position, wherein the first gesture capture position is the position of the camera in the first frame, the second gesture capture position is the position of the camera in the second frame, and the nth gesture capture position is the position of the camera in the nth frame;

[0018] Get the posture transformation formula;

[0019] Acquire the posture conversion parameters according to the posture conversion formula, and acquire the posture conversion matrix according to the posture conversion parameters;

[0020] Taking the origin of the camera of the first frame as the origin of the world coordinate system, calculating the world coordinates of the hand feature points in the world coordinate system according to the posture transformation matrix;

[0021] The world coordinates of the hand feature points of all hand feature points are obtained by analogy and the three-dimensional hand skeleton model is constructed, wherein the three-dimensional hand skeleton model includes a first-frame three-dimensional hand skeleton model, a second-frame three-dimensional hand skeleton model, ..., and an n-th-frame three-dimensional hand skeleton model.

[0022] Preferably, the obtaining of the similarity of the mirror holding posture comprises:

[0023] Acquire a standard posture 3D hand skeleton model, wherein the standard posture 3D hand skeleton model includes a first frame standard posture 3D hand skeleton model, a second frame standard posture 3D hand skeleton model, ..., an nth frame standard posture 3D hand skeleton model; acquire hand bone connections, wherein the hand bone connections include a first hand bone connection, a second hand bone connection, ..., an Mth hand bone connection;

[0024] Acquire a standard posture vector based on the standard posture three-dimensional hand skeleton model, wherein the standard posture vector includes a first standard posture vector, a second standard posture vector, ... an M-1th standard posture vector, and the standard posture vector is a vector from the mth hand bone connection to the m+1th hand bone connection in the standard posture three-dimensional hand skeleton model, where m∈M-1;

[0025] Acquire a posture vector based on the three-dimensional hand skeleton model, the posture vector comprising a first posture vector, a second posture vector, ... an M-1th posture vector, the posture vector being a vector from the mth hand bone connection to the m+1th hand bone connection in the three-dimensional hand skeleton model, m∈M-1;

[0026] The mirror holding posture similarity is calculated based on the standard posture vector and the posture vector.

[0027] Preferably, the step of obtaining the endoscopic operation assessment data according to the sensor includes:

[0028] Preset a threshold range of holding force for the mirror, obtain the holding force of the examinee through a force sensor, count the number of improper holding force times of the examinee exceeding the threshold range of holding force, and calculate the moderate holding force according to the number of improper holding force times;

[0029] Preset a threshold range of the stability of holding the mirror, obtain the stability of the examinee holding the mirror through a stability sensor, count the number of improper stability times when the examinee holding the mirror exceeds the threshold range of the stability of holding the mirror, and calculate the stability of holding the mirror according to the number of improper stability times;

[0030] The patient's simulated pain level is obtained by constructing an improved bounding box, wherein the improved bounding box is a rectangular parallelepiped with the smallest volume that can enclose the current endoscopic detection site.

[0031] Preferably, constructing an improved bounding box to obtain the patient's simulated pain level includes:

[0032] Acquire a set of triangular faces constituting the current endoscope detection part and obtain the coordinates of the triangular face vertices, and calculate the improved bounding box mean according to the coordinates of the triangular face vertices;

[0033] Calculating an improved bounding box covariance according to the improved bounding box mean;

[0034] Obtaining a covariance matrix according to the improved bounding box covariance;

[0035] Each row of the covariance matrix is ​​the first eigenvector, the second eigenvector, and the third eigenvector, respectively. The unit length directions of the first eigenvector, the second eigenvector, and the third eigenvector are taken as the basis coordinate axes of the improved bounding box. The basis coordinate axes include the first basis coordinate axis, the second basis coordinate axis, and the third basis coordinate axis. The projections of the vertices of the current endoscope detection part on the basis coordinate axes are taken as the side lengths of the improved bounding box to construct the improved bounding box.

[0036] Obtaining a mirror box distance, where the mirror box distance is the distance from the head of the simulated endoscope to the center point of the improved bounding box. When the mirror box distance is greater than or equal to the half short side length of the improved bounding box, outputting information that the endoscope touches the entity. When the mirror box distance is less than the half short side length of the improved bounding box, outputting information that the endoscope does not touch the entity. The half short side length of the improved bounding box is half the length of the shortest side of the improved bounding box.

[0037] When outputting the information of the endoscope touching the entity, acquiring the time of touching the entity and the number of times of touching the entity;

[0038] The patient's simulated pain degree is calculated according to the number of times the entity is touched and the time the entity is touched.

[0039] Preferably, obtaining the lesion judgment consistency according to the standard lesion information includes:

[0040] The lesion information of the candidate's answer is obtained, and the lesion information of the candidate's answer includes the lesion location of the candidate's answer, the lesion size of the candidate's answer, and the lesion type of the candidate's answer. The degree of consistency of the lesion judgment is obtained by comparing the lesion information of the candidate's answer with the standard lesion information.

[0041] An artificial intelligence-based automatic scoring system for endoscopy scenario simulation teaching assessment, used to execute the above-mentioned endoscopy simulation teaching assessment scoring method, including a teaching assessment module, an assessment data acquisition module, and an assessment scoring module;

[0042] The teaching assessment module is used to obtain an endoscopy simulation scenario, which is a simulated scenario of the inside of a diseased organ containing a lesion, and the inside of the diseased organ includes but is not limited to the stomach, the inside of the small intestine, and the inside of the colon, and obtain standard lesion information in the endoscopy simulation scenario, and the standard lesion information includes a standard lesion position, a standard lesion size, and a standard lesion type; the examinee conducts a simulated teaching assessment in the endoscopy simulation scenario through a virtual reality device, and the simulated teaching assessment includes an endoscope holding posture assessment, an endoscope operation assessment, and a lesion judgment assessment;

[0043] The assessment data acquisition module is used to acquire teaching assessment data, wherein the teaching assessment data includes the similarity of endoscope holding posture, endoscope operation assessment data, and lesion judgment consistency, and the endoscope operation assessment data includes the moderate force of holding the endoscope, the stability of holding the endoscope, and the patient's simulated pain degree;

[0044] The assessment and scoring module is used to calculate the teaching assessment score based on the teaching assessment data.

[0045] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the above-mentioned automatic scoring method for endoscopic scenario simulation teaching assessment is implemented.

[0046] A storage medium containing computer executable instructions, which, when executed by a computer processor, are used to execute the above-mentioned automatic scoring method for endoscopic scenario simulation teaching assessment.

[0047] The beneficial effects of the present invention are:

[0048] (1) Conduct endoscopy simulation teaching assessment through endoscopy simulation scenarios, comprehensively examine the candidates' mastery of endoscope holding, endoscopy operation, and lesion judgment, and give corresponding scores, so as to achieve fully automated, scientific, and efficient simulation teaching assessment;

[0049] (2) The camera is used to capture the hand images of the examinee during the assessment process, and the coordinates of each feature point of the hand and its position change during the movement are obtained by posture transformation. The normalized three-dimensional hand skeleton model can be constructed by integrating the coordinates and position changes of all feature points. The spatial relationship between the human body feature points and movements is fully utilized, and the skeleton is obtained by time-space superposition to eliminate individual differences. By comparing with the standard posture three-dimensional hand skeleton model, the similarity of the endoscope holding posture can be obtained, and scientific judgment can be made on whether the examinee's holding technique of the endoscope is standard;

[0050] (3) An improved bounding box is constructed by obtaining the set of triangular faces that constitute the current endoscopic detection site, simulating the collision surface of the endoscopic detection site, and judging whether the endoscope contacts the inner wall of the detection site, thereby simulating whether the patient will feel pain due to the collision. The improved bounding box is a rectangular block with the smallest volume that can wrap the current endoscopic detection site, and has good tightness. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0052] Figure 1 The present invention is a flowchart of the steps of an automatic scoring method for endoscopic scenario simulation teaching assessment based on artificial intelligence. DETAILED DESCRIPTION

[0053] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0054] The working principle and use process of the present invention:

[0055] See also Figure 1 , an automatic scoring method for endoscopy scenario simulation teaching assessment based on artificial intelligence, comprising:

[0056] S1: Acquire an endoscopy simulation scene, wherein the endoscopy simulation scene is a simulation scene of the interior of a diseased organ containing a lesion, wherein the interior of the diseased organ includes but is not limited to the stomach, the interior of the small intestine, and the interior of the colon, and acquire standard lesion information in the endoscopy simulation scene, wherein the standard lesion information includes a standard lesion position, a standard lesion size, and a standard lesion type;

[0057] S2: The examinee uses virtual reality equipment to conduct a simulated teaching assessment in the endoscopy simulation scenario, and the simulated teaching assessment includes an endoscope holding posture assessment, an endoscope operation assessment, and a lesion judgment assessment;

[0058] The endoscope holding posture assessment is an assessment of the examinee's standard posture for holding an endoscope; the endoscope operation assessment is an assessment of the examinee's endoscope holding strength, endoscope holding stability, and patient perception in the process of searching for lesions using a simulated endoscope in the endoscope simulation scenario; and the lesion judgment assessment is an assessment of the examinee's accuracy in judging the lesion location, lesion size, and lesion type;

[0059] S3: Acquiring teaching assessment data, wherein the teaching assessment data includes similarity of endoscope holding posture, endoscope operation assessment data, and consistency of lesion judgment, wherein the endoscope operation assessment data includes moderate endoscope holding force, endoscope holding stability, and patient simulated pain degree;

[0060] S4: Calculate the teaching assessment score according to the teaching assessment data, and the calculation formula is J=C+F+U+AW, wherein J is the teaching assessment score, C is the similarity of the mirror holding posture, F is the moderate strength of holding the mirror, U is the stability of holding the mirror, A is the consistency of lesion judgment, and W is the patient's simulated pain.

[0061] In this embodiment, the teaching assessment data is obtained, and the teaching assessment data includes the similarity of endoscope holding posture, endoscope operation assessment data, and lesion judgment consistency, which can be specifically implemented by the following steps:

[0062] S301: Acquire the similarity of the mirror holding posture by constructing a three-dimensional hand skeleton model;

[0063] S302: Acquiring the endoscope operation assessment data according to the sensor;

[0064] S303: Obtaining the lesion judgment consistency according to the standard lesion information;

[0065] The lesion information of the candidate's answer is obtained, and the lesion information of the candidate's answer includes the lesion location of the candidate's answer, the lesion size of the candidate's answer, and the lesion type of the candidate's answer. The lesion judgment consistency is obtained by comparing the lesion information of the candidate's answer with the standard lesion information. The lesion judgment consistency is the similarity between the lesion information of the candidate's answer and the standard lesion information, and the unit is %.

[0066] In this embodiment, the similarity of the mirror holding posture is obtained by constructing a three-dimensional hand skeleton model, which can be specifically implemented by the following steps:

[0067] S301-1: constructing the three-dimensional hand skeleton model;

[0068] S301-2: Obtaining the similarity of the mirror holding posture according to the three-dimensional hand skeleton model.

[0069] In this embodiment, constructing the three-dimensional hand skeleton model can be implemented by the following steps:

[0070] The hand gesture video is captured by a camera and processed in frames. The camera follows the hand movement of the examinee to ensure that the examinee's hand is always within the camera capture screen. Hand feature points are randomly selected to obtain the camera coordinates of the hand feature points. The hand feature point camera coordinates include the first hand feature point camera coordinates, the second hand feature point camera coordinates, ..., the nth hand feature point camera coordinates, where n is the total number of frames of the hand gesture video. The first hand feature point camera coordinates are the imaging plane coordinates of the hand feature point on the camera in the first frame, the second hand feature point camera coordinates are the imaging plane coordinates of the hand feature point on the camera in the second frame, and the nth hand feature point camera coordinates are the imaging plane coordinates of the hand feature point on the camera in the nth frame.

[0071] Acquire a gesture capture position, wherein the gesture capture position includes a first gesture capture position, a second gesture capture position, ..., and an nth gesture capture position, wherein the first gesture capture position is the position of the camera in the first frame, the second gesture capture position is the position of the camera in the second frame, and the nth gesture capture position is the position of the camera in the nth frame;

[0072] Get the posture transformation formula, the expression of the posture transformation formula is in, is the vector from the i-th posture capture position to the j-th posture capture position, R ij is the attitude transformation parameter, P i is the camera coordinate of the i-th hand feature point, P j is the camera coordinate of the jth hand feature point, and n is the total number of frames of the hand gesture video;

[0073] The posture conversion parameters are obtained according to the posture conversion formula, and the posture conversion matrix is ​​obtained according to the posture conversion parameters. The expression of the posture conversion matrix is: Among them, T ij is the pose transformation matrix of the hand feature point from the i-th pose capture position to the j-th pose capture position;

[0074] Taking the origin of the first frame camera as the origin of the world coordinate system, the world coordinates of the hand feature points in the world coordinate system are calculated according to the posture transformation matrix. The calculation formula is: in, is the world coordinates of the hand feature points in the i-th frame, T 12 is the pose transformation matrix of the hand feature points from the first pose capture position to the second pose capture position, T 23 is the pose transformation matrix of the hand feature points from the second pose capture position to the third pose capture position, T i(i-1) is the pose transformation matrix of the hand feature point from the i-th pose capture position to the i-1-th pose capture position, P i is the camera coordinate of the i-th hand feature point;

[0075] The world coordinates of the hand feature points of all hand feature points are obtained by analogy and the three-dimensional hand skeleton model is constructed, which makes full use of the spatial relationship between human body feature points and actions, obtains the skeleton through time-space superposition, and eliminates individual differences. The three-dimensional hand skeleton model includes a first frame three-dimensional hand skeleton model, a second frame three-dimensional hand skeleton model, ..., and an n-th frame three-dimensional hand skeleton model.

[0076] In this embodiment, obtaining the similarity of the mirror holding posture according to the three-dimensional hand skeleton model can be implemented by the following steps:

[0077] Acquire a standard posture 3D hand skeleton model, wherein the standard posture 3D hand skeleton model includes a first frame standard posture 3D hand skeleton model, a second frame standard posture 3D hand skeleton model, ..., an nth frame standard posture 3D hand skeleton model; acquire hand bone connections, wherein the hand bone connections include a first hand bone connection, a second hand bone connection, ..., an Mth hand bone connection;

[0078] Acquire a standard posture vector based on the standard posture three-dimensional hand skeleton model, wherein the standard posture vector includes a first standard posture vector, a second standard posture vector, ... an M-1th standard posture vector, and the standard posture vector is a vector from the mth hand bone connection to the m+1th hand bone connection in the standard posture three-dimensional hand skeleton model, where m∈M-1;

[0079] Acquire a posture vector based on the three-dimensional hand skeleton model, the posture vector comprising a first posture vector, a second posture vector, ... an M-1th posture vector, the posture vector being a vector from the mth hand bone connection to the m+1th hand bone connection in the three-dimensional hand skeleton model, m∈M-1;

[0080] The similarity of the mirror holding posture is calculated based on the standard posture vector and the posture vector, and the calculation formula is: Among them, C is the similarity of the camera holding posture, n is the total number of frames of the hand posture video, M is the total number of hand bone connections, (x αm ,y αm ,z αm ) is the coordinate of the mth standard posture vector of the i-th frame, (x βm ,y βm ,z βm ) is the coordinate of the mth posture vector of the ith frame.

[0081] In this embodiment, the endoscopic operation assessment data is obtained according to the sensor, which can be implemented by the following steps:

[0082] S302-1: Preset a threshold range of holding force, obtain the examinee's holding force through a force sensor, and count the number of improper holding force times when the examinee's holding force exceeds the threshold range of holding force, and calculate the moderate holding force according to the improper holding force times. The calculation formula is: Among them, F is the moderate force of holding the mirror, and D is the number of improper force;

[0083] S302-2: Preset a threshold range for the stability of holding the glasses, obtain the stability of the examinee holding the glasses through a stability sensor, and count the number of improper stability times when the examinee's stability of holding the glasses exceeds the threshold range for the stability of holding the glasses, and calculate the stability of holding the glasses according to the improper stability times. The calculation formula is: Among them, U is the stability of holding the mirror, and T is the number of improper stability;

[0084] S302-3: Acquire the patient's simulated pain level by constructing an improved bounding box, wherein the improved bounding box is a rectangular parallelepiped with the smallest volume that can enclose the current endoscopic detection part and has good tightness;

[0085] Get the triangular face set that constitutes the current endoscope detection part and obtain the coordinates of the triangular face vertices. r (x 1r ,y 1r ,z 1r ), P r (x 2r ,y 2r ,z 2r ), Q r (x 3r ,y 3r ,z 3r ), r = 1, 2, ..., R, R is the number of triangles in the triangle set, and the improved bounding box mean is calculated according to the vertex coordinates of the triangles. The calculation formula is: Among them, E is the improved bounding box mean, E x Improve the bounding box mean in the X-axis direction, E y Improve the bounding box mean in the Y-axis direction, E z is the mean of the improved bounding box in the Z-axis direction, R is the number of triangles, (x 1r ,y 1r ,z 1r )、(x 2r ,y 2r ,z 2r )、(x 3r ,y 3r ,z 3r ) are the coordinates of the triangle vertices of the rth triangle;

[0086] The improved bounding box covariance is calculated based on the improved bounding box mean, and the calculation formula is: Where, κ is the improved bounding box covariance, κ(x, y) is the improved bounding box covariance of the XY axis, κ(x, x) is the improved bounding box covariance of the X axis, κ(x, z) is the improved bounding box covariance of the XZ axis, κ(y, y) is the improved bounding box covariance of the Y axis, κ(y, z) is the improved bounding box covariance of the YZ axis, κ(z, z) is the improved bounding box covariance of the Z axis, and (x, y, z) is any point in the improved bounding box;

[0087] The covariance matrix is ​​obtained according to the improved bounding box covariance. The mathematical description of the covariance matrix is: Where L is the covariance matrix;

[0088] Each row of the covariance matrix is ​​the first eigenvector, the second eigenvector, and the third eigenvector, respectively. The unit length directions of the first eigenvector, the second eigenvector, and the third eigenvector are taken as the basis coordinate axes of the improved bounding box. The basis coordinate axes include the first basis coordinate axis, the second basis coordinate axis, and the third basis coordinate axis. The projections of the vertices of the current endoscope detection part on the basis coordinate axes are taken as the side lengths of the improved bounding box to construct the improved bounding box.

[0089] Obtaining a mirror box distance, where the mirror box distance is the distance from the head of the simulated endoscope to the center point of the improved bounding box. When the mirror box distance is greater than or equal to the half short side length of the improved bounding box, outputting information that the endoscope touches the entity. When the mirror box distance is less than the half short side length of the improved bounding box, outputting information that the endoscope does not touch the entity. The half short side length of the improved bounding box is half the length of the shortest side of the improved bounding box.

[0090] When outputting the endoscope touching entity information, acquiring the touching entity time and the touching entity number, wherein the touching entity time is the total time length for outputting the endoscope touching entity information, and the touching entity number is the total number of times the endoscope touching entity information is output;

[0091] The patient's simulated pain degree is calculated according to the number of times the entity is touched and the duration of the entity being touched. The calculation formula is: Where W is the patient's simulated pain level, G is the number of times the entity is touched, and t is the time of touching the entity in minutes.

[0092] An artificial intelligence-based automatic scoring system for endoscopy scenario simulation teaching and assessment, including a teaching assessment module, an assessment data acquisition module, and an assessment scoring module.

[0093] The teaching assessment module is used to obtain an endoscopy simulation scenario, which is a simulated scenario of the inside of a diseased organ containing a lesion, and the inside of the diseased organ includes but is not limited to the stomach, the inside of the small intestine, and the inside of the colon, and obtain standard lesion information in the endoscopy simulation scenario, and the standard lesion information includes a standard lesion position, a standard lesion size, and a standard lesion type; the examinee conducts a simulated teaching assessment in the endoscopy simulation scenario through a virtual reality device, and the simulated teaching assessment includes an endoscope holding posture assessment, an endoscope operation assessment, and a lesion judgment assessment;

[0094] The assessment data acquisition module is used to acquire teaching assessment data, wherein the teaching assessment data includes the similarity of endoscope holding posture, endoscope operation assessment data, and lesion judgment consistency, and the endoscope operation assessment data includes the moderate force of holding the endoscope, the stability of holding the endoscope, and the patient's simulated pain degree;

[0095] The assessment and scoring module is used to calculate the teaching assessment score based on the teaching assessment data.

[0096] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device.

[0097] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0098] The program code included on the computer readable medium can be transmitted with any appropriate medium, including but not limited to wireless, electric wire, optical cable, RF, etc., or any suitable combination of the above. The computer program code for performing the operation of the present invention can be written in one or more programming languages ​​or their combinations, and the programming language includes object-oriented programming languages-such as Java, Smalltalk, C++, and also includes conventional procedural programming languages-such as "C" language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0099] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. An automatic scoring method for endoscopy scenario simulation teaching assessment based on artificial intelligence, characterized in that: The implementation of the automatic scoring method for endoscopy scenario simulation teaching assessment includes the following steps: Acquire an endoscopic simulation scene, wherein the endoscopic simulation scene is a simulated scene inside a diseased organ containing a lesion, and acquire standard lesion information in the endoscopic simulation scene, wherein the standard lesion information includes a standard lesion position, a standard lesion size, and a standard lesion type; The examinee uses virtual reality equipment to conduct a simulated teaching assessment in the endoscopy simulation scenario, and the simulated teaching assessment includes an endoscope holding posture assessment, an endoscope operation assessment, and a lesion judgment assessment; Acquiring teaching assessment data, wherein the teaching assessment data includes similarity of endoscope holding posture, endoscope operation assessment data, and consistency of lesion judgment, wherein the endoscope operation assessment data includes moderate endoscope holding force, endoscope holding stability, and patient simulated pain degree; A teaching assessment score is calculated based on the teaching assessment data.

2. The automatic scoring method for endoscopy scenario simulation teaching assessment according to claim 1 is characterized in that: The obtaining of teaching assessment data includes: Acquire the similarity of the mirror holding posture by constructing a three-dimensional hand skeleton model; Acquiring the endoscopic operation assessment data according to the sensor; The degree of consistency of the lesion is determined based on the standard lesion information.

3. The automatic scoring method for endoscopy scenario simulation teaching assessment according to claim 2 is characterized in that: The constructing of the three-dimensional hand skeleton model comprises: The hand gesture video is captured by a camera and processed in frames. The camera follows the hand movement of the examinee to ensure that the examinee's hand is always within the camera capture screen. Hand feature points are randomly selected to obtain the camera coordinates of the hand feature points. The hand feature point camera coordinates include the first hand feature point camera coordinates, the second hand feature point camera coordinates, ..., the nth hand feature point camera coordinates, where n is the total number of frames of the hand gesture video. The first hand feature point camera coordinates are the imaging plane coordinates of the hand feature point on the camera in the first frame, the second hand feature point camera coordinates are the imaging plane coordinates of the hand feature point on the camera in the second frame, and the nth hand feature point camera coordinates are the imaging plane coordinates of the hand feature point on the camera in the nth frame; Acquire a gesture capture position, wherein the gesture capture position includes a first gesture capture position, a second gesture capture position, ..., and an nth gesture capture position, wherein the first gesture capture position is the position of the camera in the first frame, the second gesture capture position is the position of the camera in the second frame, and the nth gesture capture position is the position of the camera in the nth frame; Get the posture transformation formula; Acquire the posture conversion parameters according to the posture conversion formula, and acquire the posture conversion matrix according to the posture conversion parameters; Taking the origin of the camera of the first frame as the origin of the world coordinate system, calculating the world coordinates of the hand feature points in the world coordinate system according to the posture transformation matrix; The world coordinates of the hand feature points of all hand feature points are obtained by analogy and the three-dimensional hand skeleton model is constructed, wherein the three-dimensional hand skeleton model includes a first-frame three-dimensional hand skeleton model, a second-frame three-dimensional hand skeleton model, ..., and an n-th-frame three-dimensional hand skeleton model.

4. The automatic scoring method for endoscopy scenario simulation teaching assessment according to claim 2 is characterized in that: The obtaining of the similarity of the mirror holding posture comprises: Acquire a standard posture 3D hand skeleton model, wherein the standard posture 3D hand skeleton model includes a first frame standard posture 3D hand skeleton model, a second frame standard posture 3D hand skeleton model, ..., an nth frame standard posture 3D hand skeleton model; acquire hand bone connections, wherein the hand bone connections include a first hand bone connection, a second hand bone connection, ..., an Mth hand bone connection; Acquire a standard posture vector based on the standard posture three-dimensional hand skeleton model, wherein the standard posture vector includes a first standard posture vector, a second standard posture vector, ... an M-1th standard posture vector, and the standard posture vector is a vector from the mth hand bone connection to the m+1th hand bone connection in the standard posture three-dimensional hand skeleton model, where m∈M-1; Acquire a posture vector based on the three-dimensional hand skeleton model, the posture vector comprising a first posture vector, a second posture vector, ... an M-1th posture vector, the posture vector being a vector from the mth hand bone connection to the m+1th hand bone connection in the three-dimensional hand skeleton model, m∈M-1; The mirror holding posture similarity is calculated based on the standard posture vector and the posture vector.

5. The automatic scoring method for endoscopy scenario simulation teaching assessment according to claim 2 is characterized in that: The obtaining of the endoscopic operation assessment data according to the sensor comprises: Preset a threshold range of holding force for the mirror, obtain the holding force of the examinee through a force sensor, count the number of improper holding force times of the examinee exceeding the threshold range of holding force, and calculate the moderate holding force according to the number of improper holding force times; Preset a threshold range of the stability of holding the mirror, obtain the stability of the examinee holding the mirror through a stability sensor, count the number of improper stability times when the examinee holding the mirror exceeds the threshold range of the stability of holding the mirror, and calculate the stability of holding the mirror according to the number of improper stability times; The patient's simulated pain level is obtained by constructing an improved bounding box, where the improved bounding box is a rectangular block with the smallest volume that can enclose the current endoscopic detection site.

6. The automatic scoring method for endoscopy scenario simulation teaching assessment according to claim 5 is characterized in that: The constructing of an improved bounding box to obtain the patient's simulated pain level comprises: Acquire a set of triangular faces constituting the current endoscope detection part and obtain the coordinates of the triangular face vertices, and calculate the improved bounding box mean according to the coordinates of the triangular face vertices; Calculating an improved bounding box covariance according to the improved bounding box mean; Obtaining a covariance matrix according to the improved bounding box covariance; Each row of the covariance matrix is ​​the first eigenvector, the second eigenvector, and the third eigenvector, respectively. The unit length directions of the first eigenvector, the second eigenvector, and the third eigenvector are taken as the basis coordinate axes of the improved bounding box. The basis coordinate axes include the first basis coordinate axis, the second basis coordinate axis, and the third basis coordinate axis. The projections of the vertices of the current endoscope detection part on the basis coordinate axes are taken as the side lengths of the improved bounding box to construct the improved bounding box. Obtaining a mirror box distance, where the mirror box distance is the distance from the head of the simulated endoscope to the center point of the improved bounding box. When the mirror box distance is greater than or equal to the half short side length of the improved bounding box, outputting information that the endoscope touches the entity. When the mirror box distance is less than the half short side length of the improved bounding box, outputting information that the endoscope does not touch the entity. The half short side length of the improved bounding box is half the length of the shortest side of the improved bounding box. When outputting the endoscope touching entity information, acquiring the touching entity time and the touching entity times; The patient's simulated pain degree is calculated according to the number of times the entity is touched and the duration of the touching.

7. The automatic scoring method for endoscopy scenario simulation teaching assessment according to claim 2 is characterized in that: The obtaining of the lesion judgment consistency according to the standard lesion information comprises: The lesion information of the candidate's answer is obtained, and the lesion information of the candidate's answer includes the lesion location of the candidate's answer, the lesion size of the candidate's answer, and the lesion type of the candidate's answer. The degree of consistency of the lesion judgment is obtained by comparing the lesion information of the candidate's answer with the standard lesion information.

8. An automatic scoring system for endoscopy scenario simulation teaching assessment based on artificial intelligence, characterized in that: The system is applied to the automatic scoring method for endoscopy scenario simulation teaching assessment as described in any one of claims 1 to 7, including a teaching assessment module, an assessment data acquisition module, and an assessment scoring module; The teaching assessment module is used to obtain an endoscopy simulation scenario, which is a simulated scenario of the inside of a diseased organ containing a lesion, and the inside of the diseased organ includes but is not limited to the stomach, the inside of the small intestine, and the inside of the colon, and obtain standard lesion information in the endoscopy simulation scenario, and the standard lesion information includes a standard lesion position, a standard lesion size, and a standard lesion type; the examinee conducts a simulated teaching assessment in the endoscopy simulation scenario through a virtual reality device, and the simulated teaching assessment includes an endoscope holding posture assessment, an endoscope operation assessment, and a lesion judgment assessment; The assessment data acquisition module is used to acquire teaching assessment data, wherein the teaching assessment data includes the similarity of endoscope holding posture, endoscope operation assessment data, and lesion judgment consistency, and the endoscope operation assessment data includes the moderate force of holding the endoscope, the stability of holding the endoscope, and the patient's simulated pain degree; The assessment and scoring module is used to calculate the teaching assessment score based on the teaching assessment data.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the automatic scoring method for endoscopic scenario simulation teaching assessment as described in any one of claims 1-7 is implemented.

10. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions, when executed by a computer processor, are used to execute the automatic scoring method for endoscopic scenario simulation teaching assessment as described in any one of claims 1-7.

Images