Operation training method and system of endoscope simulation teaching aid

Abstract

The invention relates to an operation training method and system of an endoscope simulation teaching aid, the endoscope simulation teaching aid comprises a cavity with a variable structure, the cavity is used for simulating an organ, and the inner wall of the cavity is covered with a flexible display screen; the method comprises the following steps: training a selected target organ according to an endoscope, dynamically adjusting the physical structure of a cavity, and controlling a flexible display screen to load a preset texture map corresponding to the target organ for displaying, so that the cavity simulates the anatomic form of the target organ; determining a calibration object corresponding to the target organ, and generating a motion path of the calibration object; in response to the training starting signal, the flexible display screen is controlled to display the calibration object moving along the movement path, so that a trainee operates the endoscope to perform simulation training, and the problem that the operation ability of the trainee is difficult to effectively improve due to the fact that a real anatomical scene cannot be dynamically simulated in training is solved; a real anatomical scene is dynamically simulated in training, so that the operation ability of training personnel is effectively improved.

Description

Technical Field

[0001] This application relates to the field of medical teaching simulation technology, and in particular to methods and systems for operating endoscopic simulation teaching aids. Background Technology

[0002] Endoscopic techniques are a core tool in modern minimally invasive surgery, and the skills required for their operation highly depend on the surgeon's hand-eye coordination, spatial awareness, and ability to handle complex anatomical and pathological conditions. Currently, endoscopic operation training mainly relies on fixed anatomical models, resulting in a single and rigid training scenario that fails to dynamically simulate real anatomical scenarios during training, thus hindering the effective improvement of trainees' operational skills.

[0003] There is currently no effective solution to the problem that related technologies cannot dynamically simulate real anatomical scenarios during training, thus making it difficult to effectively improve the operational skills of trainees. Summary of the Invention

[0004] This embodiment provides a method and system for training the operation of an endoscope simulation teaching aid, in order to solve the problem in related technologies that it is impossible to dynamically simulate real anatomical scenarios during training, thus making it difficult to effectively improve the operational ability of trainees.

[0005] In a first aspect, this embodiment provides a method for training the operation of an endoscope simulation teaching aid, the endoscope simulation teaching aid including a cavity with a variable structure; the cavity is used to simulate an organ, and the inner wall of the cavity is covered with a flexible display screen; the method includes:

[0006] Based on the target organ selected during endoscopic training, the physical structure of the cavity is dynamically adjusted, and the flexible display screen is controlled to load and display the preset texture map corresponding to the target organ, so that the cavity simulates the anatomical shape of the target organ.

[0007] Identify the calibration object corresponding to the target organ and generate the motion path of the calibration object;

[0008] In response to a training start signal, the flexible display screen is controlled to display the calibration object moving along the motion path, so that the trainee can operate the endoscope for simulated training.

[0009] In some embodiments, the preset texture map is generated by UV unwrapping based on the three-dimensional model of the target organ;

[0010] The control of the flexible display screen to load and display the preset texture map corresponding to the target organ includes:

[0011] The bending curvature of the flexible display screen is detected in real time, and the surface parameter equation of the flexible display screen is updated based on the bending curvature.

[0012] The updated surface coordinates defined by the surface parametric equations are mapped to the UV plane to obtain the corresponding UV mapping equations.

[0013] Based on the UV mapping equation, the flexible display screen is controlled to load and display the preset texture map corresponding to the target organ.

[0014] In some embodiments, generating the motion path of the calibration object includes:

[0015] Obtain the selected training difficulty level for endoscopic training; the training difficulty level is divided based on the degree of physiological motion simulation required for endoscopic training.

[0016] Generate the motion path of the calibrator that matches the training difficulty level.

[0017] In some embodiments, the method further includes:

[0018] During the movement of the calibration object along the motion path, a corresponding noise image is generated based on preset interference factors in the endoscopy operation;

[0019] The noisy image is superimposed onto the display screen of the flexible display.

[0020] In some embodiments, the method further includes:

[0021] Acquire endoscopic images of the calibration object collected by the trainee during operation;

[0022] The endoscopic images are evaluated based on preset operation evaluation dimensions to obtain the corresponding endoscopic operation score;

[0023] The operational evaluation dimension includes at least the marker positioning accuracy; the marker positioning accuracy is used to evaluate the matching degree between the actual imaging position of the marker in the endoscopic image and the preset positioning reference.

[0024] In some embodiments, the method further includes:

[0025] During the movement of the calibration object along the motion path, the calibration object displayed on the flexible display screen is controlled to undergo a preset shape change at a preset time, so that the trainee can operate the endoscope to collect the shape of the marker.

[0026] In some embodiments, the method further includes:

[0027] Acquire endoscopic images of the calibration object collected by the trainee during operation.

[0028] The endoscopic images are evaluated based on preset operation evaluation dimensions to obtain the corresponding endoscopic operation score; the operation evaluation dimensions include marker positioning accuracy, keyframe response speed and image clarity.

[0029] The marker positioning accuracy is used to evaluate the matching degree between the actual imaging position of the marker in the endoscopic image and the preset positioning reference; the keyframe response speed is used to evaluate the operation time delay from the appearance of the preset morphological marker to the triggering of endoscopic image acquisition; and the image clarity is used to evaluate the clarity of the marker imaging.

[0030] In some embodiments, the endoscopic images are multiple;

[0031] The evaluation of the endoscopic images based on a preset operation evaluation dimension to obtain the corresponding endoscopic operation score includes:

[0032] Based on the preset operation evaluation dimensions, each of the endoscopic images collected by the trainee is evaluated to obtain a score corresponding to each endoscopic image;

[0033] The average score corresponding to each of the endoscopic images is calculated to obtain the endoscopic operation score.

[0034] In some embodiments, the method further includes:

[0035] When the trainee acquires the endoscopic image of the calibration object, corresponding operation correction information is generated based on the relative relationship between the actual imaging position of the calibration object in the endoscopic image and the preset positioning reference.

[0036] The operation correction information is converted into a corresponding operation guidance identifier; the operation guidance identifier is a combination of one or more of graphics, text, and symbols.

[0037] The operation guidance mark is superimposed on the display area of ​​the flexible display screen that is associated with the actual imaging position of the calibration object to provide operation guidance to the trainee.

[0038] Secondly, this embodiment provides an operation training system for an endoscope simulation teaching aid, the system including the endoscope simulation teaching aid and a controller; wherein, the endoscope simulation teaching aid includes a cavity with a variable structure; the cavity is used to simulate an organ, and the inner wall of the cavity is covered with a flexible display screen;

[0039] The controller is used to perform the operation training method of the endoscope simulation teaching aid described in the first aspect above.

[0040] Compared with related technologies, the operation training method and system of the endoscope simulation teaching aid provided in this embodiment includes an endoscope simulation teaching aid with a variable structure cavity for simulating organs, and the inner wall of the cavity is covered with a flexible display screen. By dynamically adjusting the physical structure of the cavity according to the target organ selected for endoscope training, and controlling the flexible display screen to load and display the preset texture map corresponding to the target organ, so that the cavity simulates the anatomical shape of the target organ; determining the calibration object corresponding to the target organ and generating the motion path of the calibration object; responding to the training start signal, controlling the flexible display screen to display the calibration object moving along the motion path, so that the trainee can operate the endoscope for simulation training. This solves the problem that it is difficult to effectively improve the trainee's operation ability by dynamically simulating real anatomical scenes during training. It realizes the dynamic simulation of real anatomical scenes during training, so as to effectively improve the trainee's operation ability.

[0041] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0042] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0043] Figure 1 This is a flowchart of an operation training method for an endoscope simulation teaching aid provided in one embodiment of this application;

[0044] Figure 2 This is a flowchart of a method for displaying a preset texture map according to an embodiment of this application;

[0045] Figure 3 This is a flowchart of a method for generating the motion path of a calibration object according to an embodiment of this application;

[0046] Figure 4 This is a flowchart illustrating the operation training method of an endoscope simulation teaching aid provided in one embodiment of this application;

[0047] Figure 5 This is a structural block diagram of an operation training system for an endoscope simulation teaching aid provided in one embodiment of this application.

[0048] In the picture: 10. Endoscope simulation teaching aid; 20. Controller. Detailed Implementation

[0049] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0050] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0051] This embodiment provides a method for training the operation of an endoscope simulation teaching aid. Figure 1 This is a flowchart of the operation training method of the endoscope simulation teaching aid in this embodiment, such as... Figure 1 As shown, the process includes the following steps:

[0052] Step S110: Based on the target organ selected during endoscopic training, dynamically adjust the physical structure of the cavity and control the flexible display screen to load the preset texture map corresponding to the target organ for display, so that the cavity simulates the anatomical shape of the target organ.

[0053] It should be noted that the endoscopic simulation teaching aid includes a variable-structure cavity whose inner wall is covered with a flexible display screen. The cavity can be adjusted to simulate different organ shapes, such as a spherical bladder. Specific implementation methods include, but are not limited to: using a reconfigurable box structure, adjusting the internal partition layout and the shape and size of the cavity to simulate different organs; or assembling the target cavity using modular units; or using a drive system to controllably expand and contract the flexible cavity. All of these methods aim to achieve adaptation and simulation of various organ shapes.

[0054] Specifically, during endoscopic simulation training, the physical structure of the cavity is dynamically adjusted in advance based on the target organ selected for the current endoscopic training (such as the bladder, stomach, uterus, bronchus, etc.) to match the shape of the cavity with that of the target organ. Simultaneously, a flexible display screen covering the inner wall of the cavity loads and displays a preset texture map corresponding to the target organ. In this way, through the coordinated control of the cavity structure and the display, a high-fidelity simulation of the anatomical morphology and tissue texture of the target organ is achieved.

[0055] The preset texture map corresponding to the target organ is used to reproduce the tissue texture of the organ with high fidelity on the flexible display screen. Its generation methods include, but are not limited to: generating the texture map through UV unwrapping and mapping based on a high-precision 3D model of the target organ; or generating it through post-processing of acquired, segmented, and calibrated real endoscopic surgical images; or synthesizing highly realistic medical images using artificial intelligence technologies such as generative adversarial networks. This embodiment does not limit the specific generation method of the texture map; any technical solution that can provide a realistic texture image can be adopted.

[0056] Step S120: Determine the calibration object corresponding to the target organ and generate the motion path of the calibration object;

[0057] Specifically, based on the selected target organ, corresponding markers are determined. Markers include target points, virtual lesions, etc., and each target organ may have one or more markers. Among them, target points are anatomical and functional reference anchors of the target organ, used to provide clear structural guidance or operational positioning, such as the pyloric marker protrusion of the stomach, vascular branch point target points, etc.; virtual lesions include, but are not limited to, polyps, masses, ulcers, or abnormal vascular patterns.

[0058] Furthermore, preset or programmable motion paths are generated for the calibrators to enhance the realism of endoscopic dynamic manipulation and tracking skills training. In practice, the path generation logic and method can be configured according to the training subject and difficulty level. For example, for basic training, the motion path is a simple linear translation or a constant arc trajectory; for advanced training, complex paths such as variable speed, variable direction, or spiral progression are used; for high-fidelity training, the motion path can integrate physiological motion characteristics and random perturbations, such as simulating the minute displacement of lesions under the influence of respiration or heartbeat. In addition, it supports importing organ or lesion motion data extracted from real clinical images to drive the calibrators to reproduce realistic physiological or pathological motion trajectories.

[0059] It should be noted that the above methods can be implemented independently or used in combination, and this embodiment does not limit the specific method of path generation.

[0060] In step S130, in response to the training start signal, the flexible display screen is controlled to display a calibration object moving along the motion path, so that the trainee can operate the endoscope for simulated training.

[0061] In response to the training start signal triggered by the trainee, the system calls up the corresponding calibration object information and its movement path based on the currently selected endoscope training information, and controls the flexible display screen to display the calibration object moving along the movement path. This provides the trainee with realistic visual targets and training scenarios for simulating operations such as dynamic tracking and precise positioning of the endoscope.

[0062] The process employs a corresponding display matching algorithm for real-time rendering, precisely mapping the moving calibration object onto the deformed cavity surface. Its core lies in obtaining the mapping relationship from a two-dimensional texture space to a three-dimensional display surface based on the real-time physical shape of the cavity, and driving the flexible display screen for adaptive display. Specific implementation methods include, but are not limited to: dynamically calculating the UV mapping equation based on real-time detected curvature data, performing geometric correction and resampling of the texture, and ultimately controlling the display screen to complete geometrically adaptive image rendering; or, directly calling the matching mapping relationship by querying a pre-calibrated shape-display parameter database, thereby quickly driving the display screen to complete image mapping and display; or, combining data from optical marker tracking or a distributed deformation sensor network to solve the display surface curvature in real time and drive image fitting.

[0063] Taking a reconfigurable box structure as an example, after starting training, the corresponding endoscope is inserted into the box, and the flexible display screen displays the corresponding calibration object. The trainee uses the endoscope to find the calibration object in the endoscopic image and adjusts the calibration object to be located in a designated area of ​​the endoscopic image. The designated area is usually an adjustable preset area superimposed on the image (such as a circular area). When the system detects that the calibration object is located in the designated area of ​​the image, it controls the calibration object to automatically start moving according to the motion path. During the movement of the calibration object, the trainee performs calibration object tracking training and acquires endoscopic images of the calibration object.

[0064] Subsequently, the evaluation and feedback phase begins. Endoscopic images of the calibrated objects collected by the trainees are acquired, and the corresponding endoscopic operation scores are evaluated based on these images. For example, image recognition algorithms can be used to analyze the endoscopic images to determine whether the trainees successfully acquired a specific morphological marker within the target organ. An endoscopic operation score is generated based on factors such as the completeness of the morphology in the image and the visibility of key features. Alternatively, a comprehensive analysis of the endoscopic images can be performed based on preset operation evaluation dimensions to obtain the corresponding endoscopic operation score. These operation evaluation dimensions include marker positioning accuracy and image clarity. Marker positioning accuracy is used to evaluate the matching degree between the actual imaging position of the calibrated object in the endoscopic image and the preset positioning benchmark, while image clarity is used to evaluate the clarity of the calibrated object's imaging.

[0065] Currently, endoscopic operation training mainly relies on fixed anatomical models, resulting in a single and rigid training scenario that cannot dynamically simulate real anatomical scenarios during training, thus making it difficult to effectively improve the operational skills of trainees.

[0066] Compared to existing technologies, this application dynamically adjusts the physical structure of the cavity based on the target organ selected for endoscopic training, and controls the flexible display screen to load and display a preset texture map corresponding to the target organ, so that the cavity simulates the anatomical morphology of the target organ; it determines the calibration object corresponding to the target organ and generates the motion path of the calibration object; in response to the training start signal, it controls the flexible display screen to display the calibration object moving along the motion path, so that the trainee can operate the endoscope for simulated training. Based on this, through real-time matching of cavity deformation and screen display, the target organ is simulated synchronously in terms of physical structure and visual texture, and the calibration object is controlled to move along the preset path, accurately reproducing the dynamic anatomical target in the real scene, and constructing a highly realistic endoscopic training environment of dynamic anatomical scene. This solves the problem that it is impossible to dynamically simulate the real anatomical scene in training, making it difficult to effectively improve the trainee's operating ability, and realizes the dynamic restoration of the real anatomical scene to effectively improve the trainee's endoscopic operating ability.

[0067] In some of these embodiments, the preset texture map is generated by UV unwrapping based on the 3D model of the target organ;

[0068] Reference Figure 2 As shown, step S110, which involves controlling the flexible display screen to load and display the preset texture map corresponding to the target organ, includes the following steps:

[0069] Step S111: Real-time detection of the bending curvature of the flexible display screen, and updating the surface parameter equation of the flexible display screen based on the bending curvature.

[0070] Step S112: Map the surface coordinates defined by the updated surface parametric equations to the UV plane to obtain the corresponding UV mapping equations;

[0071] Step S113: Based on the UV mapping equation, control the flexible display screen to load the preset texture map corresponding to the target organ for display.

[0072] In this embodiment, the preset texture map corresponding to the target organ is generated based on the UV unwrapping of the target organ's 3D model. Specifically, in the preprocessing stage, the 3D models of each target organ (such as the bladder, stomach, uterus, bronchus, etc.) are pre-unwrapped using 3D modeling software, establishing a precise coordinate correspondence between the model surface and the 2D texture image. This process unwrappes the 3D model surface into a 2D plane, assigning corresponding UV coordinates to each vertex of the model surface. U and V correspond to the horizontal and vertical axes of the texture image, respectively, thereby generating a UV coordinate map for each model. Based on this coordinate map, a highly realistic texture image is then applied to the model, ultimately forming the preset texture map (i.e., UV map) corresponding to each target organ. This ensures that when the 2D texture is displayed on a flexible display screen, it can be dynamically and accurately mapped to the surface of the 3D simulated organ according to the deformation of the cavity, thus maintaining the visual anatomical correctness and continuity of the tissue texture, providing a highly realistic visual environment for endoscopic simulation training.

[0073] During actual display, the curvature of the flexible display screen is detected in real time, and the surface parameter equation of the flexible display screen is updated based on the curvature. This surface parameter equation describes the current deformed surface of the flexible display screen. Subsequently, the surface coordinates defined by the updated surface parameter equation are mapped to the UV plane to obtain the corresponding UV mapping equation.

[0074] Taking a simulated spherical bladder as an example, the surface of a flexible display screen can be modeled as a sphere. Based on the detected curvature K, the spherical parameter equation of the flexible display screen is updated. The radius R = 1 / K in the sphere. The specific expression for the parametric equation of this sphere is as follows:

[0075] (1)

[0076] In equation (1), R represents spherical coordinates; R is the current cavity radius.

[0077] The spherical coordinates defined by equation (1) Mapping to the UV plane, the corresponding UV mapping equation is established, and the specific expression is as follows:

[0078] (2)

[0079] In equation (2), u and v are parametric coordinates on the UV plane. By mapping spherical coordinates to the UV plane, the texture is ensured to be at the extreme points. There is no distortion.

[0080] Furthermore, based on the UV mapping equation obtained through real-time calculation, coordinate transformation and sampling are performed on the preset texture map corresponding to the target organ, and the flexible display screen is controlled to display it.

[0081] Through this embodiment, dynamic UV mapping compensation enables real-time correction of texture coordinates when the flexible display screen bends or stretches as the cavity structure adjusts, thereby ensuring that the display of tissue textures (such as mucosal and vascular patterns) always conforms to anatomical morphology. This effectively overcomes the problems of image stretching, compression, or tearing that may be caused by physical deformation, and achieves high-precision adaptive display of the display content to the dynamic changes of the cavity structure, ensuring distortion-free texture display.

[0082] In some of these embodiments, reference is made to Figure 3 As shown, the motion path of generating the calibration object in step S120 includes the following steps:

[0083] Step S121: Obtain the selected training difficulty level for endoscopic training; the training difficulty level is divided based on the degree of physiological motion simulation required for endoscopic training.

[0084] Step S122: Generate the motion path of the calibration object that matches the training difficulty level.

[0085] In this embodiment, in response to user operation, the selected training difficulty level for the current training is obtained. This level system is constructed and divided according to the degree of physiological motion simulation required for endoscopic operation, and based on the determined training difficulty level, a matching calibration object movement path is dynamically generated.

[0086] For example, the training difficulty level is divided into three levels, including:

[0087] 1. Basic Training: This section simulates static or simple, regular motion environments, aiming to train basic equipment manipulation and positioning skills. At this stage, it generates predictable, highly regular trajectories such as linear or circular paths, suitable for beginners. For example, the specific expression for a linear path is as follows:

[0088] (3)

[0089] In equation (3), The initial position, This is the velocity vector.

[0090] 2. Advanced Training: Introducing complex movements with clear patterns to train the predictability and tracking stability of the operation. At this stage, parameterized models or algorithms are used to synthesize variable-speed curves, spirals, or regular complex paths simulating the orientation of specific anatomical structures. For example, a spiral path can be used to simulate the spiral structure of a bronchus or blood vessel, with the specific expression as follows:

[0091] (4)

[0092] In equation (4), R(t) is the time-varying radius, such as R(t) = R0 + αt, to simulate vasodilation; k controls the rotational progress; k controls the axial movement speed.

[0093] 3. High-fidelity training: This simulates dynamic scenarios including physiological movements (such as peristalsis and pulsation) and random environmental disturbances, aiming to train operational and real-time adaptability in a realistic and unpredictable in vivo environment. In this case, path generation integrates physiological models and stochastic processes. For example, a stochastic perturbation path based on a Brownian motion model can be used to simulate the minute displacement of lesions under the influence of respiration or heartbeat, the specific expression of which is as follows:

[0094] (5)

[0095] In equation (5), dt is approximately the time step; This is the drift coefficient (simulating directional motion); Noise intensity; This represents the Wiener process (simulating random perturbations).

[0096] This embodiment provides a clear logical basis and parameter configuration foundation for subsequent motion path generation based on clearly defined training difficulty levels, thereby enabling personalized and progressive skill training from basic operations to advanced responses. This method can dynamically configure and combine motion patterns of different complexities, effectively improving the scenario adaptability of simulated training.

[0097] In some embodiments, the operation training method of the above-mentioned endoscope simulation teaching aid further includes the following steps:

[0098] As the calibration object moves along the motion path, a corresponding noise image is generated based on the preset interference factors in the endoscopic operation.

[0099] Noisy images are superimposed onto the display screen of a flexible display.

[0100] In this embodiment, during the movement of the calibration object along the path, common sudden interferences encountered in endoscopic operations are dynamically simulated to enhance the realism of the simulation training. Specifically, based on preset interference factors (such as simulated bleeding, tissue fluid exudation, electrocoagulation fumes, or lens water stains) and / or interference intensity parameters, highly realistic noise images are generated in real time using image generation algorithms such as generative adversarial networks and diffusion models. These noise images can realistically simulate visual effects such as localized bloodstains, gradually spreading fog-like blurring, or droplet adhesion.

[0101] Furthermore, the generated noisy image is precisely superimposed onto the display screen presented on the flexible display. This process visually simulates the sudden situation where interference objects adhere to the lens surface, causing partial or complete obstruction of the field of view, which helps to train operational stability and the ability to deal with emergencies.

[0102] Understandably, the superposition and fusion of the aforementioned noisy image and endoscopic view can be achieved using various fusion formulas. Taking the Alpha fusion formula as an example, its specific expression is as follows:

[0103] (6)

[0104] (7)

[0105] (8)

[0106] In equations (6) to (8), For screen coordinates The final output pixel value after fusion; C is the pixel value of the noisy image at the corresponding coordinate point; The original endoscope simulation scene in coordinates Pixel value at; To act on coordinates The composite transparency coefficient at the location is calculated using equation (7); This is the noise distribution probability map; T is the global transparency coefficient, whose value can be dynamically adjusted between 0 and 1. The spatial attenuation factor is defined by equation (8); d is the pixel value. arrive The distance to the nearest boundary of the defined region; This is the feathering intensity factor, for example, set to 5 pixel units.

[0107] In this embodiment, during the movement of the calibration object along the motion path, a corresponding noise image is generated based on the preset interference factors in the endoscopy operation, and the noise image is superimposed on the display screen of the flexible display screen. This achieves controllable and highly realistic visual interference simulation, which can train the operator to maintain operational stability and make accurate judgments and take correct countermeasures under complex conditions such as limited field of vision or degraded image quality.

[0108] In some embodiments, the operation training method of the above-mentioned endoscope simulation teaching aid further includes the following steps:

[0109] Acquire endoscopic images of the calibration object collected by the trainee;

[0110] Based on preset operation evaluation dimensions, the endoscopic images are evaluated to obtain the corresponding endoscopic operation score;

[0111] Among them, the operational evaluation dimension includes at least the marker positioning accuracy; the marker positioning accuracy is used to evaluate the matching degree between the actual imaging position of the marker in the endoscopic image and the preset positioning reference.

[0112] Specifically, during the evaluation and feedback phase, endoscopic images captured by trainees using an endoscope during the simulated operation are obtained. Depending on the training objective, these endoscopic images can be single frames, continuous image sequences, or sets of images from different angles or operational stages; no specific limitations are imposed here. For example, for static positioning tasks, a single representative endoscopic image can be selected for evaluation; for dynamic tracking tasks, a continuous image sequence encompassing the entire movement of the calibration object needs to be analyzed.

[0113] Based on preset operational evaluation dimensions, the acquired endoscopic images are evaluated and analyzed to obtain an endoscopic operation score. When there are multiple operational evaluation dimensions, the scores corresponding to each dimension can be weighted to obtain the final endoscopic operation score. Operational evaluation dimensions include marker positioning accuracy and image sharpness. Marker positioning accuracy assesses the matching degree between the actual imaging position of the marker in the endoscopic image and the preset positioning benchmark. Specifically, it can be achieved based on parameters such as the ratio of the marker's bounding box area to the area of ​​a specified region, and the offset between the marker's center coordinates and the image center coordinates. Image sharpness assesses the clarity of the marker's image and can be calculated using image blur detection (such as Laplacian variance or deep neural network predictive motion blur kernels) to calculate the image sharpness score.

[0114] Taking the positioning accuracy of markers as an example, the operational score is calculated using the following two indicators:

[0115] 1) Determining the area of ​​the bounding box

[0116] Detect the bounding box of the calibration object, compare its area with the area of ​​a preset specified region, and calculate the score. The specific expression is as follows:

[0117] (9)

[0118] In equation (9), Indicates the area of ​​the bounding box of the calibration object; This indicates the area of ​​the specified region (the ideal range is within ±10% of the threshold).

[0119] 2) Determination of center position offset

[0120] Calculate the Euclidean distance between the coordinates of the calibration object center and the coordinates of the image center, and calculate the corresponding score based on the distance value. The specific expression is as follows:

[0121] (10)

[0122] In equation (10), (x0, y0) represent the coordinates of the image center; (x c y c () indicates the coordinates of the center of the calibration object.

[0123] Then, the two scores are weighted and fused (the weights can be dynamically configured) to obtain the score corresponding to the marker positioning accuracy. The specific expression is as follows:

[0124] (11)

[0125] The scores of the two indicators are weighted to obtain the score corresponding to the marker positioning accuracy dimension, which is used as the endoscopy operation score.

[0126] This embodiment acquires endoscopic images of the calibration object collected by the trainee, evaluates the endoscopic images based on preset operation evaluation dimensions, and obtains the corresponding endoscopic operation score. This allows for a quantitative assessment of the trainee's operational level, accurately reflecting the operational stability of the endoscope.

[0127] In some embodiments, the operation training method of the above-mentioned endoscope simulation teaching aid further includes the following steps:

[0128] As the calibration object moves along the motion path, the calibration object displayed on the flexible display screen is controlled to undergo preset shape changes at preset times, so that trainees can use the endoscope to collect the shape of the marker.

[0129] Specifically, as the calibration object moves along the path, the calibration object displayed on the flexible display screen can undergo preset shape changes at preset times, such as changes in size, color, or texture, and the changed shape can be displayed within a preset time period t. For example, it can simulate the dynamic changes of an active bleeding point from oozing to spurting.

[0130] Accordingly, after the aforementioned morphological transformation occurs, the trainee needs to operate an endoscope to capture an image of a calibration object showing the preset morphology. This training design simulates a scenario where typical features of lesions need to be captured in a timely manner, significantly improving the realism of the training.

[0131] This embodiment makes the training scenario more closely resemble the uncertainties and time pressures faced in real-world scenarios. Trainees not only need to track moving targets, but also need to judge and capture the key shapes of the target object within a limited time window, thereby helping to improve the trainees' operational stability and ability to deal with emergencies.

[0132] In some embodiments, the operation training method of the above-mentioned endoscope simulation teaching aid further includes the following steps:

[0133] Acquire endoscopic images of the calibration object collected by the trainee.

[0134] Based on preset operation evaluation dimensions, the endoscopic images are evaluated to obtain the corresponding endoscopic operation score; the operation evaluation dimensions include marker positioning accuracy, keyframe response speed and image clarity.

[0135] Among them, the marker positioning accuracy is used to evaluate the matching degree between the actual imaging position of the marker in the endoscopic image and the preset positioning reference; the keyframe response speed is used to evaluate the operation time delay from the appearance of the preset morphological marker to the triggering of endoscopic image acquisition; and the image sharpness is used to evaluate the clarity of the marker imaging.

[0136] Specifically, during the evaluation and feedback phase, endoscopic images captured by trainees using an endoscope during simulated operations are obtained. Depending on the training objective, these endoscopic images can be single frames, continuous image sequences, or sets of images from different angles or operational stages; no specific limitations are imposed here.

[0137] Based on preset operational evaluation dimensions, the acquired endoscopic images are evaluated and analyzed to obtain an endoscopic operation score. Operational evaluation dimensions include marker positioning accuracy, keyframe response speed, and image sharpness. Marker positioning accuracy assesses the matching degree between the actual imaging position of the marker in the endoscopic image and the preset positioning reference. Specifically, it can be achieved based on parameters such as the ratio of the marker's bounding box area to the area of ​​a specified region, and the offset between the marker's center coordinates and the image center coordinates. Keyframe response speed assesses the operation time delay from the appearance of a preset morphological marker to triggering endoscopic image acquisition. Image sharpness assesses the clarity of the marker imaging, and can be calculated using image blur detection (such as Laplacian variance or deep neural network predictive motion blur kernels) to calculate an image sharpness score.

[0138] The following section will explain in detail the evaluation methods for endoscopic operations, taking marker positioning accuracy, keyframe response speed, and image clarity as examples.

[0139] 1. Assessment of marker positioning accuracy

[0140] The accuracy of marker positioning is evaluated by calculating the conformity between the area of ​​the detected bounding box of the marker and the area of ​​the specified region, as well as the offset between the center coordinates of the marker and the center coordinates of the image. The specific calculation methods are given in equations (9) to (11) above, and will not be repeated here. The operation score corresponding to this dimension is denoted as... .

[0141] 2. Keyframe response speed evaluation

[0142] As the calibration object moves along the path, its shape undergoes specific transformations at preset time points. The trainer must successfully acquire endoscopic images of the calibration object within the maximum permissible delay window after the shape transformation. Based on this, the operation score corresponds to the keyframe response speed dimension. Its specific expression is as follows:

[0143]

[0144] In equation (12), Indicates the timestamp of the appearance of the preset shape marker; The timestamp indicating the endoscopic image captured by the preset morphological calibration object; This represents the maximum allowable delay window that can be dynamically configured, for example... .

[0145] 3. Image sharpness assessment

[0146] Image sharpness assessment methods based on image gradients are employed, such as using Laplacian variance to calculate image sharpness scores. The specific expression is as follows:

[0147]

[0148] In equation (13), Laplacian represents Laplacian operator convolution; Var() represents variance calculation; This indicates the resolution threshold.

[0149] Finally, the scores from the three operational evaluation dimensions are weighted and fused to obtain the endoscopic operation score. The specific expression is as follows:

[0150] (14)

[0151] In equation (14), the score weights corresponding to each operation evaluation dimension can be dynamically adjusted. For example, different score weights can be set for different operation evaluation dimensions based on different training difficulty levels, training objectives, etc.

[0152] This embodiment acquires endoscopic images of the calibrated objects collected by the trainees. Based on operational evaluation dimensions such as calibrated object positioning accuracy, keyframe response speed, and image clarity, the endoscopic images are evaluated to obtain an endoscopic operation score. This multi-dimensional quantitative assessment of the trainees' operational capabilities provides objective and detailed feedback for endoscopic skills training, effectively guiding and promoting the improvement of trainees' comprehensive skills.

[0153] In some embodiments, when there are multiple endoscopic images, the endoscopic images are evaluated based on a preset operation evaluation dimension to obtain the corresponding endoscopic operation score, including the following steps:

[0154] Based on the preset operation evaluation dimensions, each endoscopic image collected by the trainee is evaluated to obtain a score corresponding to each endoscopic image;

[0155] The average score of each endoscopic image is calculated to obtain the endoscopic operation score.

[0156] Specifically, if multiple endoscopic images are acquired during training, such as n endoscopic images acquired from specific morphologies of n calibrators, or n endoscopic images acquired from different operational angles of the same calibrator, each endoscopic image is evaluated to obtain a score corresponding to each endoscopic image. It is understood that the score corresponding to each endoscopic image can be obtained based on the operational evaluation dimension described in the aforementioned embodiments, and will not be elaborated further here.

[0157] Next, the scores corresponding to each endoscopic image are averaged, and the resulting average score is used as the final endoscopic operation score reflecting the overall level of the operation. In practice, the average scores of multiple endoscopic images under each operation evaluation dimension can also be pre-assessed, and then weighted and averaged to obtain the corresponding endoscopic operation score.

[0158] This embodiment enables quantitative evaluation of trainees' continuous operation process, making the final operation score more stable and comprehensive in reflecting the trainees' overall skill level throughout the operation period. It is especially suitable for tasks such as dynamic tracking, further improving the objectivity and reliability of the evaluation results.

[0159] In some embodiments, the operation training method of the above-mentioned endoscope simulation teaching aid further includes the following steps:

[0160] When trainees collect endoscopic images of the calibration object, corresponding operation correction information is generated based on the relative relationship between the actual imaging position of the calibration object in the endoscopic image and the preset positioning reference.

[0161] The operation correction information is converted into a corresponding operation guidance identifier; the operation guidance identifier is a combination of one or more of the following: graphics, text, and symbols;

[0162] Operation guidance labels are overlaid onto the display area of ​​the flexible display screen that is associated with the actual imaging position of the calibration object to provide operation guidance to trainees.

[0163] In this embodiment, when the trainer acquires endoscopic images of the calibration object, the integrated target detection and tracking algorithm performs real-time analysis on the images to calculate the coordinates of the center of the calibration object in the image. and its bounding box area The system compares the real-time calculation results with the preset positioning benchmark and generates corresponding operation correction information based on the comparison results.

[0164] Furthermore, the operation correction information is converted into corresponding operation guidance labels, which can be represented as graphical prompts, dynamic directional arrows, brief text, or a combination thereof. For example, the operation guidance labels "zoom in" and "zoom out" can use a combination of dynamic arrows and text to ensure that trainees can quickly identify the guidance content.

[0165] Subsequently, the operation guidance labels are overlaid onto the display area of ​​the flexible display screen that is associated with the actual imaging position of the calibration object, thereby providing trainees with intuitive operation guidance. The associated display area can be set as an adjacent area to the imaging area of ​​the calibration object, the outer edge of the boundary, or the periphery of the center, creating a clear spatial correspondence between the operation guidance labels and the imaging position of the calibration object.

[0166] More specifically, the area of ​​the bounding box of the calibrated object. Area of ​​the specified region A comparison is performed. If the area of ​​the boundary frame of the calibration object is greater than the area of ​​the specified region, then... If the bounding box area of ​​the calibrated object is smaller than the specified area, then an operation guide icon for "zooming in" will be generated; if the bounding box area of ​​the calibrated object is smaller than the specified area, then... Then, a "zoom in" operation guide icon is generated. This logic is consistent with the judgment condition of the aforementioned formula (9). Furthermore, the coordinates of the calibration object center and the coordinates of the image center are... Compare them to calculate the offset between their centers. Its specific expression is as follows:

[0167] (15)

[0168] In equation (15), the width W and height H represent the image resolution. Based on the sign and magnitude of the offset, operation guidance indicators such as "move left / right" or "move up / down" are generated. In specific implementations, this can be changed when the offset exceeds a preset threshold. At that time, a corresponding operation guidance identifier is generated. For example, if If so, a "Move right" prompt will be displayed; if If so, a "Move Down" prompt will be displayed. Among them, The preferred value is 5% of the image width.

[0169] It is understood that the visual cues in this embodiment are designed to provide clear guidance while ensuring that they do not obstruct or interfere with the observation of the main anatomical structures.

[0170] In this way, through real-time visual feedback, trainees can intuitively understand the deviation between their current operational state and the ideal state, and make targeted adjustments based on clear visual guidance. This mechanism enhances the immediate feedback and purposefulness of skills training, thereby effectively accelerating the formation of correct operating habits.

[0171] The present embodiment will be described and explained below through specific examples.

[0172] The endoscopic simulation teaching aid includes a variable-structure cavity with a reconfigurable box structure, and the inner wall of the cavity is covered with a flexible display screen. Based on this, different organ shapes can be simulated by adjusting the layout of its internal partitions and the shape and size of the cavity.

[0173] Reference Figure 4 As shown, the operation training method of this endoscope simulation teaching aid includes the following steps:

[0174] 1. Initialization Phase

[0175] In response to user input, the target organ for this endoscopic training is determined. Subsequently, based on a preset template corresponding to the target organ, the physical structure of the cavity is dynamically adjusted to match the shape of the cavity with that of the target organ.

[0176] Furthermore, the bending curvature of the flexible display screen is detected in real time, and the surface parameter equation of the flexible display screen is updated based on the bending curvature. The surface coordinates defined by the updated surface parameter equation are mapped to the UV plane to obtain the corresponding UV mapping equation. Then, based on the UV mapping equation, the flexible display screen is controlled to load the preset texture map corresponding to the target organ for display.

[0177] Subsequently, based on the selected target organ, a corresponding training calibrator is determined, and in response to the training difficulty level set by the user, the motion path of the calibrator is automatically generated. The complexity of this path is related to the difficulty level: basic training uses a simple linear trajectory; advanced training uses a spiral path; and high-fidelity training uses a random perturbation path.

[0178] 2. Dynamic Training Phase

[0179] After training begins, the trainee places the endoscope into the training kit. The system controls the flexible display screen to show the initially stationary calibration object. The trainee uses the endoscope to locate the calibration object in the endoscopic view and adjusts it to fit within a designated area on the screen. This area is typically an adjustable, pre-defined circular area superimposed on the image. Once the system detects that the calibration object is within the designated area, it automatically begins to move the calibration object along a path. The trainee must perform dynamic tracking training during the calibration object's movement and acquire multiple frames of endoscopic images.

[0180] During the movement of the calibration object, the calibration object displayed on the flexible display screen can be controlled to undergo preset shape changes at preset times, such as controlling the calibration object to flash red, and displaying the changed shape within a preset time period. After the above shape change occurs, the trainee needs to operate the endoscope to take an image of the calibration object displaying the preset shape. In addition, based on preset interference factors in the endoscopy operation (such as bloodstains, smoke, tissue fluid, water stains), corresponding noise images can be generated and superimposed on the display screen to simulate sudden visual interference.

[0181] During the image acquisition process for trainees, integrated target detection and tracking algorithms are used to analyze the images in real time, calculate the coordinates of the center of the calibrated object and the area of ​​its bounding box, and compare the real-time calculation results with a preset positioning benchmark. Based on the comparison results, corresponding operation correction information is generated and converted into corresponding operation guidance indicators (such as "move left" or "zoom out"), which are then overlaid on the screen to provide real-time visual guidance for trainees.

[0182] 3. Evaluation and Feedback Phase

[0183] The endoscopic images acquired by trainees during simulated operations are obtained. Based on multiple operational evaluation dimensions (such as marker positioning accuracy, keyframe response speed, and image clarity), the acquired endoscopic images are evaluated and analyzed, and scores for each dimension are calculated. Finally, the endoscopic operation score for this training is obtained by combining the scores.

[0184] Finally, based on the above evaluation results, a detailed training report is generated. This report, through the scoring and analysis of each operational evaluation dimension, objectively reflects the trainees' operational level and the learning effectiveness of this training, providing a clear basis for skills improvement.

[0185] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0186] This embodiment also provides an operation training system for an endoscope simulation teaching aid, referring to... Figure 5 As shown, the system includes an endoscope simulation teaching aid 10 and a controller 20. The endoscope simulation teaching aid 10 includes a variable-structure cavity for simulating organs, and the inner wall of the cavity is covered with a flexible display screen. The controller 20 is communicatively connected to the endoscope simulation teaching aid 10 and is used to execute the steps in any of the above method embodiments, thereby driving the teaching aid to complete dynamic scene construction, training process control, and operational evaluation feedback. It is understood that, in specific implementations, the controller 20 can also be integrated inside the endoscope simulation teaching aid 10. This system is used to implement the above embodiments and preferred embodiments; details already described will not be repeated.

[0187] This embodiment also provides a computer device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0188] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0189] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0190] S1, based on the target organ selected during endoscopic training, dynamically adjusts the physical structure of the cavity and controls the flexible display screen to load the preset texture map corresponding to the target organ for display, so that the cavity simulates the anatomical shape of the target organ.

[0191] S2, determine the calibration object corresponding to the target organ, and generate the motion path of the calibration object;

[0192] S3, in response to the training start signal, controls the flexible display screen to display a calibration object moving along the motion path, so that the trainee can operate the endoscope for simulated training.

[0193] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0194] Furthermore, in conjunction with the operation training methods for the endoscopic simulation teaching aids provided in the above embodiments, this embodiment can also provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any one of the operation training methods for the endoscopic simulation teaching aids in the above embodiments.

[0195] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0196] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0197] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0198] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. An operation training method of an endoscope simulation teaching tool, characterized by, The endoscope simulation teaching aid comprises a cavity with variable structure; the cavity is used for simulating an organ, and an inner wall of the cavity is covered with a flexible display screen; the method comprises: According to a target organ selected for endoscope training, dynamically adjusting a physical structure of the cavity, and controlling the flexible display screen to load and display a preset texture map corresponding to the target organ, so that the cavity simulates an anatomical morphology of the target organ; Determining a calibration object corresponding to the target organ, and generating a motion path of the calibration object; In response to a training start signal, controlling the flexible display screen to display the calibration object moving along the motion path, so that a trainee operates an endoscope for simulation training.

2. The method of claim 1, wherein the endoscope simulation training device is a simulator for a flexible endoscope. The preset texture map is generated based on UV unfolding of a three-dimensional model of the target organ; The control of the flexible display screen to load and display the preset texture map corresponding to the target organ comprises: Real-time detection of bending curvature of the flexible display screen, and updating of a curved surface parameter equation of the flexible display screen based on the bending curvature; Mapping of curved surface coordinates defined by the updated curved surface parameter equation to a UV plane to obtain a corresponding UV mapping equation; Based on the UV mapping equation, controlling the flexible display screen to load and display the preset texture map corresponding to the target organ.

3. The method of claim 1, wherein the endoscope simulation training device is a simulator for a flexible endoscope. The generation of the motion path of the calibration object comprises: Obtaining a training difficulty level selected for endoscope training; the training difficulty level is divided based on physiological motion fidelity required for simulation of endoscope training; Generating the motion path of the calibration object matching the training difficulty level.

4. The method of claim 1, wherein the endoscope simulation training device is a simulator for a flexible endoscope. The method further comprises: During the motion of the calibration object along the motion path, generating a corresponding noise image according to a preset interference factor in endoscope operation; Superimposing the noise image to a display picture of the flexible display screen.

5. The method of claim 1, wherein the endoscope simulation training device is a simulator for a flexible endoscope. The method further comprises: Obtaining an endoscope image of the calibration object operated and collected by the trainee; Based on a preset operation evaluation dimension, evaluating the endoscope image to obtain a corresponding endoscope operation score; The operation evaluation dimension at least comprises a marker positioning accuracy; the marker positioning accuracy is used for evaluating a matching degree between an actual imaging position of the calibration object in the endoscope image and a preset positioning reference.

6. The method of claim 1, wherein the endoscope simulation training device is a simulator for a flexible endoscope. The method further comprises: During the motion of the calibration object along the motion path, controlling the calibration object displayed by the flexible display screen to perform preset morphology transformation at a preset time, so that the trainee operates an endoscope for marker morphology collection training.

7. The method of claim 6, wherein the endoscope simulator is a simulator for training a user to perform an operation on a patient's body cavity. The method further comprises: Obtaining an endoscope image of the calibration object operated and collected by the trainee; Based on a preset operation evaluation dimension, evaluating the endoscope image to obtain a corresponding endoscope operation score; the operation evaluation dimension comprises a marker positioning accuracy, a key frame response speed, and an image clarity. The marker positioning accuracy is used to evaluate the matching degree between the actual imaging position of the calibration object in the endoscope image and the preset positioning reference; the key frame response speed is used to evaluate the operation time delay from the appearance of the preset morphological marker to triggering the endoscope image acquisition; and the image definition is used to evaluate the definition of the calibration object imaging.

8. The method of claim 5 or claim 7, wherein The endoscope image is a plurality of endoscope images. The endoscope image is evaluated based on the preset operation evaluation dimension to obtain the corresponding endoscope operation score, including: Each of the endoscope images collected by the training personnel is evaluated based on the preset operation evaluation dimension to obtain the score corresponding to each of the endoscope images; The scores corresponding to each of the endoscope images are averaged to obtain the endoscope operation score.

9. The method of claim 1, wherein the endoscope simulator is a simulator for a flexible endoscope. The method further includes: When the training personnel collects the endoscope image of the calibration object, operation correction information corresponding to the relative relationship between the actual imaging position of the calibration object in the endoscope image and the preset positioning reference is generated; The operation correction information is converted into corresponding operation guide marks; the operation guide marks are a combination of one or more of graphics, characters and symbols; The operation guide marks are superimposed on the display area of the flexible display screen associated with the actual imaging position of the calibration object to provide operation guidance to the training personnel.

10. A training system for operating an endoscope simulation teaching aid, characterized in that, The system includes the endoscope simulation teaching aid and a controller; wherein the endoscope simulation teaching aid includes a cavity with variable structure; the cavity is used to simulate an organ, and the inner wall of the cavity is covered with a flexible display screen; The controller is used to execute the operation training method of the endoscope simulation teaching aid according to any one of claims 1 to 9.

Images