Endoscope operation evaluation method and system and computer equipment

Abstract

The invention relates to an endoscope operation evaluation method and system and computer equipment, and the endoscope operation evaluation method comprises the steps: obtaining an operation image sequence in the process that an operator executes endoscope operation on a simulated body cavity; wherein a plurality of identification patterns are preset in the simulated body cavity; the operation image sequence is acquired aiming at each identification pattern, and the operation image sequence comprises operation images corresponding to the endoscope operation stage; based on a preset evaluation index of the endoscope operation stage, analyzing the operation image corresponding to the endoscope operation stage to obtain related feature information of each identification pattern in the operation image; and based on the related feature information of each identification pattern, generating an operation evaluation result corresponding to the endoscope operation stage. According to the method and the device, the problem that the endoscope operation of an operator cannot be accurately evaluated is solved, and the skill level of the operator is accurately evaluated.

Description

Technical Field

[0001] This application relates to the field of endoscopy technology, and in particular to methods, systems and computer equipment for evaluating endoscopic procedures. Background Technology

[0002] The standardization and precision of endoscopic procedures directly affect treatment outcomes and patient safety; therefore, skills training and assessment for operators are crucial. Currently, skills assessment for endoscopic procedures in existing simulation training environments still relies primarily on subjective, experience-based judgment, which fails to provide accurate evaluation of operators' endoscopic skills.

[0003] There is currently no effective solution to the problem that related technologies cannot accurately assess the endoscopic procedures performed by operators. Summary of the Invention

[0004] This embodiment provides an endoscopic operation assessment method, system, and computer device to address the problem in related technologies that cannot accurately assess the endoscopic operations performed by operators.

[0005] Firstly, this embodiment provides a method for evaluating endoscopic procedures, including:

[0006] A sequence of operation images is acquired during the process of an operator performing an endoscopy on a simulated body cavity; wherein, the simulated body cavity is pre-set with multiple marker patterns; the operation image sequence is acquired for each of the marker patterns, and the operation image sequence includes operation images corresponding to the endoscopic operation stage;

[0007] Based on the preset evaluation indicators of the endoscopic operation stage, the operation images corresponding to the endoscopic operation stage are analyzed to obtain the relevant feature information of each of the marked patterns in the operation images;

[0008] Based on the relevant feature information of each of the aforementioned identification patterns, an operational evaluation result corresponding to the endoscopic operation stage is generated.

[0009] In some of these embodiments, the endoscopic procedure phase is a combination of one or more of the following: a cavity creation phase, an intracavitary observation phase, an intracavitary procedure phase, and a suturing phase.

[0010] In some embodiments, the relevant feature information includes the integrity, tilt, and degree of distortion of each of the identification patterns;

[0011] The process of generating an operational evaluation result corresponding to the endoscopic operation stage based on the relevant feature information of each of the aforementioned identifier patterns includes:

[0012] When the endoscopic operation stage is the body cavity creation stage, based on the relevant feature information of each of the marked patterns, it is analyzed whether each of the marked patterns matches a preset standard marked pattern, and the analysis result of each marked pattern is obtained;

[0013] Based on the analysis results of each of the aforementioned identifier patterns, the operational evaluation results corresponding to the body cavity establishment stage are generated.

[0014] In some embodiments, the relevant feature information includes the actual observation order of each of the identification patterns;

[0015] The process of generating an operational evaluation result corresponding to the endoscopic operation stage based on the relevant feature information of each of the aforementioned identifier patterns includes:

[0016] When the endoscopic operation stage is the intracavitary observation stage, it is determined whether the actual observation order of each of the marked patterns is the same as the preset observation order;

[0017] Based on the observation sequence judgment results, the operation evaluation results corresponding to the intracavitary observation stage are generated.

[0018] In some embodiments, generating the operation evaluation result corresponding to the endoscopic operation stage based on the relevant feature information of each of the identified patterns further includes:

[0019] Obtain the actual endoscopic trajectory and the actual endoscopic posture corresponding to each of the marked patterns during the intracavitary observation phase;

[0020] Determine whether the actual endoscope trajectory corresponding to each of the marked patterns matches the preset endoscope trajectory, and whether the actual endoscope posture corresponding to each of the marked patterns is within the preset posture range;

[0021] Based on the observation sequence judgment result, endoscope trajectory judgment result, and endoscope posture judgment result, the operation evaluation result corresponding to the intracavitary observation stage is generated.

[0022] In some embodiments, the relevant feature information includes the actual cutting order and cutting quality of each of the marking patterns;

[0023] The process of generating an operational evaluation result corresponding to the endoscopic operation stage based on the relevant feature information of each of the aforementioned identifier patterns includes:

[0024] When the endoscopic operation stage is the intracavitary operation stage, it is determined whether the actual cutting order of each of the marked patterns is the same as the preset cutting order;

[0025] Based on the judgment results of each of the aforementioned marking patterns and the cutting quality, the operation evaluation results corresponding to the intracavitary operation stage are generated.

[0026] In some embodiments, the relevant feature information includes the actual stitching sequence and stitching quality of each of the identification patterns;

[0027] The process of generating an operational evaluation result corresponding to the endoscopic operation stage based on the relevant feature information of each of the aforementioned identifier patterns includes:

[0028] When the endoscopic operation stage is the suturing stage, it is determined whether the actual suturing sequence of each of the marked patterns is the same as the preset suturing sequence.

[0029] Based on the judgment results of each of the aforementioned marking patterns and the suturing quality, the operation evaluation result corresponding to the suturing stage is generated.

[0030] In some of these embodiments, the endoscopic procedure consists of multiple stages;

[0031] After generating the operation evaluation result corresponding to the endoscopic operation stage based on the relevant feature information of each of the aforementioned identifier patterns, the method further includes:

[0032] The operation evaluation results corresponding to each of the endoscopic operation stages are weighted and fused to obtain the endoscopic operation evaluation result of the operator.

[0033] Secondly, this embodiment provides an endoscopic operation evaluation system, including:

[0034] The acquisition module is used to acquire a sequence of operation images during the endoscopic operation performed by the operator on the simulated body cavity; wherein, the simulated body cavity is pre-set with multiple marker patterns; the operation image sequence is acquired for each of the marker patterns, and the operation image sequence includes operation images corresponding to the endoscopic operation stage;

[0035] The analysis module is used to analyze the operation image corresponding to the endoscopy operation stage based on the preset evaluation indicators of the endoscopy operation stage, so as to obtain the relevant feature information of each of the marked patterns in the operation image;

[0036] The evaluation module is used to generate an operation evaluation result corresponding to the endoscopic operation stage based on the relevant feature information of each of the aforementioned identification patterns.

[0037] Thirdly, this embodiment provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the endoscopic operation evaluation method described in the first aspect above.

[0038] Compared with related technologies, the endoscopic operation assessment method, system, and computer equipment provided in this embodiment acquire a sequence of operation images during the endoscopic operation performed by the operator on a simulated body cavity. Multiple marker patterns are preset within the simulated body cavity. The operation image sequence is acquired for each marker pattern and includes operation images corresponding to each stage of the endoscopic operation. Based on preset assessment indicators for each stage of the endoscopic operation, the operation images corresponding to that stage are analyzed to obtain relevant feature information of each marker pattern in the operation images. Based on the relevant feature information of each marker pattern, an operation assessment result corresponding to each stage of the endoscopic operation is generated. This solves the problem of the inability to accurately assess the operator's endoscopic operation and achieves accurate assessment of the operator's skill level.

[0039] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0040] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0041] Figure 1 This is a flowchart of an endoscopic operation evaluation method provided in an embodiment of this application;

[0042] Figure 2 This is a schematic diagram of different status indicator patterns provided in one embodiment of this application;

[0043] Figure 3 This is a schematic diagram of a marking pattern using numerical designation provided in an embodiment of this application;

[0044] Figure 4 This is a schematic diagram of the cutting sequence of the marking pattern provided in one embodiment of this application;

[0045] Figure 5 This is a schematic diagram of the stitching sequence of the marking pattern provided in one embodiment of this application;

[0046] Figure 6 This is a schematic flowchart of an endoscopic operation evaluation method provided in an embodiment of this application;

[0047] Figure 7 This is a structural block diagram of an endoscopic operation evaluation system provided in one embodiment of this application.

[0048] In the diagram: 10. Acquisition module; 20. Analysis module; 30. Evaluation module. Detailed Implementation

[0049] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0050] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0051] This embodiment provides a method for evaluating endoscopic procedures. Figure 1 This is a flowchart of the endoscopic operation evaluation method in this embodiment, as shown below. Figure 1 As shown, the process includes the following steps:

[0052] Step S110: Obtain the sequence of operation images during the endoscopic operation performed by the operator on the simulated body cavity; wherein, multiple marker patterns are preset in the simulated body cavity; the operation image sequence is acquired for each marker pattern, and the operation image sequence includes operation images corresponding to the endoscopic operation stage;

[0053] Specifically, operators are trained using simulated body cavities such as the abdominal cavity and thoracic cavity. Inside the simulated body cavities, multiple marker patterns are pre-set as evaluation references. These marker patterns can be circular, star-shaped, or other similar shapes, and are clearly marked with numbers (1~n) or letters (such as a, b, ...) to distinguish different marker patterns during subsequent image acquisition and analysis.

[0054] When the operator performs endoscopic procedures on the simulated body cavity, a corresponding sequence of images is simultaneously acquired. This sequence is captured for each marked pattern, and each image in the sequence corresponds to a different stage of the endoscopic procedure. These stages include cavity creation, intracavitary observation, intracavitary manipulation, and suturing.

[0055] Step S120: Based on the preset evaluation indicators of the endoscopic operation stage, analyze the operation images corresponding to the endoscopic operation stage to obtain the relevant feature information of each symbol pattern in the operation image.

[0056] It should be noted that corresponding evaluation indicators are set for each stage of the endoscopic procedure. This embodiment performs targeted analysis and processing of the operation images corresponding to each stage of the endoscopic procedure based on the preset evaluation indicators. Its core lies in calculating the relevant feature information of each marker pattern associated with the preset evaluation indicators according to the specific requirements of those indicators.

[0057] For example, in the cavity establishment stage, the preset evaluation index is the qualification assessment of cavity establishment. At this time, the integrity, tilt state, and degree of distortion of each marker pattern in the simulated cavity are extracted as relevant feature information. In the cavity observation stage, the preset evaluation index is the standardization assessment of the observation sequence, that is, whether the operator observes according to the preset sequence. At this time, the actual observation sequence of each marker pattern in the simulated cavity is identified as relevant feature information. In the cavity operation stage, the preset evaluation index is the standardization assessment of the operation (such as cutting) sequence and the operation stability assessment. At this time, the actual operation sequence and operation quality of each marker pattern in the simulated cavity are identified as relevant feature information. In the suturing stage, the preset evaluation index is the standardization assessment of the suturing sequence and the suturing result assessment. At this time, the actual suturing sequence and suturing quality of each marker pattern in the simulated cavity are identified as relevant feature information.

[0058] The aforementioned image analysis process can be implemented using well-trained deep learning network models or general image processing algorithms. For example, during the abdominal cavity establishment phase of laparoscopic surgery training, normal, distorted, and incomplete pattern data are collected to train three independent abdominal cavity evaluation models, which are then used to evaluate the integrity, tilt, and distortion degree of the marker patterns, respectively. Alternatively, edge detection and contour analysis can be used to quantify the integrity of the marker patterns, principal component analysis can be used to calculate the tilt angle of the patterns, and template matching error or feature point deformation can be used to measure the degree of distortion.

[0059] Step S130: Based on the relevant feature information of each logo pattern, generate the operation evaluation results corresponding to the endoscopic operation stage.

[0060] After acquiring the aforementioned relevant feature information, the system automatically generates operational evaluation results for each endoscopic procedure stage associated with the relevant feature information, based on a pre-set algorithm model or judgment rules. For example, based on the completeness, tilt, and distortion of each marker pattern, the system determines whether each marker pattern is qualified, and assesses the qualification of body cavity establishment based on the judgment results for each marker pattern. The system compares the actual operational sequence (such as observation sequence, cutting sequence, or suturing sequence) obtained from the identification with a pre-set standard operational sequence, and determines whether the corresponding endoscopic procedure is qualified based on the comparison results. In other implementations, scoring or rating can also be performed based on the relevant feature information of each marker pattern to obtain operational evaluation results.

[0061] The standardization and precision of endoscopic procedures directly affect treatment outcomes and patient safety; therefore, skills training and assessment for operators are crucial. Currently, skills assessment for endoscopic procedures in existing simulation training environments still relies primarily on subjective, experience-based judgment, which fails to provide accurate evaluation of operators' endoscopic skills.

[0062] Compared to existing technologies, this application acquires a sequence of operation images during endoscopic procedures performed by an operator on a simulated body cavity. The simulated body cavity contains multiple pre-set marker patterns. The operation image sequence is acquired for each marker pattern and includes operation images corresponding to different endoscopic operation stages. Based on pre-set evaluation indicators for each endoscopic operation stage, the operation images corresponding to that stage are analyzed to obtain relevant feature information of each marker pattern. Based on this feature information, an operation evaluation result corresponding to each endoscopic operation stage is generated. By introducing pre-set marker patterns as objective references, continuous monitoring and analysis of the marker pattern morphology during endoscopic procedures allows for the acquisition of relevant feature information for each marker pattern at different operation stages. This relevant feature information objectively reflects the endoscopic operation skill level, enabling accurate evaluation of each stage of the endoscopic operation based on the acquired feature information. This solves the problem of inaccurate evaluation of operator endoscopic procedures and achieves accurate assessment of operator skill levels.

[0063] In some embodiments, the endoscopic procedure phase is a combination of one or more of the following: a body cavity creation phase, an intracavitary observation phase, an intracavitary manipulation phase, and a suturing phase. Specifically:

[0064] 1. Stage of body cavity establishment

[0065] The operator establishes a stable surgical operating space by injecting gas into the simulated body cavity. The core assessment objective at this stage is the suitability of the cavity establishment, ensuring that the space meets the basic requirements for subsequent operations; if it fails to meet the requirements, it must be re-established.

[0066] 2. Intracavitary observation stage

[0067] Operators use endoscopes to systematically observe multiple pre-set marker patterns within a simulated body cavity. The core assessment objective of this stage is the standardization of the observation sequence, i.e., whether the pre-set observation order is strictly followed, aiming to train operators' proficiency in using endoscopes.

[0068] 3. Intracavitary operation stage

[0069] Under endoscopic guidance, operators use instruments such as electrocautery to simulate cutting or tissue manipulation of various marked patterns within a simulated body cavity. The core evaluation objective of this stage is the standardization and stability of the operational sequence. Here, the operational sequence includes the specific processing order within each marked pattern, as well as the operational order between different marked patterns.

[0070] 4. Suturing stage

[0071] Under endoscopic guidance, the operator performs simulated suturing of the cut marking patterns within the simulated body cavity. The core evaluation objective of this stage is the standardization of the suturing sequence and the level of suturing skill. The operational sequence includes the specific suturing order within each marking pattern and the suturing order between each marking pattern. Furthermore, the preset suturing order may be the same as or different from the aforementioned cutting sequence.

[0072] In this way, the complex and continuous endoscopic operation process is deconstructed into multiple standardized and independently assessable operation stages. This not only helps to achieve targeted and accurate assessment of different operation stages, but also ensures systematic coverage of the entire operation process through stage combination, laying a structural foundation for establishing a standardized, full-process quantitative assessment system for endoscopic skills.

[0073] In some of these embodiments, the relevant feature information includes the integrity, tilt, and degree of distortion of each identification pattern;

[0074] Step S210, which generates the operation evaluation results corresponding to the endoscopic operation stage based on the relevant feature information of each identification pattern, includes the following steps:

[0075] During the endoscopic procedure, specifically the cavity creation stage, each marker pattern is analyzed to determine whether it matches a preset standard marker pattern based on its relevant characteristic information, thus obtaining the analysis results for each marker pattern.

[0076] Based on the analysis results of each identification pattern, the operation evaluation results corresponding to the body cavity establishment stage are generated.

[0077] Specifically, during the cavity creation stage, the relevant feature information obtained from the operational images includes the integrity, tilt status, and distortion degree of each marker pattern. The output of the relevant feature information can be in the form of continuous specific numerical values ​​(such as integrity percentage, tilt angle value, distortion coefficient) or discrete state judgment results (such as complete / incomplete, tilted / not tilted, distorted / not distorted).

[0078] Furthermore, based on the acquired feature information of each sign pattern, the system analyzes whether each sign pattern matches a preset standard sign pattern, i.e., determines whether each sign pattern is qualified. The standard sign pattern is a complete sign pattern that is not tilted or distorted. The specific implementation method for qualification judgment includes: when the relevant feature information is a continuous value, comparing the three feature values—completeness percentage, tilt angle value, and distortion coefficient—with preset threshold ranges. The preset threshold ranges are determined based on the preset standard sign pattern. If all feature values ​​are within the corresponding normal threshold range, the sign pattern is determined to match the standard sign pattern, i.e., the sign pattern is qualified. When the relevant feature information is a discrete state judgment result, if any state judgment result is detected to be abnormal (such as incomplete, tilted, or distorted), the sign pattern is determined to not match the standard sign pattern, i.e., the sign pattern is unqualified. (Refer to...) Figure 2 As shown, taking the five-pointed star-shaped logo as an example, (a) is a qualified logo, and (b) to (f) are unqualified logos due to tilting, distortion, etc.

[0079] Subsequently, based on the analysis results of each marker pattern, an operational evaluation result corresponding to the body cavity establishment stage is generated. The operational evaluation result is used to indicate whether the body cavity establishment is qualified. In specific implementation, the number of qualified marker patterns can be counted. If the number of qualified marker patterns is greater than a preset threshold, the body cavity establishment is deemed qualified.

[0080] It should be noted that the matching result for each sign pattern can also be a specific score. That is, based on the relevant feature information of each sign pattern, the matching degree between each sign pattern and the preset standard sign pattern is analyzed, thereby accurately quantifying the evaluation of the sign pattern. The specific implementation of the matching analysis includes: using a pre-trained pattern matching analysis model (such as a deep learning model based on convolutional neural networks) to predict and output the matching degree corresponding to each sign pattern. This pattern matching analysis model is trained based on a large number of labeled sign pattern images and their corresponding matching degree scores; calculating the deviation values ​​of the integrity percentage, tilt angle value, and distortion coefficient of each sign pattern from the preset threshold range. The preset threshold range is determined based on the preset standard sign pattern. Then, the deviation values ​​of each feature are normalized and weighted summed to finally obtain the matching degree corresponding to the sign pattern.

[0081] Accordingly, the matching degree of each marker pattern is used as the operation evaluation result for the body cavity establishment stage. For example, the matching degree of each marker pattern is out of 10 points. The matching degrees of marker patterns numbered 1 to 5 are 8, 9, 10, 8 and 9 points respectively. The total matching degree of each marker pattern is calculated as 44 points, and the total matching degree score is used as the operation evaluation result for the body cavity establishment stage.

[0082] This embodiment enables accurate operational evaluation during the body cavity creation stage and allows for precise location of operational defects (such as determining uneven spatial creation in the corresponding area based on the location of the abnormal marker pattern), thereby providing operators with clear basis for improvement.

[0083] In some embodiments, the relevant feature information includes the actual viewing order of each identification pattern;

[0084] Based on the relevant feature information of each logo pattern, the operation evaluation results corresponding to the endoscopic operation stage are generated, including the following steps:

[0085] During the intracavitary observation stage of the endoscopic procedure, determine whether the actual observation order of each marker pattern is the same as the preset observation order.

[0086] Based on the observation sequence, the corresponding operational evaluation results for the intracavitary observation stage are generated.

[0087] Specifically, for the intracavitary observation phase, the relevant feature information obtained from the operational images includes the actual observation order of each marker pattern. It can be understood that each marker pattern is clearly marked by a number (1~n) or a letter (such as a, b, ...). By using a trained character recognition model to perform character recognition on the operational images during the intracavitary observation phase, the actual observation order of each marker pattern within the simulated body cavity can be identified and obtained. (Refer to...) Figure 3 As shown, taking numerical designations as an example, operators are required to use an endoscope to observe the marking patterns numbered from 1 to n in sequence.

[0088] Furthermore, the actual observation order of each logo pattern is compared with the preset observation order to determine whether the actual observation order is the same as the preset observation order. If the actual observation order is the same as the preset observation order, the operation evaluation result is generated as qualified; otherwise, the operation evaluation result is generated as unqualified.

[0089] This embodiment enables accurate operational assessment of the intraluminal observation stage and precisely pinpoints the specific deviations from the observation sequence, effectively improving the training efficiency and quality of endoscopic observation skills.

[0090] In some embodiments, generating operational evaluation results corresponding to the endoscopic operation stage based on the relevant feature information of each identification pattern further includes the following steps:

[0091] Acquire the actual endoscopic trajectory and the actual endoscopic posture corresponding to each marker pattern during the intracavitary observation phase;

[0092] Determine whether the actual endoscope trajectory corresponding to each marker pattern matches the preset endoscope trajectory, and whether the actual endoscope posture corresponding to each marker pattern is within the preset posture range;

[0093] Based on the results of observation sequence judgment, endoscope trajectory judgment, and endoscope posture judgment, the operation evaluation results corresponding to the intracavitary observation stage are generated.

[0094] It should be noted that the operational evaluation during the intraluminal observation phase can be further combined with endoscopic trajectory data and endoscopic posture data to improve the comprehensiveness of this stage of evaluation. Specifically, through measurement units such as gyroscopes integrated on the endoscope handle, the actual endoscopic trajectory corresponding to each marker pattern and the actual endoscopic posture corresponding to each marker pattern are simultaneously acquired during the intraluminal observation phase. While analyzing whether the actual observation order of each marker pattern is the same as the preset observation order, it is determined whether the actual endoscopic trajectory corresponding to each marker pattern matches the preset endoscopic trajectory, and whether the actual endoscopic posture corresponding to each marker pattern is within the preset posture range.

[0095] Furthermore, if the actual endoscopic trajectory corresponding to the marked pattern matches the preset endoscopic trajectory, the actual endoscopic trajectory corresponding to that marked pattern is determined to be normal; otherwise, it is determined to be abnormal. In specific implementation, the endoscopic trajectory evaluation model can also be trained in advance using trajectory data from normal and abnormal endoscopic operations. The trained endoscopic trajectory evaluation model can then be used to analyze whether the actual endoscopic trajectory corresponding to each marked pattern is normal. Moreover, if the actual endoscopic posture corresponding to the marked pattern is within the preset posture range, the actual endoscopic posture corresponding to that marked pattern is determined to be normal; otherwise, it is determined to be abnormal.

[0096] When the actual observation order of each marker pattern is the same as the preset observation order, and the actual endoscopic trajectory and actual endoscopic posture corresponding to each marker pattern are normal, the operation evaluation result is generated as qualified observation operation. It can be understood that during the intracavitary observation stage, the observation operation of a single marker pattern can also be evaluated, that is, the observation sequence of a single marker pattern is evaluated to see if it is correct, and the evaluation result of a single marker pattern is output. The principle is the same as the above process.

[0097] In this embodiment, it is determined whether the actual endoscope trajectory corresponding to each marker pattern matches the preset endoscope trajectory, and whether the actual endoscope posture corresponding to each marker pattern is within the preset posture range. Based on the observation sequence judgment result, endoscope trajectory judgment result, and endoscope posture judgment result, the operation evaluation result corresponding to the intracavitary observation stage is generated, thereby improving the comprehensiveness of the operation evaluation and more realistically and accurately reflecting the skill level of the operator.

[0098] In some embodiments, the relevant feature information includes the actual cutting order and cutting quality of each identification pattern;

[0099] Based on the relevant feature information of each logo pattern, the operation evaluation results corresponding to the endoscopic operation stage are generated, including the following steps:

[0100] During the intracavitary operation stage of the endoscopic procedure, determine whether the actual cutting order of each marking pattern is the same as the preset cutting order.

[0101] Based on the judgment results of each marking pattern and the cutting quality, the operation evaluation results corresponding to the intracavitary operation stage are generated.

[0102] Specifically, for the intracavitary manipulation stage, the relevant feature information obtained from the manipulation images includes the actual cutting order and cutting quality of each marker pattern. It should be noted that the areas to be manipulated within each marker pattern are clearly marked with numbers or letters. By using a trained character recognition model to perform character recognition on the manipulation images during the intracavitary manipulation stage, the actual cutting order of each marker pattern within the simulated body cavity can be identified and obtained. (Refer to...) Figure 4 As shown, taking numerical numbering as an example, the middle number of each marker pattern indicates the cutting order between the marker patterns, and the number at each corner of each marker pattern indicates the cutting order within the marker pattern. The standard operating procedure for a single marker pattern is to cut the areas numbered 1 to 5 within the marker pattern sequentially, while the standard operating procedure for multiple marker patterns is to process the marker patterns with middle numbers 1 to 3 sequentially. Subsequently, the actual cutting order of each marker pattern is compared with the preset cutting order to determine whether the actual cutting order is the same as the preset cutting order.

[0103] Meanwhile, the cutting quality can be obtained by analyzing visual features such as the cutting flatness of the marked patterns in the operation image through a cutting evaluation model. This cutting evaluation model is pre-trained using a dataset of marked pattern images containing qualified and unqualified cutting effects, and outputs a qualified judgment on the cutting quality.

[0104] Ultimately, if the actual cutting order of each marking pattern is found to be the same as the preset cutting order, and the cutting quality of each marking pattern is qualified, then the operation evaluation result is "cutting operation qualified"; otherwise, the operation evaluation result is "cutting operation unqualified". It is understood that during the intracavity operation stage, the cutting operation of a single marking pattern can also be evaluated, and the evaluation result for that single marking pattern can be output, with the same principle as the above process.

[0105] In this embodiment, during the intracavitary operation phase of the endoscopy procedure, it is determined whether the actual cutting sequence of each marked pattern is the same as the preset cutting sequence. Based on the determination results of each marked pattern and the cutting quality, an operation evaluation result corresponding to the intracavitary operation phase is generated, thereby achieving accurate operation evaluation of the intracavitary operation phase. This method can not only identify errors in the operation process but also accurately locate areas where unstable operation leads to uneven cutting edges, which helps to improve the pertinence and efficiency of skills training.

[0106] In some embodiments, the relevant feature information includes the actual stitching sequence and stitching quality of each identification pattern;

[0107] Based on the relevant feature information of each logo pattern, the operation evaluation results corresponding to the endoscopic operation stage are generated, including the following steps:

[0108] During the suture stage of the endoscopic procedure, determine whether the actual suture sequence of each marker pattern is the same as the preset suture sequence.

[0109] Based on the judgment results of each marking pattern and the stitching quality, the corresponding operation evaluation results for the stitching stage are generated.

[0110] Specifically, for the suturing stage, the relevant feature information obtained from the operational images includes the actual suturing sequence of each marker pattern. It should be noted that the areas to be operated on within each marker pattern are clearly marked with numbers or letters. By using a trained character recognition model to perform character recognition on the operational images of the suturing stage, the actual suturing sequence of each marker pattern within the simulated body cavity can be identified and obtained. (Refer to...) Figure 5 As shown, taking numerical numbering as an example, the middle number of each marker pattern indicates the stitching order between the marker patterns, and the number at each corner of each marker pattern indicates the stitching order within the marker pattern. The standard operating procedure for a single marker pattern is to stitch the areas numbered 1 to 5 within the marker pattern sequentially, while the standard operating procedure for multiple marker patterns is to process the marker patterns numbered 1 to 3 in the middle sequentially. Subsequently, the actual stitching order of each marker pattern is compared with the preset stitching order to determine whether the actual stitching order is the same as the preset stitching order.

[0111] The quality of the suture can be obtained by analyzing visual features such as the flatness of the suture in the marked pattern in the operation image through a suture evaluation model. This suture evaluation model is pre-trained using a dataset of marked pattern images containing acceptable and unacceptable cutting effects, and outputs a judgment on the acceptance or rejection of the suture quality.

[0112] If the actual sewing sequence of each marking pattern is found to be the same as the preset sewing sequence, and the sewing quality of each marking pattern is qualified, then the operation evaluation result is "cutting operation qualified"; otherwise, the operation evaluation result is "cutting operation unqualified". It is understood that during the sewing stage, the sewing operation of a single marking pattern can also be evaluated, and the evaluation result of a single marking pattern can be output. The principle is the same as the above process.

[0113] In this embodiment, when the endoscopic operation is in the suturing stage, it is determined whether the actual suturing sequence of each marker pattern is the same as the preset suturing sequence. Based on the judgment results of each marker pattern and the suturing quality, the operation evaluation result corresponding to the suturing stage is generated. This enables accurate operation evaluation of the intracavitary operation stage and can identify errors in the suturing process, which helps to improve the pertinence and efficiency of skills training.

[0114] In some of these embodiments, the endoscopic procedure consists of multiple stages;

[0115] After generating the operational evaluation results corresponding to the endoscopic operation stage based on the relevant feature information of each logo pattern, the following steps are also included:

[0116] The operation evaluation results corresponding to each endoscopic operation stage are weighted and fused to obtain the operator's endoscopic operation evaluation result.

[0117] Specifically, the operational evaluation results for each endoscopic procedure stage are uniformly converted into binary values, with 1 representing a satisfactory operation at each stage, including satisfactory cavity establishment, satisfactory intracavitary observation, satisfactory intracavitary manipulation, and satisfactory suturing, and 0 representing an unsatisfactory operation. Subsequently, the values ​​corresponding to each endoscopic procedure stage are weighted and fused to finally calculate the operator's endoscopic operation evaluation result.

[0118] It should be noted that the granularity of the operational evaluation at each stage can be refined to the analysis results of specific feature information. The analysis results of each relevant feature information are represented by binary values. Here, 1 represents that the evaluation of marker pattern distortion, tilt, and integrity is satisfactory; the actual observation order of the marker pattern is correct; the actual endoscopic trajectory is normal; the actual endoscopic posture is normal; the actual cutting order is correct; the cutting quality is satisfactory; the actual suturing order is correct; and the suturing quality is satisfactory, etc.; conversely, 0 represents the opposite. Based on this, the values ​​corresponding to each relevant feature information are weighted and summed to obtain the operator's endoscopic operation evaluation result.

[0119] This embodiment enables a multi-dimensional and comprehensive operational assessment of procedures such as establishing body cavities, intracavitary observation, intracavitary manipulation, and suturing, thereby objectively and comprehensively reflecting the overall skill level of the operators.

[0120] The following describes and illustrates this embodiment through specific examples.

[0121] Reference Figure 6 As shown, taking laparoscopic surgery as an example, the simulated abdominal cavity is equipped with multiple marker patterns numbered from 1 to n. The endoscopic surgery stages include the abdominal cavity creation stage, the intracavitary observation stage, the intracavitary manipulation stage, and the suturing stage.

[0122] 1. Abdominal cavity establishment stage

[0123] The operator establishes a stable surgical operating space by injecting CO2 gas into the simulated abdominal cavity. After the abdominal cavity is established, multiple operation images are acquired for each marker pattern within the simulated abdominal cavity. Each operation image is input into three trained abdominal cavity evaluation models. Each abdominal cavity evaluation model uses a deep learning network model to analyze the integrity, tilt, and distortion of the marker patterns in the operation images, and outputs whether the marker patterns are complete, tilted, or distorted. The specific expressions are as follows:

[0124] (1)

[0125] In equation (1), Represents each abdominal cavity assessment model, i=1,2,3; This indicates the evaluation result for each model; The input operation image is represented by j = 1, 2, 3, ..., n.

[0126] Based on the completeness, tilt, and distortion of each marker pattern, the suitability of each marker pattern is determined. The overall assessment of all marker patterns then determines the suitability of the abdominal cavity reconstruction. If the abdominal cavity reconstruction is satisfactory, intracavitary observation is performed; otherwise, the abdominal cavity must be reconstructed.

[0127] 2. Intracavitary observation stage

[0128] The operator uses an endoscope to sequentially observe multiple marked patterns (numbered from 1 to n) within a simulated abdominal cavity. During the observation of the marked patterns, corresponding operational images are captured. A character recognition model (implemented using a deep learning detection model) identifies the actual observation order of the marked patterns and determines whether the actual observation order is correct. The specific expression is as follows:

[0129] (2)

[0130] In equation (2), X is the input operation image; The pattern number should be identified in sequence; k indicates whether the current image is observed in sequence; K is the character recognition model.

[0131] Furthermore, through measurement units such as gyroscopes integrated into the endoscope handle, the actual endoscopic trajectory and actual endoscopic posture of the observation marker pattern are simultaneously acquired during the intracavitary observation phase. The endoscopic trajectory evaluation model is pre-trained using trajectory data from normal and abnormal endoscopic operations. The trained endoscopic trajectory evaluation model analyzes whether the actual endoscopic trajectory corresponding to the marker pattern is normal. The specific expression is as follows:

[0132] (3)

[0133] In equation (3), This indicates the actual endoscopic trajectory of the observed marking pattern; This represents an endoscope trajectory evaluation model, which can be implemented using a deep learning model; This indicates whether the actual endoscopic trajectory is normal.

[0134] In addition, determine whether the actual endoscope posture of the observed marking pattern is within the preset posture range. If the actual endoscope posture of the observed marking pattern is within the preset posture range, the actual endoscope posture is determined to be normal; otherwise, the actual endoscope posture is determined to be abnormal.

[0135] 3. Intracavitary operation stage

[0136] Under endoscopic guidance, operators use instruments such as electrocautery to simulate cutting various marked patterns within a simulated abdominal cavity. During the cutting process, corresponding operation images are acquired. A trained character recognition model is then used to identify the actual cutting sequence of the marked patterns and determine if the sequence is correct. The specific principle is the same as the character recognition model used in the intracavitary observation phase.

[0137] Meanwhile, the cutting evaluation model is pre-trained using a dataset of marked pattern images containing both acceptable and unacceptable cutting results. The trained model analyzes visual features such as the cut flatness of the marked patterns in the operation images and outputs a judgment on the acceptable cutting quality to reflect operational stability. The specific expression is as follows:

[0138] (4)

[0139] In equation (4), u represents whether the cutting quality is qualified; This represents the input operation image, which includes the cut marker pattern. This represents the segmentation evaluation model, which can be implemented using a deep learning model.

[0140] 4. Suturing stage

[0141] Under endoscopic guidance, the operator sutures the cut markings within the simulated abdominal cavity. During the suturing process, corresponding images are captured. A trained character recognition model is then used to identify the actual suturing sequence of the markings and determine if it is correct. The specific principle is the same as the character recognition model used in the intracavitary observation phase.

[0142] Meanwhile, the suture evaluation model is pre-trained using a dataset of marked pattern images containing both acceptable and unacceptable suture results. The trained suture evaluation model analyzes visual features such as the suture flatness of the marked patterns in the operation images and outputs a pass / fail judgment on the suture quality to reflect the actual suture result. The specific expression is as follows:

[0143] (5)

[0144] In equation (5), t represents whether the stitching quality is acceptable; This represents the input operation image, which includes the stitched marker pattern. This represents a suture evaluation model, which can be implemented using a deep learning model.

[0145] 5. Endoscopic procedure evaluation stage

[0146] The procedures performed in each of the above endoscopic stages are summarized and evaluated, and the specific expression is as follows:

[0147] (6)

[0148] In equation (6), , and These indicate whether the deformity, tilt, and integrity of the abdominal cavity are assessed as satisfactory during the abdominal cavity construction phase. This indicates whether the actual observation order of the marked patterns is correct during the intracavitary observation phase; This indicates whether the actual endoscopic trajectory is normal during the intracavitary observation phase; This indicates whether the actual endoscopic posture is normal during the intracavitary observation phase; This indicates whether the actual cutting sequence during the intracavitary operation phase is correct; This indicates whether the cutting quality is acceptable during the intracavitary operation phase; This indicates whether the actual suturing sequence during the suturing stage is correct; This indicates whether the suturing quality is acceptable during the suturing stage. Furthermore, the specific values ​​of the weighting coefficients can be adjusted according to different training objectives and skill importance.

[0149] This embodiment also provides an endoscopic operation evaluation system, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below refer to combinations of software and / or hardware that perform a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0150] Figure 7 This is a structural block diagram of the endoscopic operation evaluation system of this embodiment, as shown below. Figure 7 As shown, the system includes:

[0151] The acquisition module 10 is used to acquire the sequence of operation images during the endoscopic operation performed by the operator on the simulated body cavity; wherein, multiple marker patterns are preset in the simulated body cavity; the operation image sequence is acquired for each marker pattern, and the operation image sequence includes operation images corresponding to the endoscopic operation stage;

[0152] Analysis module 20 is used to analyze the operation images corresponding to the endoscopy operation stage based on the preset evaluation indicators of the endoscopy operation stage, so as to obtain the relevant feature information of each symbol pattern in the operation image.

[0153] The evaluation module 30 is used to generate operation evaluation results corresponding to the endoscopic operation stage based on the relevant feature information of each logo pattern.

[0154] The system provided in this embodiment acquires a sequence of operation images during an operator's endoscopic procedure on a simulated body cavity. The simulated body cavity contains multiple pre-set marker patterns. The operation image sequence is acquired for each marker pattern and includes operation images corresponding to each endoscopic operation stage. Based on pre-set evaluation indicators for each endoscopic operation stage, the operation images corresponding to that stage are analyzed to obtain relevant feature information of each marker pattern. Based on the relevant feature information of each marker pattern, an operation evaluation result corresponding to each endoscopic operation stage is generated. This solves the problem of inaccurate evaluation of the operator's endoscopic procedures and enables accurate assessment of the operator's skill level.

[0155] In some embodiments, the evaluation module 30 is further configured to, during the endoscopic operation stage, specifically the body cavity establishment stage, analyze whether each marker pattern matches a preset standard marker pattern based on the relevant feature information of each marker pattern, and obtain the analysis result of each marker pattern; based on the analysis results of each marker pattern, generate the operation evaluation result corresponding to the body cavity establishment stage.

[0156] In some embodiments, the evaluation module 30 is further configured to determine whether the actual observation order of each marker pattern is the same as the preset observation order when the endoscopic operation stage is the intracavitary observation stage; and generate the operation evaluation result corresponding to the intracavitary observation stage based on the observation order judgment result.

[0157] In some embodiments, the evaluation module 30 is further configured to acquire the actual endoscopic trajectory during the intracavitary observation phase and the actual endoscopic posture corresponding to each marker pattern; determine whether the actual endoscopic trajectory matches the preset endoscopic trajectory and whether the actual endoscopic posture corresponding to each marker pattern is within the preset posture range; and generate the operation evaluation result corresponding to the intracavitary observation phase based on the observation sequence judgment result, the endoscopic trajectory judgment result, and the endoscopic posture judgment result.

[0158] In some embodiments, the evaluation module 30 is further configured to determine whether the actual cutting order of each marking pattern is the same as the preset cutting order when the endoscopic operation stage is the intracavitary operation stage; and to generate the operation evaluation result corresponding to the intracavitary operation stage based on the judgment result of each marking pattern and the cutting quality.

[0159] In some embodiments, the evaluation module 30 is further configured to determine whether the actual suturing sequence of each marker pattern is the same as the preset suturing sequence when the endoscopic operation stage is the suturing stage; and to generate an operation evaluation result corresponding to the suturing stage based on the judgment result of each marker pattern and the suturing quality.

[0160] In some embodiments, the evaluation module 30 is also used to weight and fuse the operation evaluation results corresponding to each endoscopic operation stage to obtain the operator's endoscopic operation evaluation result.

[0161] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0162] This embodiment also provides a computer device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0163] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0164] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0165] S1, acquire the sequence of operation images during the endoscopic operation performed by the operator on the simulated body cavity; wherein, multiple marker patterns are preset in the simulated body cavity; the operation image sequence is acquired for each marker pattern, and the operation image sequence includes operation images corresponding to the endoscopic operation stage;

[0166] S2, based on the preset evaluation indicators of the endoscopic operation stage, analyze the operation images corresponding to the endoscopic operation stage to obtain the relevant feature information of each symbol pattern in the operation image;

[0167] S3. Based on the relevant feature information of each logo pattern, generate the operation evaluation results corresponding to the endoscopic operation stage.

[0168] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0169] Furthermore, in conjunction with the endoscopic operation evaluation methods provided in the above embodiments, this embodiment can also provide a storage medium for implementation. The storage medium stores a computer program; when executed by a processor, the computer program implements any of the endoscopic operation evaluation methods in the above embodiments.

[0170] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0171] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0172] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0173] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A method for evaluating endoscopic procedures, characterized in that, include: A sequence of operation images is acquired during the process of an operator performing an endoscopy on a simulated body cavity; wherein, the simulated body cavity is pre-set with multiple marker patterns; the operation image sequence is acquired for each of the marker patterns, and the operation image sequence includes operation images corresponding to the endoscopic operation stage; Based on the preset evaluation indicators of the endoscopic operation stage, the operation images corresponding to the endoscopic operation stage are analyzed to obtain the relevant feature information of each of the marked patterns in the operation images; Based on the relevant feature information of each of the aforementioned identification patterns, an operational evaluation result corresponding to the endoscopic operation stage is generated.

2. The endoscopic operation evaluation method according to claim 1, characterized in that, The endoscopic operation stage is a combination of one or more of the following: the body cavity creation stage, the intracavitary observation stage, the intracavitary operation stage, and the suturing stage.

3. The endoscopic operation evaluation method according to claim 2, characterized in that, The relevant feature information includes the completeness, tilt state, and degree of distortion of each of the identification patterns; The process of generating an operational evaluation result corresponding to the endoscopic operation stage based on the relevant feature information of each of the aforementioned identifier patterns includes: When the endoscopic operation stage is the body cavity creation stage, based on the relevant feature information of each of the marked patterns, it is analyzed whether each of the marked patterns matches a preset standard marked pattern, and the analysis result of each marked pattern is obtained; Based on the analysis results of each of the aforementioned identifier patterns, the operational evaluation results corresponding to the body cavity establishment stage are generated.

4. The endoscopic operation evaluation method according to claim 2, characterized in that, The relevant feature information includes the actual observation order of each of the aforementioned identification patterns; The process of generating an operational evaluation result corresponding to the endoscopic operation stage based on the relevant feature information of each of the aforementioned identifier patterns includes: When the endoscopic operation stage is the intracavitary observation stage, it is determined whether the actual observation order of each of the marked patterns is the same as the preset observation order; Based on the observation sequence judgment results, the operation evaluation results corresponding to the intracavitary observation stage are generated.

5. The endoscopic operation evaluation method according to claim 4, characterized in that, The step of generating the operation evaluation result corresponding to the endoscopic operation stage based on the relevant feature information of each of the aforementioned identifier patterns further includes: Obtain the actual endoscopic trajectory and the actual endoscopic posture corresponding to each of the marked patterns during the intracavitary observation phase; Determine whether the actual endoscope trajectory corresponding to each of the marked patterns matches the preset endoscope trajectory, and whether the actual endoscope posture corresponding to each of the marked patterns is within the preset posture range; Based on the observation sequence judgment result, endoscope trajectory judgment result, and endoscope posture judgment result, the operation evaluation result corresponding to the intracavitary observation stage is generated.

6. The endoscopic operation evaluation method according to claim 2, characterized in that, The relevant feature information includes the actual cutting sequence and cutting quality of each of the marked patterns; The process of generating an operational evaluation result corresponding to the endoscopic operation stage based on the relevant feature information of each of the aforementioned identifier patterns includes: When the endoscopic operation stage is the intracavitary operation stage, it is determined whether the actual cutting order of each of the marked patterns is the same as the preset cutting order; Based on the judgment results of each of the aforementioned marking patterns and the cutting quality, the operation evaluation results corresponding to the intracavitary operation stage are generated.

7. The endoscopic operation evaluation method according to claim 1, characterized in that, The relevant feature information includes the actual stitching sequence and stitching quality of each of the identified patterns; The process of generating an operational evaluation result corresponding to the endoscopic operation stage based on the relevant feature information of each of the aforementioned identifier patterns includes: When the endoscopic operation stage is the suturing stage, it is determined whether the actual suturing sequence of each of the marked patterns is the same as the preset suturing sequence. Based on the judgment results of each of the aforementioned marking patterns and the suturing quality, the operation evaluation result corresponding to the suturing stage is generated.

8. The endoscopic operation evaluation method according to claim 1, characterized in that, The endoscopic procedure consists of multiple stages; After generating the operation evaluation result corresponding to the endoscopic operation stage based on the relevant feature information of each of the aforementioned identifier patterns, the method further includes: The operation evaluation results corresponding to each of the endoscopic operation stages are weighted and fused to obtain the endoscopic operation evaluation result of the operator.

9. An endoscopic operation evaluation system, characterized in that, include: The acquisition module is used to acquire a sequence of operation images during the endoscopic operation performed by the operator on the simulated body cavity; wherein, the simulated body cavity is pre-set with multiple marker patterns; the operation image sequence is acquired for each of the marker patterns, and the operation image sequence includes operation images corresponding to the endoscopic operation stage; The analysis module is used to analyze the operation image corresponding to the endoscopy operation stage based on the preset evaluation indicators of the endoscopy operation stage, so as to obtain the relevant feature information of each of the marked patterns in the operation image; The evaluation module is used to generate an operation evaluation result corresponding to the endoscopic operation stage based on the relevant feature information of each of the aforementioned identification patterns.

10. A computer device, comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the steps of the endoscopic operation evaluation method according to any one of claims 1 to 7.

Images