An endoscopic procedure assessment system

Abstract

The application relates to an endoscope operation evaluation system. The system comprises a data acquisition module, a data analysis module and an operation evaluation module; the data acquisition module is used for synchronously collecting target data during the execution of a target operation task by an operator, so as to obtain target data of the entire operation process; the target data comprises image data and motion sensing data of an instrument; the data analysis module is used for constructing a time sequence data feature sequence of the entire operation process based on the received target data of the entire operation process, and analyzing the constructed time sequence data feature sequence to determine target data corresponding to each operation stage in the entire operation process; and the operation evaluation module is used for determining an endoscope operation evaluation result of the entire operation process based on the target data corresponding to each operation stage. The system solves the problem that the existing endoscope operation evaluation system cannot adaptively evaluate different operation stages and the evaluation result is inaccurate.

Description

Technical Field

[0001] This application relates to the field of endoscopic education and training technology, and in particular to an endoscopic operation assessment system. Background Technology

[0002] With the development of endoscopic technology, endoscopes have been widely used in the medical field. Performing endoscopic examinations of target areas (such as cystoscopy) is an essential clinical skill for endoscopic practitioners. To master this clinical skill, training in endoscopic examination procedures is necessary for endoscopic practitioners.

[0003] Current technology primarily uses endoscopic operation assessment systems to evaluate the endoscopic procedures performed by endoscopic practitioners, thus providing training in endoscopic operation. These existing systems indirectly assess the quality of the operator's work by collecting organ feedback data. Furthermore, these systems use a uniform assessment standard throughout the entire process, failing to provide adaptive assessments for different stages of the procedure, and resulting in inaccurate assessment results.

[0004] Existing endoscopic procedure assessment systems are unable to provide adaptive assessments for different stages of the procedure, resulting in inaccurate assessment results. Currently, no effective solution has been proposed. Summary of the Invention

[0005] Therefore, it is necessary to provide an endoscopic operation evaluation system to address the aforementioned technical problems.

[0006] In a first aspect, this application provides an endoscopic operation evaluation system. The system includes: a data acquisition module, a data analysis module, and an operation evaluation module;

[0007] The data acquisition module, connected to the data analysis module, is used to synchronously acquire target data during the operator's execution of the target operation task, obtain target data for the entire operation process, and send the obtained target data for the entire operation process to the data analysis module; the target data includes image data and motion sensing data of the device;

[0008] The data analysis module, connected to the operation evaluation module, is used to construct a time-series data feature sequence of the entire operation process based on the target data of the entire operation process received, analyze the constructed time-series data feature sequence, determine the target data corresponding to each operation stage in the entire operation process, and send the determined target data corresponding to each operation stage to the operation evaluation module.

[0009] The operation evaluation module is used to determine the endoscopic operation evaluation result of the entire operation process based on the target data corresponding to each of the operation stages.

[0010] In one embodiment, the data acquisition module includes: an image acquisition submodule and a sensor data acquisition submodule;

[0011] The image acquisition submodule is connected to the data analysis module and is used to acquire image frame sequences of the operator performing the target operation task using the target endoscope, obtain image data of the entire operation process, and send the obtained image data of the entire operation process to the data analysis module; the target endoscope is an endoscope matched to the target operation task;

[0012] The sensor data acquisition submodule is connected to the data analysis module and is used to acquire motion sensing data of the instrument when the operator performs the target operation task using sensors installed in the instrument, and send the acquired motion sensing data of the instrument when the operator performs the target operation task to the data analysis module.

[0013] In one embodiment, the data analysis module includes: a sequence construction submodule and a data analysis submodule;

[0014] The sequence construction submodule connects the data analysis submodule and the data acquisition module. It is used to construct the time-series data feature sequence of the entire operation process based on the target data of the entire operation process sent by the data acquisition module, and send the constructed time-series data feature sequence to the data analysis submodule.

[0015] The data analysis submodule, connected to the operation evaluation module, is used to analyze the received time-series data feature sequence, determine the target data corresponding to each operation stage in the entire operation process, and send the determined target data corresponding to each operation stage to the operation evaluation module.

[0016] In one embodiment, the data analysis submodule includes: a data analysis unit and a data determination unit;

[0017] The data analysis unit is connected to the sequence construction submodule and the data determination unit, respectively. It is used to analyze the time series data feature sequence sent by the sequence construction submodule, determine the operation stage corresponding to different sequence intervals in the time series data feature sequence, and send the determined operation stage corresponding to the different sequence intervals to the data determination unit.

[0018] The data determination unit, connected to the operation evaluation module, is used to determine the target data corresponding to each operation stage in the entire operation process based on the operation stages corresponding to the different sequence intervals received, and to send the target data corresponding to each operation stage to the operation evaluation module.

[0019] In one embodiment, the data analysis unit includes: a model construction subunit and a data analysis subunit;

[0020] The model construction subunit is connected to the data analysis subunit and is used to train the initial time series classification model to obtain the trained time series classification model, and send the trained time series classification model to the data analysis subunit.

[0021] The data analysis subunit is connected to the data determination unit and the sequence construction submodule, respectively. It is used to receive the time-series data feature sequence sent by the sequence construction submodule, and to analyze the received time-series data feature sequence using the received trained time-series classification model, determine the operation stage corresponding to different sequence intervals in the time-series data feature sequence, and send the determined operation stage corresponding to the different sequence intervals to the data determination unit.

[0022] In one embodiment, the operation evaluation module includes: a first evaluation submodule and a second evaluation submodule;

[0023] The first evaluation submodule, connected to the data analysis module and the second evaluation submodule, is used to receive the target data corresponding to each operation stage sent by the data analysis module, and based on the received target data corresponding to each operation stage and the preset evaluation rules for each operation stage, determine the operation evaluation result of each operation stage, and send the operation evaluation result of each operation stage to the second evaluation submodule.

[0024] The second evaluation submodule is used to perform weighted calculation on the received operation evaluation results of each operation stage based on the weight of the operation evaluation results of each operation stage, so as to obtain the endoscopy operation evaluation result of the entire operation process.

[0025] In one embodiment, the operation phase includes at least two of a positioning phase, a focusing phase, and a holding phase; the first evaluation submodule includes at least two of a positioning evaluation unit, a focusing evaluation unit, and a holding evaluation unit.

[0026] The positioning evaluation unit, connected to the data analysis module and the second evaluation submodule, is used to receive target data corresponding to each operation stage sent by the data analysis module; determine the positioning time evaluation result and the first instrument angle change evaluation result corresponding to the positioning stage based on the target data corresponding to the positioning stage; determine the operation evaluation result of the positioning stage based on the positioning time evaluation result and the first instrument angle change evaluation result corresponding to the positioning stage; and send the operation evaluation result of the positioning stage to the second evaluation submodule.

[0027] The focusing evaluation unit, connected to the data analysis module and the second evaluation submodule, is used to receive target data corresponding to each of the operation stages sent by the data analysis module; based on the target data corresponding to the focusing stage, determine the target's position evaluation result, target's proportion evaluation result, target's angle evaluation result, and second instrument angle change evaluation result corresponding to the focusing stage; based on the target's position evaluation result, target's proportion evaluation result, target's angle evaluation result, and second instrument angle change evaluation result corresponding to the focusing stage, determine the operation evaluation result of the focusing stage; and send the operation evaluation result of the focusing stage to the second evaluation submodule.

[0028] The holding evaluation unit, connected to the data analysis module and the second evaluation submodule, is used to receive target data corresponding to each of the operation stages sent by the data analysis module; based on the target data corresponding to the holding stage, determine the jitter evaluation result, stable holding duration evaluation result, and third instrument angle change evaluation result corresponding to the holding stage; based on the jitter evaluation result, stable holding duration evaluation result, and third instrument angle change evaluation result corresponding to the holding stage, determine the operation evaluation result of the holding stage; and send the operation evaluation result of the holding stage to the second evaluation submodule.

[0029] In one embodiment, the positioning evaluation unit includes: a time consumption evaluation subunit, a first angle change evaluation subunit, and a first evaluation subunit;

[0030] The time consumption evaluation subunit is connected to the data analysis module and the first evaluation subunit. It is used to receive target data corresponding to each operation stage sent by the data analysis module; determine the time consumption of the positioning target based on the target data corresponding to the positioning stage; determine the positioning time consumption evaluation result corresponding to the positioning stage based on the positioning time consumption and a preset positioning target time consumption threshold; and send the positioning time consumption evaluation result corresponding to the positioning stage to the first evaluation subunit.

[0031] The first angle change assessment subunit, connected to the data analysis module and the first assessment subunit, is used to receive target data corresponding to each of the operation stages sent by the data analysis module; determine the cumulative angle change of the instrument corresponding to the positioning stage based on the target data corresponding to the positioning stage; determine the first instrument angle change assessment result corresponding to the positioning stage based on the cumulative change of the instrument corresponding to the positioning stage and a preset first cumulative change threshold, and send the first instrument angle change assessment result corresponding to the positioning stage to the first assessment subunit.

[0032] The first evaluation subunit, connected to the second evaluation submodule, is used to determine the operation evaluation result of the positioning stage based on the received positioning time evaluation result and the first instrument angle change evaluation result, as well as the weights corresponding to the positioning time evaluation result and the first instrument angle change evaluation result, and then send the operation evaluation result of the positioning stage to the second evaluation submodule.

[0033] In one embodiment, the focus evaluation unit includes: a position evaluation subunit, a proportion evaluation subunit, an angle evaluation subunit, a second angle change evaluation subunit, and a second evaluation subunit;

[0034] The position evaluation subunit, connected to the data analysis module and the second evaluation subunit, is used to receive target data corresponding to each of the operation stages sent by the data analysis module; determine the distance between the target and the center point of the image frame in each image frame of the focusing stage based on the target data corresponding to the focusing stage; determine the position evaluation result of the target corresponding to the focusing stage based on the distance between the target and the center point of the image frame in each image frame of the focusing stage and a preset maximum tolerance distance; and send the position evaluation result of the target corresponding to the focusing stage to the second evaluation subunit.

[0035] The proportion evaluation subunit, connected to the data analysis module and the second evaluation subunit, is used to receive target data corresponding to each of the operation stages sent by the data analysis module; determine the area proportion of the target in each image frame of the focusing stage based on the target data corresponding to the focusing stage; determine the proportion evaluation result of the target corresponding to the focusing stage based on the area proportion of the target in each image frame of the focusing stage and a preset reference area proportion value; and send the proportion evaluation result of the target corresponding to the focusing stage to the second evaluation subunit.

[0036] The angle evaluation subunit, connected to the data analysis module and the second evaluation subunit, is used to receive target data corresponding to each of the operation stages sent by the data analysis module; determine the angle of the target in each image frame of the focusing stage based on the target data corresponding to the focusing stage; determine the angle evaluation result of the target corresponding to the focusing stage based on the angle of the target in each image frame of the focusing stage, as well as a preset optimal reference angle and a preset maximum allowable angle deviation; and send the angle evaluation result of the target corresponding to the focusing stage to the second evaluation subunit.

[0037] The second angle change evaluation subunit, connected to the data analysis module and the second evaluation subunit, is used to receive target data corresponding to each of the operation stages sent by the data analysis module; determine the cumulative angle change of the instrument corresponding to the focusing stage based on the target data corresponding to the focusing stage; determine the second instrument angle change evaluation result corresponding to the focusing stage based on the cumulative angle change of the instrument corresponding to the focusing stage and a preset second cumulative change threshold; and send the second instrument angle change evaluation result corresponding to the focusing stage to the second evaluation subunit.

[0038] The second evaluation subunit, connected to the second evaluation submodule, is used to determine the operation evaluation result of the focusing stage based on the received position evaluation result of the target, the proportion evaluation result of the target, the angle evaluation result of the target, and the angle change evaluation result of the second instrument corresponding to the focusing stage, as well as the weights corresponding to the position evaluation result of the target, the proportion evaluation result of the target, the angle evaluation result of the target, and the angle change evaluation result of the second instrument corresponding to the focusing stage, and send the operation evaluation result of the focusing stage to the second evaluation submodule.

[0039] In one embodiment, the holding evaluation unit includes: a jitter evaluation subunit, a stability holding evaluation subunit, a third angle change evaluation subunit, and a third evaluation subunit;

[0040] The jitter evaluation subunit, connected to the data analysis module and the third evaluation subunit, is used to receive target data corresponding to each operation stage sent by the data analysis module; determine the displacement of the target in each image frame of the holding stage based on the target data corresponding to the holding stage; determine the jitter evaluation result corresponding to the holding stage based on the displacement of the target in each image frame of the holding stage and a preset maximum displacement threshold; and send the jitter evaluation result corresponding to the holding stage to the third evaluation subunit.

[0041] The stability maintenance evaluation subunit, connected to the data analysis module and the third evaluation subunit, is used to receive target data corresponding to each operation stage sent by the data analysis module; determine the stability maintenance duration corresponding to the maintenance stage based on the target data corresponding to the maintenance stage; determine the stability maintenance duration evaluation result corresponding to the maintenance stage based on the stability maintenance duration corresponding to the maintenance stage and a preset stability duration threshold; and send the stability maintenance duration evaluation result corresponding to the maintenance stage to the third evaluation subunit.

[0042] The third angle change evaluation subunit is connected to the data analysis module and the third evaluation subunit. It is used to receive target data corresponding to each operation stage sent by the data analysis module; determine the cumulative angle change of the instrument corresponding to the holding stage based on the target data corresponding to the holding stage; determine the third instrument angle change evaluation result corresponding to the holding stage based on the cumulative angle change of the instrument corresponding to the holding stage and a preset third cumulative change threshold; and send the third instrument angle change evaluation result corresponding to the holding stage to the third evaluation subunit.

[0043] The third evaluation subunit, connected to the second evaluation submodule, is used to determine the operation evaluation result of the holding phase based on the received jitter evaluation result, stable holding duration evaluation result, and third instrument angle change evaluation result corresponding to the holding phase, as well as the weights corresponding to the jitter evaluation result, stable holding duration evaluation result, and third instrument angle change evaluation result corresponding to the holding phase, and to send the operation evaluation result of the holding phase to the second evaluation submodule.

[0044] The aforementioned endoscopic operation assessment system, through a data acquisition module, synchronously collects target data during the operator's execution of the target operation task, obtaining target data for the entire operation process. Using a data analysis module, based on the image data and instrument motion sensor data within the received target data of the entire operation process, it constructs a temporal data feature sequence of the entire operation process. This temporal data feature sequence is then analyzed to determine the target data corresponding to each operation stage. Finally, using an operation assessment module, based on the target data corresponding to each operation stage, the system determines the endoscopic operation assessment result for the entire operation process. By constructing and analyzing the temporal data feature sequence of the entire operation process, and by dividing the system into operation stages and determining the target data corresponding to each stage, it can perform adaptive assessments of each operation stage based on the target data corresponding to each operation stage, thereby determining the endoscopic operation assessment result for the entire operation process. This solves the problem of existing endoscopic operation assessment systems being unable to perform adaptive assessments of different operation stages, resulting in inaccurate assessment results.

[0045] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0046] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0047] Figure 1 This is a structural block diagram of the endoscopic operation evaluation system provided in Embodiment 1 of this application;

[0048] Figure 2 This is a structural block diagram of the endoscopic operation evaluation system provided in Embodiment 2 of this application;

[0049] Figure 3 This is a structural block diagram of the endoscopic operation evaluation system provided in Embodiment 3 of this application;

[0050] Figure 4 This is a structural block diagram of the endoscopic operation evaluation system provided in Embodiment 4 of this application;

[0051] Figure 5 This is a structural block diagram of the endoscopic operation evaluation system provided in Embodiment 5 of this application;

[0052] Figure 6 This is a structural block diagram of the endoscopic operation evaluation system provided in Embodiment Six of this application;

[0053] Figure 7 This is a structural block diagram of the endoscopic operation evaluation system provided in Embodiment 7 of this application;

[0054] Figure 8 This is a structural block diagram of the endoscopic operation evaluation system provided in Embodiment 8 of this application;

[0055] Figure 9 This is a structural block diagram of the endoscopic operation evaluation system provided in Embodiment 9 of this application;

[0056] Figure 10 This is a structural block diagram of the endoscopic operation evaluation system provided in Embodiment 10 of this application. Detailed Implementation

[0057] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0058] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0059] See Figure 1 , Figure 1 This is a structural block diagram of the endoscopic operation evaluation system provided in Embodiment 1 of this application, as follows: Figure 1As shown, the system includes: a data acquisition module 110, a data analysis module 120, and an operation evaluation module 130. The data acquisition module 110 is connected to the data analysis module 120 and is used to synchronously acquire target data during the operation of the operator, obtain target data for the entire operation process, and send the obtained target data for the entire operation process to the data analysis module 120. The target data includes image data and motion sensing data of the instrument. The data analysis module 120 is connected to the operation evaluation module 130 and is used to construct a time-series data feature sequence of the entire operation process based on the received target data of the entire operation process, analyze the constructed time-series data feature sequence, determine the target data corresponding to each operation stage in the entire operation process, and send the determined target data corresponding to each operation stage to the operation evaluation module 130. The operation evaluation module 130 is used to determine the endoscopic operation evaluation result of the entire operation process based on the target data corresponding to each operation stage.

[0060] The aforementioned operators can be students or other endoscopic operators requiring training in endoscopic procedures, i.e., operators who need to be observed, recorded, or evaluated by the system in real time. The aforementioned endoscopes can refer to optical or electronic imaging instruments that can be inserted into natural cavities, surgical channels, or the cavity of a phantom, such as gastroscopes, colonoscopes, and bronchoscopes. The aforementioned target operation tasks can be pre-set operations performed on a target area using the endoscope. The aforementioned target areas can be regions of different organs in the human body, such as the stomach or uterus, or different simulated regions of the phantom. The aforementioned pre-set operation tasks can be one or more of the following: comparison tasks, examination tasks, observation tasks, sampling tasks, and quality analysis tasks. It should be noted that the pre-set operation tasks corresponding to different target areas are not entirely the same; therefore, the aforementioned pre-set operation tasks can be operation tasks related to the target area. The aforementioned instruments can be auxiliary tools for operators when performing target operation tasks, and can be tools that can penetrate deep into the phantom to complete the target operation tasks. Specifically, they can be one or more of the following: biopsy forceps, cell brushes, foreign body forceps, hysteroscopic scissors, and electrocoagulation hooks. The entire operation process described above can be seen as the entire process from start to finish for the operator to perform the target operation task.

[0061] The aforementioned image data includes at least image frame data of each target corresponding to the target operation task collected by the operator using an endoscope, and video stream of the entire operation process captured by the endoscopic camera. The aforementioned targets can be one or more of the following: abnormal tissue or lesion, surgical location, instruments, anatomical landmarks, normal anatomical structures, secretions, or body fluids. The aforementioned motion sensing data of the instruments can be motion data of the instruments collected by sensors installed within the instruments. The aforementioned motion data of the instruments can be at least one of the following: three-dimensional position, attitude, velocity, and acceleration of the instruments. The aforementioned sensors can be accelerometers, gyroscopes, magnetometers, or other sensors capable of collecting the motion of the instruments. When the sensor is a gyroscope, the data information collected by the gyroscope can be used as the motion sensing data of the instruments. In this case, the aforementioned motion sensing data of the instruments at least includes the angle change information of the gyroscope. The aforementioned construction of the temporal data feature sequence of the entire operation process based on the received target data of the entire operation process can be achieved by using a preset temporal data feature construction model to construct temporal data features of the target data of the entire operation process, thereby obtaining the temporal data feature sequence of the entire operation process. The aforementioned preset time-series data feature construction model is not specifically limited in this embodiment. As long as it can construct time-series data features for the target data of the entire operation process and obtain the time-series data feature sequence of the entire operation process, it is acceptable.

[0062] The aforementioned construction of a temporal data feature sequence for the entire operation process based on the received target data of the entire operation process can also involve extracting image features from each image frame in the video stream of the entire operation process captured by the endoscope camera, determining the image features of each image frame in the video stream of the entire operation process, arranging the extracted image features in chronological order to obtain the temporal features of the images of the entire operation process, and then extracting motion sensing features from the motion sensing data of the instrument in the entire operation process to obtain the motion sensing features of the motion sensing data of the instrument in the entire operation process. Arranging the extracted motion sensing features in chronological order to obtain the temporal features of the motion sensing of the entire operation process, and adjusting the temporal features of the images of the entire operation process with the temporal features of the motion sensing of the entire operation process through interpolation or resampling methods to align them temporally with the temporal features of the motion sensing of the entire operation process, and then performing feature fusion to obtain the temporal data feature sequence of the entire operation process. The aforementioned feature fusion can be achieved by using a simple stitching method or employing methods such as independent component analysis to fuse the image features and motion sensing features at the same moment into a comprehensive feature vector. The aforementioned image features can include at least one of the following: color features, texture features, shape features, or motion features. The aforementioned motion sensing features may include at least one of the following: position features, attitude features, velocity features, acceleration features, and motion pattern features. The aforementioned operation phase includes at least two of the following: a positioning phase, a focusing phase, and a holding phase. The aforementioned temporal data feature sequence is a temporal feature sequence formed by fusing image features and the motion sensing features of the device.

[0063] In addition, the aforementioned positioning stage describes the process by which the operator moves the endoscopic instrument from the inlet to the target area. The aforementioned focusing stage describes the process by which the operator adjusts the position and angle of the endoscope to make the target area or target clearly visible. The aforementioned holding stage describes the process by which the operator keeps the endoscope stable after focusing and continues to observe the target area.

[0064] In this embodiment, a time-series data feature sequence of the entire operation process is constructed. Then, by analyzing the constructed time-series data feature sequence, the target data corresponding to each operation stage in the entire operation process is determined. By dividing each operation stage, the target data corresponding to each operation stage is determined. Based on the target data corresponding to each operation stage, an adaptive evaluation of each operation stage can be performed to obtain the endoscopic operation evaluation results of each operation stage. Then, based on the endoscopic operation evaluation results of each operation stage, the endoscopic operation evaluation result of the entire operation process is determined. This solves the problem that existing endoscopic operation evaluation systems cannot perform adaptive evaluation of different operation stages and have inaccurate evaluation results.

[0065] In one embodiment, after generating the endoscopic operation evaluation results for the entire procedure, a result evaluation report can also be generated. The result evaluation report may include the evaluation results for each stage, as well as the endoscopic operation evaluation results for the entire procedure, and may list the points of failure in the evaluation results for each stage to identify the weak links in the entire procedure.

[0066] In Example 2, another endoscopic procedure evaluation system is also provided, such as... Figure 2 As shown, based on Embodiment 1, the system includes a data acquisition module 110 comprising an image acquisition submodule 112 and a sensor data acquisition submodule 114. The image acquisition submodule 112, connected to the data analysis module 120, is used to acquire image frame sequences of the operator performing the target operation task using the target endoscope, obtaining image data of the entire operation process, and sending the obtained image data of the entire operation process to the data analysis module 120. The target endoscope is an endoscope matched to the target operation task. The sensor data acquisition submodule 114, connected to the data analysis module 120, is used to acquire motion sensing data of the instrument when the operator performs the target operation task using sensors installed in the instrument, and sending the acquired motion sensing data of the instrument when the operator performs the target operation task to the data analysis module 120.

[0067] Specifically, the target endoscope mentioned above is an endoscope matched to the target operational task. Specifically, the type of endoscope can be one of the following: gastroscopy, colonoscopy, bronchoscopy, nasopharyngoscopy, laryngoscopy, cystoscopy, etc. For example, when the target operational task is to observe the stomach, the target endoscope is a gastroscopy.

[0068] Specifically, in Embodiment 3, another structural block diagram of an endoscopic operation assessment system is also provided, such as... Figure 3 As shown, the system is an improvement on the first embodiment. The data analysis module 120 includes a sequence construction submodule 122 and a data analysis submodule 124. The sequence construction submodule 122 is connected to the data analysis submodule 124 and the data acquisition module 110. It is used to construct a time-series data feature sequence of the entire operation process based on the target data of the entire operation process sent by the data acquisition module 110, and send the constructed time-series data feature sequence to the data analysis submodule 124. The data analysis submodule 124 is connected to the operation evaluation module 130. It is used to analyze the received time-series data feature sequence, determine the target data corresponding to each operation stage in the entire operation process, and send the determined target data corresponding to each operation stage to the operation evaluation module 130.

[0069] The above-mentioned analysis of the received time-series data feature sequence to determine the target data corresponding to each operational stage in the entire operation process can be achieved by using a preset time-series classification model to analyze the received time-series data feature sequence, determine the operational stages corresponding to different sequence intervals in the time-series data feature sequence, and then determine the target data corresponding to each operational stage in the entire operation process based on the operational stages corresponding to different sequence intervals. It should be noted that the type of the preset time-series classification model is not specifically limited in this embodiment. For example, the preset time-series classification model can be an LSTM (Long Short-Term Memory) network or a video classification model based on the Transformer architecture, as long as it can be used to analyze the received time-series data feature sequence and determine the operational stages corresponding to different sequence intervals in the time-series data feature sequence. The aforementioned sequence interval can be obtained by segmenting the time-series data feature sequence according to preset segmentation rules, and the interval range corresponding to each segmentation result is the sequence interval of the segmentation result. The aforementioned preset segmentation rules can be used to segment according to preset intervals in time sequence or to segment according to actual content. This embodiment does not specifically limit the specific segmentation rules, as long as they can reasonably segment the time sequence data feature sequence.

[0070] In addition, in Embodiment 4, a structural block diagram of another endoscopic operation evaluation system is also provided, such as... Figure 4 As shown, the system provided in Embodiment 4 is based on Embodiment 3, wherein the data analysis submodule 124 includes: a data analysis unit 1242 and a data determination unit 1244; the data analysis unit 1242 is connected to the sequence construction submodule 122 and the data determination unit 1244 respectively, and is used to analyze the time series data feature sequence sent by the sequence construction submodule 122, determine the operation stage corresponding to different sequence intervals in the time series data feature sequence, and send the determined operation stages corresponding to different sequence intervals to the data determination unit 1244; the data determination unit 1244 is connected to the operation evaluation module 130, and is used to determine the target data corresponding to each operation stage in the entire operation process based on the operation stages corresponding to the received different sequence intervals, and send the target data corresponding to each operation stage to the operation evaluation module 130.

[0071] The above-mentioned determination of the target data corresponding to each operation stage in the entire operation process based on the operation stages corresponding to different sequence intervals can be based on the operation stages corresponding to different sequence intervals and the target data corresponding to different sequence intervals to determine the target data corresponding to each operation stage in the entire operation process.

[0072] In Embodiment 5, another structural block diagram of an endoscopic operation assessment system is also provided, such as... Figure 5 As shown, in Embodiment 5, based on Embodiment 4, the data analysis unit 1242 includes: a model construction subunit 12422 and a data analysis subunit 12424; the model construction subunit 12422 is connected to the data analysis subunit 12424 and is used to train the initial time series classification model to obtain the trained time series classification model, and send the trained time series classification model to the data analysis subunit 12424; the data analysis subunit 12424 is connected to the data determination unit 1244 and the sequence construction submodule 122 respectively, and is used to receive the time series data feature sequence sent by the sequence construction submodule 122, and use the received trained time series classification model to analyze the received time series data feature sequence, determine the operation stage corresponding to different sequence intervals in the time series data feature sequence, and send the determined operation stages corresponding to different sequence intervals to the data determination unit 1244.

[0073] Furthermore, in Embodiment Six, another structural block diagram of an operational evaluation system is also provided, such as... Figure 6 As shown, in Embodiment Six, based on Embodiment One, the operation evaluation module 130 includes: a first evaluation submodule 132 and a second evaluation submodule 134; the first evaluation submodule 132, connected to the data analysis module 120 and the second evaluation submodule 134, is used to receive the target data corresponding to each operation stage sent by the data analysis module 120, and based on the received target data corresponding to each operation stage and the preset evaluation rules for each operation stage, determine the operation evaluation result of each operation stage, and send the operation evaluation result of each operation stage to the second evaluation submodule 134; the second evaluation submodule 134 is used to perform weighted calculation on the received operation evaluation results of each operation stage based on the weight of the operation evaluation results of each operation stage, to obtain the endoscopic operation evaluation result of the entire operation process.

[0074] The above-mentioned weighted calculation of the operation evaluation results of each operation stage is performed on the received operation evaluation results of each operation stage to obtain the endoscopic operation evaluation result of the entire operation process. The calculation process is as follows:

[0075] ;

[0076] in, The operation evaluation result for the positioning phase is represented by α, where α is the weight corresponding to the operation evaluation result for the positioning phase. β represents the operational evaluation result of the focusing stage, and β represents the weight corresponding to the operational evaluation result of the focusing stage. To maintain the operational evaluation results of the retention phase, γ is the weight corresponding to the operational evaluation results of the retention phase. The values ​​of α, β, and γ can be specifically set based on specific needs; this embodiment does not impose specific limitations, as long as the sum of α, β, and γ is 1. For example, α, β, and γ are all... .

[0077] Furthermore, in Embodiment Seven, another structural block diagram of an endoscopic operation evaluation system is also provided, such as... Figure 7 As shown, the above operation stages include at least two of the following: a positioning stage, a focusing stage, and a holding stage. Based on Embodiment Six, the first evaluation submodule 132 includes at least two of the following: a positioning evaluation unit 1322, a focusing evaluation unit 1324, and a holding evaluation unit 1326. The positioning evaluation unit 1322 is connected to the data analysis module 120 and the second evaluation submodule 134, and is used to receive target data corresponding to each operation stage sent by the data analysis module 120; based on the target data corresponding to the positioning stage, it determines the positioning time evaluation result and the first instrument angle change evaluation result corresponding to the positioning stage; based on the positioning time evaluation result and the first instrument angle change evaluation result corresponding to the positioning stage, it determines the operation evaluation result of the positioning stage; and sends the operation evaluation result of the positioning stage to the second evaluation submodule 134. The focusing evaluation unit 1324 is connected to the data analysis module 120 and the second evaluation submodule 134, and is used to receive target data corresponding to each operation stage sent by the data analysis module 120; based on the target data corresponding to the focusing stage, it determines the positioning time evaluation result and the first instrument angle change evaluation result corresponding to the positioning stage; based on the positioning time evaluation result and the first instrument angle change evaluation result corresponding to the positioning stage, it determines the operation evaluation result of the positioning stage; and sends the operation evaluation result of the positioning stage to the second evaluation submodule 134. Based on the corresponding target data, the following evaluation results are determined for the focusing stage: target position evaluation result, target proportion evaluation result, target angle evaluation result, and second instrument angle change evaluation result. The operation evaluation result for the focusing stage is then determined based on these results. This operation evaluation result is sent to the second evaluation submodule 134. The aforementioned holding evaluation unit 1326, connected to the data analysis module 120 and the second evaluation submodule 134, receives the target data corresponding to each operation stage sent by the data analysis module 120. Based on the target data for the holding stage, the following evaluation results are determined: shaking evaluation result, stable holding time evaluation result, and third instrument angle change evaluation result. The operation evaluation result for the holding stage is then determined based on these results. This operation evaluation result is then sent to the second evaluation submodule 134.

[0078] The above-mentioned positioning time assessment results are used to characterize the speed at which the operator determines the target during the positioning phase. A higher score indicates a faster target determination by the operator; a lower score indicates a slower target determination by the operator. The above-mentioned first instrument angle change assessment results are used to characterize the cumulative angle change of the instrument during the positioning phase. A higher score indicates a smaller cumulative angle change by the instrument during the positioning phase; a lower score indicates a larger cumulative angle change by the instrument during the positioning phase.

[0079] Furthermore, the target position evaluation results described above are used to characterize the quality of the target's position in the image frames captured during the focusing phase. A higher target position evaluation score indicates a better target position in the image frames captured during the focusing phase; a lower score indicates a worse target position. The target proportion evaluation results described above are used to characterize the quality of the target's proportion in the image frames captured during the focusing phase. A higher proportion evaluation score indicates a better proportion of the target in the image frames captured during the focusing phase; a lower score indicates a worse proportion. The target proportion can be considered as the percentage of the target's size within the entire image frame. The target angle evaluation results described above are used to characterize the deviation of the target's angle in the image frames captured during the focusing phase. A higher score in the target angle evaluation indicates a smaller angular deviation of the target in the image frames captured during the focusing phase; conversely, a lower score indicates a larger angular deviation of the target in the image frames captured during the focusing phase. The aforementioned second instrument angle change evaluation result is used to characterize the cumulative angular change of the instrument during the focusing phase. A higher score in the second instrument angle change evaluation result indicates a smaller cumulative angular change of the instrument during the focusing phase; conversely, a lower score indicates a larger cumulative angular change of the instrument during the focusing phase.

[0080] Furthermore, the aforementioned jitter evaluation results are used to characterize the jitter of the target in the image frames captured during the holding phase. A higher jitter evaluation score indicates less jitter in the image frames captured during the holding phase; a lower score indicates greater jitter. The aforementioned stable holding duration evaluation results are used to characterize the duration of stable holding during the holding phase. A higher stable holding duration evaluation score indicates a longer duration of stable holding during the holding phase; a lower score indicates a shorter duration of stable holding during the holding phase. The aforementioned third instrument angle change evaluation results are used to characterize the cumulative angle change of the instrument during the holding phase. A higher score indicates a smaller cumulative angle change of the instrument during the holding phase; a lower score indicates a larger cumulative angle change of the instrument during the holding phase.

[0081] In embodiment eight, another structural block diagram of an operational evaluation system is also provided, such as... Figure 8 As shown, based on Embodiment 7, the positioning evaluation unit 1322 includes: a time consumption evaluation subunit 13222, a first angle change evaluation subunit 13224, and a first evaluation subunit 13226; the time consumption evaluation subunit 13222 is connected to the data analysis module 120 and the first evaluation subunit 13226, and is used to receive the target data corresponding to each operation stage sent by the data analysis module 120; determine the time consumption of the positioning target based on the target data corresponding to the positioning stage; determine the positioning time consumption evaluation result corresponding to the positioning stage based on the positioning time consumption and a preset positioning target time consumption threshold, and send the positioning time consumption evaluation result corresponding to the positioning stage to the first evaluation subunit 13226.

[0082] Among them, the positioning time evaluation result corresponding to the positioning stage is determined based on the time spent locating the target and the preset positioning target time threshold. The calculation process is as follows:

[0083] ;

[0084] in, The time taken to locate the target, This is the preset target positioning time threshold. The above positioning time evaluation results... , represented by a value between (0, 1).

[0085] The preset target positioning time threshold can be set based on experience or actual needs, and this embodiment does not impose specific limitations on it. The target positioning time can be obtained by analyzing the target data corresponding to the positioning stage using a preset positioning time determination model. This embodiment does not impose specific limitations on the preset positioning time determination model, as long as it can be used to analyze the video stream in the target data corresponding to the positioning stage to obtain the target positioning time. Alternatively, the target positioning time can be obtained using a target detection algorithm, starting timing from the start of camera movement to the time the algorithm detects the target, and using the time from the start to the stop of timing as the target positioning time. This embodiment does not impose specific limitations on the target detection algorithm, as long as it can detect the target.

[0086] See also Figure 8 The aforementioned first angle change assessment subunit 13224 is connected to the data analysis module 120 and the first assessment subunit 13226. It is used to receive the target data corresponding to each operation stage sent by the data analysis module 120; determine the cumulative angle change of the instrument corresponding to the positioning stage based on the target data corresponding to the positioning stage; determine the first instrument angle change assessment result corresponding to the positioning stage based on the cumulative change of the instrument corresponding to the positioning stage and the preset first cumulative change threshold; and send the first instrument angle change assessment result corresponding to the positioning stage to the first assessment subunit 13226.

[0087] The above-mentioned cumulative change of the instrument during the positioning phase, and the preset first cumulative change threshold, determine the evaluation result of the first instrument angle change during the positioning phase. The calculation process is as follows:

[0088] ;

[0089] in, This refers to the cumulative change in the instrument during the positioning phase, specifically the cumulative change in the gyroscope angle. The preset first cumulative change threshold can be the maximum cumulative magnitude allowed during the positioning phase.

[0090] The value of the aforementioned preset first cumulative change threshold can be set according to specific circumstances, and this embodiment does not impose a specific limitation here. The aforementioned determination of the cumulative angle change of the instrument corresponding to the positioning stage based on the target data corresponding to the positioning stage can be based on the data received by the sensors in the instrument during the positioning stage to determine the cumulative angle change of the instrument corresponding to the positioning stage.

[0091] See also Figure 8The first evaluation subunit 13226 is connected to the second evaluation submodule 134. It is used to determine the operation evaluation result of the positioning stage based on the received positioning time evaluation result and the first instrument angle change evaluation result, as well as the weights corresponding to the positioning time evaluation result and the first instrument angle change evaluation result, and send the operation evaluation result of the positioning stage to the second evaluation submodule 134.

[0092] Based on the received positioning time evaluation results and first instrument angle change evaluation results for the positioning phase, as well as the weights corresponding to the positioning time evaluation results and the first instrument angle change evaluation results for the positioning phase, the operation evaluation result for the positioning phase is determined. The calculation process is as follows:

[0093] ;

[0094] in, The weights corresponding to the positioning time evaluation results for the positioning phase. This represents the weight corresponding to the evaluation result of the first instrument angle change. Wherein, and The value can be set according to specific needs. This embodiment does not impose specific limitations here, as long as it ensures... and The sum of the terms must be 1. For example, and All .

[0095] In addition, in Embodiment Nine, another structural block diagram of an operational evaluation system is also provided, such as... Figure 9 As shown, based on Embodiment 7, the focus evaluation unit 1324 includes: a position evaluation subunit 13241, a proportion evaluation subunit 13242, an angle evaluation subunit 13243, a second angle change evaluation subunit 13244, and a second evaluation subunit 13245. The position evaluation subunit 13241 is connected to the data analysis module 120 and the second evaluation subunit 13245, and is used to receive target data corresponding to each operation stage sent by the data analysis module 120; determine the distance between the target and the center point of the image frame in each image frame of the focus stage based on the target data corresponding to the focus stage; determine the position evaluation result of the target corresponding to the focus stage based on the distance between the target and the center point of the image frame in each image frame of the focus stage, and the preset maximum tolerance distance; and send the position evaluation result of the target corresponding to the focus stage to the second evaluation subunit 13245.

[0096] The aforementioned image frames are image frames of the targets corresponding to the target operation tasks acquired by the operator using an endoscope. The determination of the target's position assessment result during the focusing stage, based on the distance between the target and the center point of each image frame in the focusing stage, and a preset maximum tolerance distance, can be achieved by calculating the average distance between the target and the center point of each image frame in the focusing stage, and then using this average distance, along with the preset maximum tolerance distance, to determine the target's position assessment result during the focusing stage. Specifically, the distance between the center point of the target in one image frame and the center point of the image frame can be used as the distance between the target and the center point of the image frame.

[0097] Specifically, the target position evaluation result for the focusing stage is determined based on the average distance between the target and the center point of each image frame during the focusing stage, and a preset maximum tolerance distance. The calculation process is as follows:

[0098] ;

[0099] Where d is the average distance between the target and the center point of the image frame in each image frame during the focusing phase. This is the preset maximum tolerance distance. The value of the preset maximum tolerance distance can be specifically set based on experience or actual needs, and this embodiment does not impose a specific limitation here.

[0100] See also Figure 9 The aforementioned proportion evaluation subunit 13242 is connected to the data analysis module 120 and the second evaluation subunit 13245. It is used to receive the target data corresponding to each operation stage sent by the data analysis module 120; determine the area proportion of the target in each image frame of the focusing stage based on the target data corresponding to the focusing stage; determine the proportion evaluation result of the target corresponding to the focusing stage based on the area proportion of the target in each image frame of the focusing stage and a preset reference area proportion value; and send the proportion evaluation result of the target corresponding to the focusing stage to the second evaluation subunit 13245.

[0101] The above-mentioned determination of the target proportion evaluation result in the focusing stage based on the area proportion of the target in each image frame in the focusing stage and the preset reference area proportion value can be used to determine the average area proportion of the target in each image frame in the focusing stage, and then, based on the average area proportion of the target in each image frame in the focusing stage and the preset reference area proportion value, to determine the target proportion evaluation result in the focusing stage.

[0102] Specifically, the target area ratio assessment result for the focusing stage is determined based on the average area ratio of the target in each image frame during the focusing stage, and a preset reference area ratio value. The calculation process is as follows:

[0103] ;

[0104] in, This represents the average area ratio of the target within each image frame during the focusing phase. This is a preset reference area percentage value.

[0105] The aforementioned preset reference area ratio is the optimal area ratio and can be specifically set based on experience or actual needs. This embodiment does not impose any specific limitations on it.

[0106] See also Figure 9 The aforementioned angle evaluation subunit 13243 is connected to the data analysis module 120 and the second evaluation subunit 13245. It is used to receive target data corresponding to each operation stage sent by the data analysis module 120; determine the angle of the target in each image frame of the focusing stage based on the target data corresponding to the focusing stage; determine the angle evaluation result of the target corresponding to the focusing stage based on the angle of the target in each image frame of the focusing stage, as well as the preset optimal reference angle and the preset maximum allowable angle deviation; and send the angle evaluation result of the target corresponding to the focusing stage to the second evaluation subunit 13245.

[0107] The above method, based on the angle of the target in each image frame during the focusing stage, as well as the preset optimal reference angle and the preset maximum allowable angle deviation, determines the angle evaluation result of the target corresponding to the focusing stage. This method can determine the average angle of the target in each image frame during the focusing stage, and then use the average offset angle of the target in each image frame during the focusing stage, as well as the preset optimal reference angle and the preset maximum allowable angle deviation, to determine the angle evaluation result of the target corresponding to the focusing stage.

[0108] The above method utilizes the average offset angle of the target in each image frame during the focusing stage, along with a preset optimal reference angle and a preset maximum allowable angle deviation, to determine the target angle evaluation result corresponding to the focusing stage. The calculation process is as follows:

[0109] ;

[0110] in, This represents the average offset angle of the target across all image frames during the focusing phase. This is the preset optimal reference angle. This is the preset maximum allowable angle deviation.

[0111] The preset optimal reference angle and the preset maximum allowable angle deviation can be specifically set based on experience or actual needs, and this embodiment does not impose specific limitations here.

[0112] The angle of the target mentioned above can be the offset angle of the target's position in the image frame relative to the standard position of the target in the image frame. The standard position can be specifically set based on specific circumstances, and this embodiment does not impose a specific limitation on it. The angle of the target in the image frame can be calculated using a preset angle calculation model for each image frame. This embodiment does not impose a specific limitation on the preset angle calculation model, as long as it can calculate the angle of the target in each image frame.

[0113] See also Figure 9 The aforementioned second angle change assessment subunit 13244 is connected to the data analysis module 120 and the second assessment subunit 13245. It is used to receive the target data corresponding to each operation stage sent by the data analysis module 120; determine the cumulative angle change of the instrument corresponding to the focusing stage based on the target data corresponding to the focusing stage; determine the second instrument angle change assessment result corresponding to the focusing stage based on the cumulative angle change of the instrument corresponding to the focusing stage and the preset second cumulative change threshold; and send the second instrument angle change assessment result corresponding to the focusing stage to the second assessment subunit 13245.

[0114] The above-mentioned assessment result of the second instrument angle change corresponding to the focusing stage is determined based on the cumulative change in instrument angle during the focusing stage and the preset second cumulative change threshold. The calculation process is as follows:

[0115] ;

[0116] in, This represents the cumulative change in the instrument's angle during the focusing phase. This is the preset second cumulative change threshold.

[0117] The value of the aforementioned preset second cumulative change threshold can be set according to specific circumstances, and this embodiment does not impose a specific limitation here. The aforementioned determination of the cumulative angle change of the instrument corresponding to the focusing stage based on the target data corresponding to the focusing stage can be based on the data received by the sensor in the instrument during the focusing stage to determine the cumulative angle change of the instrument corresponding to the focusing stage.

[0118] See also Figure 9 The aforementioned second evaluation subunit 13245 is connected to the second evaluation submodule 134. It is used to determine the operation evaluation result of the focusing stage based on the received target position evaluation result, target proportion evaluation result, target angle evaluation result, and second instrument angle change evaluation result corresponding to the focusing stage, as well as the weights corresponding to the target position evaluation result, target proportion evaluation result, target angle evaluation result, and second instrument angle change evaluation result corresponding to the focusing stage, and send the operation evaluation result of the focusing stage to the second evaluation submodule 134.

[0119] The above determines the operational evaluation results of the focusing stage. The calculation process is as follows:

[0120] ;

[0121] in, The weights corresponding to the target's position assessment results during the focusing phase. The weight corresponding to the proportion of the target evaluation results. The weights corresponding to the evaluation results from the perspective of the objective. The weight corresponding to the evaluation result of the second instrument angle change.

[0122] in, , , as well as The value can be set according to specific needs. This embodiment does not impose specific limitations here, as long as it ensures... , , as well as The sum of the terms must be 1. For example, , , All , for .

[0123] In Embodiment 10, another structural block diagram of an operational evaluation system is also provided, such as... Figure 10 As shown, based on Embodiment 7, the holding evaluation unit 1326 includes: a jitter evaluation subunit 13262, a stable holding evaluation subunit 13264, a third angle change evaluation subunit 13266, and a third evaluation subunit 13268. The jitter evaluation subunit is connected to the data analysis module 120 and the third evaluation subunit 13268, and is used to receive the target data corresponding to each operation stage sent by the data analysis module 120; determine the displacement of the target in each image frame of the holding stage based on the target data corresponding to the holding stage; determine the jitter evaluation result corresponding to the holding stage based on the displacement of the target in each image frame of the holding stage and a preset maximum displacement threshold; and send the jitter evaluation result corresponding to the holding stage to the third evaluation subunit 13268.

[0124] The displacement of the target in the aforementioned image frame can be considered as the displacement difference between the target's position in the current image frame and its position in the previous image frame within the video stream of the entire operation captured by the endoscopic camera. The target's position in the current image frame can be represented by the position of the target's center within the current image frame.

[0125] The above method determines the jitter evaluation result corresponding to the holding phase based on the displacement of the target in each image frame during the holding phase and a preset maximum displacement threshold. The calculation process is as follows:

[0126] ;

[0127] in, To maintain the displacement of the target in the i-th image frame of the stage, This is the preset maximum displacement threshold.

[0128] The value of the preset maximum displacement threshold can be set according to specific circumstances, and this embodiment does not impose any specific limitations on it.

[0129] See also Figure 10 The aforementioned stability maintenance evaluation subunit is connected to the data analysis module 120 and the third evaluation subunit 13268. It is used to receive the target data corresponding to each operation stage sent by the data analysis module 120; determine the stability maintenance duration corresponding to the maintenance stage based on the target data corresponding to the maintenance stage; determine the stability maintenance duration evaluation result corresponding to the maintenance stage based on the stability maintenance duration corresponding to the maintenance stage and the preset stability duration threshold; and send the stability maintenance duration evaluation result corresponding to the maintenance stage to the third evaluation subunit 13268.

[0130] The above-mentioned stable holding duration assessment result is determined based on the stable holding duration corresponding to the holding phase and the preset stable duration threshold. The calculation process is as follows:

[0131] ;

[0132] in, In order to maintain the stability of the corresponding stage for a certain duration, This is the preset stable duration threshold.

[0133] The aforementioned preset stabilization duration threshold value can be set according to specific circumstances, and this embodiment does not impose a specific limitation. The determination of the stable holding duration corresponding to the holding phase based on the target data corresponding to the holding phase can be achieved by using a preset stable holding duration calculation model to analyze the video stream in the target data corresponding to the holding phase and determine the stable holding duration. This embodiment does not impose a specific limitation on the aforementioned preset stable holding duration calculation model, as long as it can analyze the video stream in the target data corresponding to the holding phase and determine the stable holding duration.

[0134] See also Figure 10 The aforementioned third angle change assessment subunit is connected to the data analysis module 120 and the third assessment subunit 13268. It is used to receive the target data corresponding to each operation stage sent by the data analysis module 120; determine the cumulative angle change of the device corresponding to the holding stage based on the target data corresponding to the holding stage; determine the third device angle change assessment result corresponding to the holding stage based on the cumulative angle change of the device corresponding to the holding stage and the preset third cumulative change threshold; and send the third device angle change assessment result corresponding to the holding stage to the third assessment subunit 13268.

[0135] The above-mentioned assessment result of the third instrument angle change during the holding phase is determined based on the cumulative change in the instrument angle during the holding phase and a preset third cumulative change threshold. The calculation process is as follows:

[0136] ;

[0137] in, To maintain the cumulative change in the instrument's angle corresponding to each stage, This is the preset third cumulative change threshold.

[0138] The value of the aforementioned preset third cumulative change threshold can be set according to specific circumstances, and this embodiment does not impose a specific limitation. The determination of the cumulative angle change of the device corresponding to the holding phase based on the target data corresponding to the holding phase can be based on data received by the sensors in the device during the holding phase.

[0139] See also Figure 10 The aforementioned third evaluation subunit 13268 is connected to the second evaluation submodule 134 and is used to determine the operation evaluation result of the holding phase based on the received jitter evaluation result, stable holding duration evaluation result, and third instrument angle change evaluation result corresponding to the holding phase, as well as the weights corresponding to the jitter evaluation result, stable holding duration evaluation result, and third instrument angle change evaluation result corresponding to the holding phase, and send the operation evaluation result of the holding phase to the second evaluation submodule 134.

[0140] The above-mentioned operational assessment results for the maintenance phase were determined. The calculation process is as follows:

[0141] ;

[0142] in, To maintain the weights corresponding to the jitter evaluation results for each stage, To maintain the weights corresponding to the duration evaluation results stably, The weight corresponding to the evaluation result of the angle change of the third instrument.

[0143] in, , as well as The value can be set according to specific needs. This embodiment does not impose specific limitations here, as long as it ensures... , as well as The sum of the terms must be 1. For example, and All , for .

[0144] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0145] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0146] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0147] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0148] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. An endoscopic procedure evaluation system, characterized in that, The system includes: a data acquisition module, a data analysis module, and an operation evaluation module; The data acquisition module, connected to the data analysis module, is used to synchronously acquire target data during the operator's execution of the target operation task, obtain target data for the entire operation process, and send the obtained target data for the entire operation process to the data analysis module; the target data includes image data and motion sensing data of the device; The data analysis module, connected to the operation evaluation module, is used to extract image features from the image data based on the received target data of the entire operation process to obtain the temporal features of the image of the entire operation process; extract motion sensing features from the motion sensing data of the device to obtain the temporal features of motion sensing of the entire operation process; align the temporal features of the image and the temporal features of the motion sensing through interpolation or resampling methods, and fuse the temporal features of the time-aligned image and the temporal features of the motion sensing to obtain the temporal data feature sequence of the entire operation process; analyze the constructed temporal data feature sequence using a preset temporal classification model to determine the operation stage corresponding to different sequence intervals in the temporal data feature sequence; determine the target data corresponding to each operation stage in the entire operation process based on the operation stages corresponding to the different sequence intervals, and send the determined target data corresponding to each operation stage to the operation evaluation module. The operation evaluation module is used to determine the endoscopic operation evaluation result of the entire operation process based on the target data corresponding to each of the operation stages.

2. The endoscopic operation evaluation system according to claim 1, characterized in that, The data acquisition module includes: an image acquisition submodule and a sensor data acquisition submodule; The image acquisition submodule is connected to the data analysis module and is used to acquire image frame sequences of the operator performing the target operation task using the target endoscope, obtain image data of the entire operation process, and send the obtained image data of the entire operation process to the data analysis module; the target endoscope is an endoscope matched to the target operation task; The sensor data acquisition submodule is connected to the data analysis module and is used to acquire motion sensing data of the instrument when the operator performs the target operation task using sensors installed in the instrument, and send the acquired motion sensing data of the instrument when the operator performs the target operation task to the data analysis module.

3. The endoscopic operation evaluation system according to claim 1, characterized in that, The data analysis module includes: a sequence construction submodule and a data analysis submodule; The sequence construction submodule, connecting the data analysis submodule and the data acquisition module, is used to extract image features from the image data based on the target data of the entire operation process sent by the data acquisition module, to obtain the temporal features of the image of the entire operation process; to extract motion sensing features from the motion sensing data of the device, to obtain the temporal features of motion sensing of the entire operation process; to align the temporal features of the image with the temporal features of the motion sensing through interpolation or resampling methods; and to fuse the temporal features of the time-aligned image with the temporal features of the motion sensing to construct a temporal data feature sequence of the entire operation process, and to send the constructed temporal data feature sequence to the data analysis submodule. The data analysis submodule, connected to the operation evaluation module, is used to analyze the received time-series data feature sequence using the preset time-series classification model, determine the operation stage corresponding to different sequence intervals in the time-series data feature sequence, determine the target data corresponding to each operation stage in the entire operation process based on the operation stages corresponding to different sequence intervals, and send the determined target data corresponding to each operation stage to the operation evaluation module.

4. The endoscopic operation evaluation system according to claim 3, characterized in that, The data analysis submodule includes: a data analysis unit and a data determination unit; The data analysis unit is connected to the sequence construction submodule and the data determination unit, respectively. It is used to analyze the time-series data feature sequence sent by the sequence construction submodule using the preset time-series classification model, determine the operation stage corresponding to different sequence intervals in the time-series data feature sequence, and send the determined operation stage corresponding to the different sequence intervals to the data determination unit. The data determination unit, connected to the operation evaluation module, is used to determine the target data corresponding to each operation stage in the entire operation process based on the operation stages corresponding to the different sequence intervals received, and to send the target data corresponding to each operation stage to the operation evaluation module.

5. The endoscopic operation evaluation system according to claim 1, characterized in that, The operation evaluation module includes: a first evaluation submodule and a second evaluation submodule; The first evaluation submodule, connected to the data analysis module and the second evaluation submodule, is used to receive the target data corresponding to each operation stage sent by the data analysis module, and based on the received target data corresponding to each operation stage and the preset evaluation rules for each operation stage, determine the operation evaluation result of each operation stage, and send the operation evaluation result of each operation stage to the second evaluation submodule. The second evaluation submodule is used to perform weighted calculation on the received operation evaluation results of each operation stage based on the weight of the operation evaluation results of each operation stage, so as to obtain the endoscopy operation evaluation result of the entire operation process.

6. The endoscopic operation evaluation system according to claim 5, characterized in that, The operation phase includes at least two of the following: a positioning phase, a focusing phase, and a holding phase; the first evaluation submodule includes at least two of the following: a positioning evaluation unit, a focusing evaluation unit, and a holding evaluation unit. The positioning evaluation unit is connected to the data analysis module and the second evaluation submodule, and is used to receive target data corresponding to each of the operation stages sent by the data analysis module; Based on the target data corresponding to the positioning stage, determine the positioning time evaluation result and the first instrument angle change evaluation result corresponding to the positioning stage; based on the positioning time evaluation result and the first instrument angle change evaluation result corresponding to the positioning stage, determine the operation evaluation result of the positioning stage; The operation evaluation results of the positioning phase are sent to the second evaluation submodule; The focusing evaluation unit is connected to the data analysis module and the second evaluation submodule, and is used to receive target data corresponding to each of the operation stages sent by the data analysis module; Based on the target data corresponding to the focusing stage, determine the target's position evaluation result, target's proportion evaluation result, target's angle evaluation result, and second instrument angle change evaluation result corresponding to the focusing stage; Based on the target's position evaluation result, target's proportion evaluation result, target's angle evaluation result, and second instrument angle change evaluation result corresponding to the focusing stage, the operation evaluation result of the focusing stage is determined; The operation evaluation results of the focusing stage are sent to the second evaluation submodule; The holding evaluation unit, connected to the data analysis module and the second evaluation submodule, is used to receive target data corresponding to each of the operation stages sent by the data analysis module; based on the target data corresponding to the holding stage, determine the jitter evaluation result, stable holding duration evaluation result, and third instrument angle change evaluation result corresponding to the holding stage; based on the jitter evaluation result, stable holding duration evaluation result, and third instrument angle change evaluation result corresponding to the holding stage, determine the operation evaluation result of the holding stage; and send the operation evaluation result of the holding stage to the second evaluation submodule.

7. The endoscopic operation evaluation system according to claim 6, characterized in that, The positioning evaluation unit includes: a time consumption evaluation subunit, a first angle change evaluation subunit, and a first evaluation subunit; The time consumption evaluation subunit is connected to the data analysis module and the first evaluation subunit. It is used to receive target data corresponding to each operation stage sent by the data analysis module; determine the time consumption of the positioning target based on the target data corresponding to the positioning stage; determine the positioning time consumption evaluation result corresponding to the positioning stage based on the positioning time consumption and a preset positioning target time consumption threshold; and send the positioning time consumption evaluation result corresponding to the positioning stage to the first evaluation subunit. The first angle change assessment subunit, connected to the data analysis module and the first assessment subunit, is used to receive target data corresponding to each of the operation stages sent by the data analysis module; determine the cumulative angle change of the instrument corresponding to the positioning stage based on the target data corresponding to the positioning stage; determine the first instrument angle change assessment result corresponding to the positioning stage based on the cumulative change of the instrument corresponding to the positioning stage and a preset first cumulative change threshold, and send the first instrument angle change assessment result corresponding to the positioning stage to the first assessment subunit. The first evaluation subunit, connected to the second evaluation submodule, is used to determine the operation evaluation result of the positioning stage based on the received positioning time evaluation result and the first instrument angle change evaluation result, as well as the weights corresponding to the positioning time evaluation result and the first instrument angle change evaluation result, and then send the operation evaluation result of the positioning stage to the second evaluation submodule.

8. The endoscopic operation evaluation system according to claim 6, characterized in that, The focus evaluation unit includes: a position evaluation subunit, a proportion evaluation subunit, an angle evaluation subunit, a second angle change evaluation subunit, and a second evaluation subunit; The position evaluation subunit, connected to the data analysis module and the second evaluation subunit, is used to receive target data corresponding to each of the operation stages sent by the data analysis module; determine the distance between the target and the center point of the image frame in each image frame of the focusing stage based on the target data corresponding to the focusing stage; determine the position evaluation result of the target corresponding to the focusing stage based on the distance between the target and the center point of the image frame in each image frame of the focusing stage and a preset maximum tolerance distance; and send the position evaluation result of the target corresponding to the focusing stage to the second evaluation subunit. The proportion evaluation subunit, connected to the data analysis module and the second evaluation subunit, is used to receive target data corresponding to each of the operation stages sent by the data analysis module; determine the area proportion of the target in each image frame of the focusing stage based on the target data corresponding to the focusing stage; determine the proportion evaluation result of the target corresponding to the focusing stage based on the area proportion of the target in each image frame of the focusing stage and a preset reference area proportion value; and send the proportion evaluation result of the target corresponding to the focusing stage to the second evaluation subunit. The angle evaluation subunit, connected to the data analysis module and the second evaluation subunit, is used to receive target data corresponding to each of the operation stages sent by the data analysis module; determine the angle of the target in each image frame of the focusing stage based on the target data corresponding to the focusing stage; determine the angle evaluation result of the target corresponding to the focusing stage based on the angle of the target in each image frame of the focusing stage, as well as a preset optimal reference angle and a preset maximum allowable angle deviation; and send the angle evaluation result of the target corresponding to the focusing stage to the second evaluation subunit. The second angle change evaluation subunit, connected to the data analysis module and the second evaluation subunit, is used to receive target data corresponding to each of the operation stages sent by the data analysis module; determine the cumulative angle change of the instrument corresponding to the focusing stage based on the target data corresponding to the focusing stage; determine the second instrument angle change evaluation result corresponding to the focusing stage based on the cumulative angle change of the instrument corresponding to the focusing stage and a preset second cumulative change threshold; and send the second instrument angle change evaluation result corresponding to the focusing stage to the second evaluation subunit. The second evaluation subunit, connected to the second evaluation submodule, is used to determine the operation evaluation result of the focusing stage based on the received position evaluation result of the target, the proportion evaluation result of the target, the angle evaluation result of the target, and the angle change evaluation result of the second instrument corresponding to the focusing stage, as well as the weights corresponding to the position evaluation result of the target, the proportion evaluation result of the target, the angle evaluation result of the target, and the angle change evaluation result of the second instrument corresponding to the focusing stage, and send the operation evaluation result of the focusing stage to the second evaluation submodule.

9. The endoscopic operation evaluation system according to claim 6, characterized in that, The stability evaluation unit includes: a jitter evaluation subunit, a stability maintenance evaluation subunit, a third angle change evaluation subunit, and a third evaluation subunit; The jitter evaluation subunit, connected to the data analysis module and the third evaluation subunit, is used to receive target data corresponding to each operation stage sent by the data analysis module; determine the displacement of the target in each image frame of the holding stage based on the target data corresponding to the holding stage; determine the jitter evaluation result corresponding to the holding stage based on the displacement of the target in each image frame of the holding stage and a preset maximum displacement threshold; and send the jitter evaluation result corresponding to the holding stage to the third evaluation subunit. The stability maintenance evaluation subunit, connected to the data analysis module and the third evaluation subunit, is used to receive target data corresponding to each operation stage sent by the data analysis module; determine the stability maintenance duration corresponding to the maintenance stage based on the target data corresponding to the maintenance stage; determine the stability maintenance duration evaluation result corresponding to the maintenance stage based on the stability maintenance duration corresponding to the maintenance stage and a preset stability duration threshold; and send the stability maintenance duration evaluation result corresponding to the maintenance stage to the third evaluation subunit. The third angle change evaluation subunit is connected to the data analysis module and the third evaluation subunit. It is used to receive target data corresponding to each operation stage sent by the data analysis module; determine the cumulative angle change of the instrument corresponding to the holding stage based on the target data corresponding to the holding stage; determine the third instrument angle change evaluation result corresponding to the holding stage based on the cumulative angle change of the instrument corresponding to the holding stage and a preset third cumulative change threshold; and send the third instrument angle change evaluation result corresponding to the holding stage to the third evaluation subunit. The third evaluation subunit, connected to the second evaluation submodule, is used to determine the operation evaluation result of the holding phase based on the received jitter evaluation result, stable holding duration evaluation result, and third instrument angle change evaluation result corresponding to the holding phase, as well as the weights corresponding to the jitter evaluation result, stable holding duration evaluation result, and third instrument angle change evaluation result corresponding to the holding phase, and to send the operation evaluation result of the holding phase to the second evaluation submodule.

Images