Intelligent evaluation system for gastrointestinal tumor minimally invasive diagnosis and treatment data based on deep learning

Abstract

The present application relates to gastrointestinal tumor minimally invasive diagnosis and treatment data intelligent evaluation technical field, specifically to gastrointestinal tumor minimally invasive diagnosis and treatment data intelligent evaluation system based on deep learning, system includes lesion perception module tracking abnormal edge and describing moving path, evolution sketch module extracts direction change to express evolution order, stage interpretation module contrasts guide feature and makes clear stage attribution, diagnosis and treatment mapping module combines operation time sequence to make clear stage and operation order, evaluation convergence module is associated with whole process data, generates gastrointestinal tumor minimally invasive diagnosis and treatment data intelligent evaluation result.The present application, through the relationship between the displacement of the lesion boundary and the direction evolution, makes the lesion change present a continuous evolution process, uses the sequential expression feature to make clear the stage direction, corresponds the image evolution and the diagnosis and treatment semantics to establish the judgment basis, associates the lesion evolution and the operation behavior on the unified time axis to form consistent mapping, thereby improves the coherent expression and the explanation of diagnosis and treatment, enhances the support value of evaluation result.

Description

Technical Field

[0001] This invention relates to the field of intelligent assessment technology for minimally invasive diagnosis and treatment data of gastrointestinal tumors, and particularly to an intelligent assessment system for minimally invasive diagnosis and treatment data of gastrointestinal tumors based on deep learning. Background Technology

[0002] The field of intelligent data assessment technology for minimally invasive diagnosis and treatment of gastrointestinal tumors includes technologies for collecting, organizing, analyzing, and evaluating multi-source medical data generated during minimally invasive diagnosis and treatment of gastrointestinal tumors. This field takes endoscopic diagnosis and treatment and minimally invasive intervention of gastrointestinal tumors as its application background. The core content involves the structured processing and correlation analysis of various types of diagnosis and treatment data, such as endoscopic image data, pathological examination data, intraoperative operation records, and postoperative follow-up data. By constructing unified models and evaluation rules for the data of the entire diagnosis and treatment process, it provides data foundation support for risk assessment, treatment plan evaluation, and process analysis in the minimally invasive diagnosis and treatment of gastrointestinal tumors.

[0003] Among them, the intelligent evaluation system for minimally invasive diagnosis and treatment data of gastrointestinal tumors based on deep learning refers to a systematic technical solution that uses deep learning models to analyze and evaluate data related to minimally invasive diagnosis and treatment of gastrointestinal tumors. It addresses technical matters such as tumor morphology annotation information, lesion location data, intraoperative operation parameters, and pathological examination results generated during the minimally invasive diagnosis and treatment of gastrointestinal tumors. It uses convolutional neural networks to extract lesion region features in endoscopic images, performs correlation analysis on multi-temporal diagnosis and treatment data based on sequence learning networks, and performs unified evaluation and classification processing on data at different stages of diagnosis and treatment through preset data evaluation rules, thereby forming a complete intelligent evaluation system for diagnosis and treatment data.

[0004] Existing technologies primarily rely on the aggregation and evaluation of multiple types of diagnostic and treatment data. Imaging data plays a more significant role in determining outcomes, lacking a continuous representation of the morphological changes in lesions. Lesion changes are often presented in stages, making it difficult to reflect the directional correlation and sequential relationship of boundary evolution. When dividing diagnostic and treatment stages, it depends on the matching of overall features, and the boundaries are unclear when facing gradual changes in lesion morphology or transitions between stages. At the same time, the records of operational behaviors are often analyzed independently of the imaging evolution process, and the expression of time correspondence is rough, which easily leads to insufficient analysis of the continuity of the diagnostic and treatment process and the transition relationship between stages. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing a deep learning-based intelligent evaluation system for minimally invasive diagnosis and treatment data of gastrointestinal tumors.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a deep learning-based intelligent evaluation system for minimally invasive diagnosis and treatment data of gastrointestinal tumors, the system comprising: The lesion perception module acquires continuous gastrointestinal images, locates the gastrointestinal mucosal region, identifies the edge of abnormal tissue, tracks the change of edge position, analyzes the movement path in adjacent images, and performs segmented description of continuous moving edges to form a continuous movement path description of the lesion boundary. The evolution characterization module extracts the direction of advancement based on each edge movement path in the continuous movement path description of the lesion boundary, compares the direction changes segment by segment, distinguishes between advancement segments with consistent direction and those with changing direction, and expresses them in the order of the images to obtain a description of the evolution sequence of the lesion boundary. The stage interpretation module analyzes the boundary change characteristics item by item based on the order of each advancement segment in the description of the lesion boundary evolution sequence, compares with the diagnosis and treatment interpretation guidelines, and combines the connection method of the advancement segment and the edge advancement pattern to determine the corresponding diagnosis and treatment stage type, gives an explanation of the diagnosis and treatment stage attribution, and forms a minimally invasive diagnosis and treatment stage interpretation result. The diagnosis and treatment mapping module, based on the diagnosis and treatment stage attribution description in the minimally invasive diagnosis and treatment stage interpretation result, obtains the timing information of the operation behavior, identifies the diagnosis and treatment stage position corresponding to the operation behavior, clarifies the order of operation behavior and the transition method within the stage, and forms a description of the correspondence between the diagnosis and treatment stage and the operation sequence.

[0007] As a further aspect of the present invention, the description of the continuous movement path of the lesion boundary includes the boundary space trajectory, continuous displacement segments, path stability identifier, and temporal continuity attribute; the description of the evolution sequence of the lesion boundary specifically includes the direction change sequence, continuous advancement segments, turning node identifier, and evolution sequence relationship; the interpretation result of the minimally invasive diagnosis and treatment stage includes the diagnosis and treatment stage type, stage interpretation label, stage sequence structure, and stage feature attribution; the description of the correspondence between the diagnosis and treatment stage and the operation sequence specifically includes the stage operation correspondence relationship, operation time sequence arrangement, stage switching association identifier, and operation connection structure.

[0008] As a further aspect of the present invention, the lesion perception module includes an image acquisition submodule, an edge extraction submodule, and a trajectory recognition submodule; The image acquisition submodule acquires the sequence of images continuously output by the gastrointestinal endoscopy device, sorts the image frames according to the time sequence number generated by the device, calls the position identification information, color channel distribution and brightness coverage area of ​​the clear area in the sorted image, removes incomplete or blurry areas in the image frame, filters the continuous image frame segments that meet the image clarity requirements, and obtains the effective continuous image frame segments. The edge extraction submodule, based on the visible area of ​​each image frame in the effective frame segment of the continuous image, calls the visual extension range, color change area and texture direction distribution of the gastrointestinal mucosal surface boundary within the image frame, and combines the closed structure features of the boundary area in the image to perform the edge marker position delineation operation, extract the boundary set that constitutes a complete closed shape in each frame, and obtain the suspicious tissue boundary area. The trajectory recognition submodule, based on the sequential arrangement of the suspected tissue boundary regions in consecutive image frames, calls the image number and displacement direction annotation of the boundary position in adjacent frames, tracks the visual path shape formed by the boundary set in consecutive frames, identifies segments with consistent extension directions in the boundary path, and divides and merges them according to continuous extension segments to obtain a description of the continuous movement path of the lesion boundary.

[0009] As a further aspect of the present invention, the evolution characterization module includes a direction extraction submodule, a fragment segmentation submodule, and a sequence representation submodule; The direction extraction submodule obtains each edge movement path in the description of the continuous movement path of the lesion boundary, extracts the direction of advancement of each path in the continuous image frame, calls the start and end annotation positions of the direction of advancement, maps the direction of advancement to the position index of each frame according to the image frame number, marks the advancement direction status of all paths in a unified format, and obtains the edge advancement direction annotation set. The segmentation submodule, based on the direction annotation content of each edge path in the edge advancement direction annotation set, calls the continuous frame advancement direction state sequence, identifies whether the direction state between adjacent frames is consistent, determines whether the frame number at the direction change position constitutes a segmentable condition, extracts the frame sequence interval corresponding to the direction consistent frame segment and the direction change frame segment, and obtains a list of continuous advancement direction segments. The sequential expression submodule calls the image sequence number and direction change annotation corresponding to each frame sequence interval in the continuous segment list of the advancement direction, performs sorting operation according to the frame sequence number, marks the start and end sequence positions of all advancement segments in the complete image sequence, integrates the sorted advancement segment set and adds direction change annotation information to obtain the lesion boundary evolution sequence description.

[0010] As a further aspect of the present invention, the stage judgment module includes a sequential parsing submodule, a feature correspondence submodule, and a stage attribution submodule; The sequential parsing submodule obtains the arrangement number of each advancing segment in the description of the evolution sequence of the lesion boundary, calls the start frame number, end frame number and advancing direction change mark of the advancing segment in the image sequence, performs sequential comparison of the connection mode between segments, identifies the combination mode of segments with jump, overlap or continuous advancing relationship, and obtains the advancing sequence connection status. The feature-corresponding submodule, based on the combined performance of each advancement segment in the advancement sequence connection, calls the edge advancement morphology direction, path turning position and advancement direction mutation number within the corresponding frame range of the segment, compares with the stage standards of lesion expansion trend and morphological change type in the diagnosis and treatment interpretation guidelines, marks the mappable stage features, and obtains the advancement stage feature matching results; The stage attribution submodule calls the items already marked in the feature matching results of the advancement stage, classifies the changes in the advancement path shown in the description of the evolution sequence of each lesion boundary, and matches the classified content with the treatment stage number item to identify the treatment stage type to which each segment belongs, and obtains the minimally invasive treatment stage interpretation result.

[0011] As a further aspect of the present invention, the diagnostic mapping module includes an operation extraction submodule, a location association submodule, and a sequence correspondence submodule; The operation extraction submodule obtains the treatment stage attribution description in the minimally invasive treatment stage interpretation result, collects the operation behavior record entries generated synchronously during the minimally invasive treatment process, extracts the execution timestamp and action category identifier corresponding to each operation behavior, sorts the operation behaviors according to the timestamp content, and obtains the operation behavior sorting sequence. The location association submodule, based on the time sequence number of each operation in the operation behavior sorting sequence, calls the image frame index number and treatment stage number corresponding to the operation behavior, performs pairing processing on the stage belonging to the operation occurrence time, classifies each operation behavior into the corresponding treatment stage, and obtains the list corresponding to the operation stage. The sequence correspondence submodule calls the stage affiliation information of all operation behaviors in the operation stage correspondence list, extracts the operation arrangement order and stage transition point content in the stage transition frame interval, performs sequential processing on the order of operation execution and the stage transition connection method within the stage, and obtains the description of the correspondence between the diagnosis and treatment stage and the operation sequence.

[0012] As a further aspect of the present invention, the system further includes: The evaluation and aggregation module, based on the sequence of operational behaviors and the operational connection method during the transition between treatment stages in the description of the corresponding treatment stages and operation sequences, and combined with the treatment stage attribution description in the minimally invasive treatment stage interpretation results, associates the lesion boundary evolution sequence, treatment stage evolution sequence, and operation behavior sequence under a unified time axis, as well as the correspondence between them and the lesion boundary evolution sequence, and expresses the treatment stage attribution description as a whole, generating intelligent evaluation results of minimally invasive treatment data for gastrointestinal tumors; The intelligent evaluation results of the minimally invasive diagnosis and treatment data for gastrointestinal tumors include boundary evolution consistency indicators, stage evolution integrity characterization, operation matching rationality evaluation, and overall diagnosis and treatment process evaluation.

[0013] As a further embodiment of the present invention, the evaluation and aggregation module includes a sequential fusion submodule, a stage comparison submodule, and a process merging submodule; The sequential fusion submodule obtains the operation unfolding order and stage connection method of each diagnosis and treatment stage in the description of the correspondence between the diagnosis and treatment stage and the operation sequence, calls the image frame number range where the operation behavior sorting sequence and the advancement segment are located, performs unified alignment processing according to the arrangement order of each number on the time axis, establishes fusion information between the operation behavior sequence and the stage evolution sequence, and obtains the sequential fusion result of the diagnosis and treatment process. The stage comparison submodule, based on the time series fusion result of the operation behavior and stage sequence in the fusion result of the diagnosis and treatment process sequence, calls the list of stage numbers already assigned in the minimally invasive diagnosis and treatment stage interpretation result, compares the stage assignment number with the time frame index in the advancement path sequence, identifies the mapping between the lesion boundary change segment and the corresponding stage, and obtains the correspondence between stage and boundary evolution. The process merging submodule calls the time distribution content of each stage number and boundary advancement segment in the corresponding relationship between the stages and boundaries. It unifies the stage attribution description, stage number sorting, operation behavior sequence and image sequence into the same sequence node, merges and integrates all nodes according to the numbering order, and obtains intelligent evaluation results of minimally invasive diagnosis and treatment data for gastrointestinal tumors.

[0014] Compared with the prior art, the advantages and positive effects of the present invention are as follows: In this invention, by continuously organizing the displacement trajectory and directional evolution of lesion boundaries in endoscopic images, the changes in lesions are presented as an evolutionary process with temporal continuity. By sequentially expressing the characteristics of boundary advancement and transition, the morphological evolution of lesions has a clear stage orientation. Furthermore, the image evolution results are correlated with the semantics of the diagnosis and treatment stages, so that the stage determination is based on the logic of boundary evolution. At the same time, the lesion evolution sequence and the relationship between the operation behavior are associated within a unified time axis, so that the diagnosis and treatment behavior and lesion changes form a consistent mapping. This improves the overall coherence and process interpretation ability of the diagnosis and treatment process, and enhances the supporting value of the diagnosis and treatment evaluation results for process control and quality analysis. Attached Figure Description

[0015] Figure 1 This is a system flowchart of the present invention; Figure 2 This is a flowchart illustrating the acquisition process of the lesion sensing module of the present invention. Figure 3 This is a flowchart illustrating the acquisition process of the evolution characterization module in this invention. Figure 4 This is a flowchart illustrating the acquisition process of the stage judgment module in this invention. Figure 5 This is a flowchart illustrating the acquisition process of the diagnostic mapping module of the present invention; Figure 6 This is a flowchart illustrating the acquisition process of the aggregation module in this invention. Detailed Implementation

[0016] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0017] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0018] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0019] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0020] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0021] Please see Figure 1 This invention provides a technical solution: a deep learning-based intelligent evaluation system for minimally invasive diagnosis and treatment data of gastrointestinal tumors, the system comprising: The lesion perception module acquires continuously output images from the gastrointestinal endoscopy equipment during minimally invasive diagnosis and treatment of gastrointestinal tumors, locates the gastrointestinal mucosal region in each image, identifies the edge position of abnormal tissue in the gastrointestinal mucosal region, continuously tracks the positional changes of the abnormal tissue edge in the continuous images in chronological order, clarifies the movement path of the abnormal tissue edge in adjacent images, and segments the edge path that presents coherent spatial movement characteristics in the continuous images to form a continuous movement path description of the lesion boundary. The evolution characterization module extracts the advancement direction information of each edge movement path in the continuous movement path description of the lesion boundary in continuous image frames. It compares the advancement direction changes of the same edge movement path in multiple image frames segment by segment, distinguishes between continuous advancement segments with consistent advancement direction and continuous advancement segments with changing advancement direction, clarifies the order of appearance of different advancement segments in the image sequence, and expresses the direction change process of each advancement segment in a sequential manner according to the image frame order, thus obtaining a description of the evolution sequence of the lesion boundary. The stage interpretation module, based on the arrangement order of each advancement segment in the description of lesion boundary evolution sequence, compares with the diagnostic and treatment interpretation guidelines used to describe lesion morphological changes, expansion trends, and invasive manifestations during minimally invasive diagnosis and treatment of gastrointestinal tumors. According to the connection mode and edge advancement pattern of the advancement segments in the image sequence, it performs a corresponding analysis on the boundary change characteristics reflected in the description of lesion boundary evolution sequence and the characteristics of different diagnosis and treatment stages in the diagnostic and treatment interpretation guidelines, clarifies the diagnosis and treatment stage type to which each segment of the lesion boundary evolution sequence description points, and provides a stage classification explanation in turn, forming the minimally invasive diagnosis and treatment stage interpretation results; The diagnosis and treatment mapping module, based on the diagnosis and treatment stage attribution description in the minimally invasive diagnosis and treatment stage interpretation results, obtains the timing information of the operation behaviors generated synchronously during the minimally invasive diagnosis and treatment process. According to the time sequence of the image and the operation behavior, it identifies the corresponding diagnosis and treatment stage position of each operation behavior in the minimally invasive diagnosis and treatment process, compares and explains the correspondence between the order of operation behavior and the attribution of diagnosis and treatment stage, clarifies the unfolding order of operation behaviors in different diagnosis and treatment stages and the connection method of operation behaviors on the time axis when the diagnosis and treatment stage changes, and forms a description of the correspondence between diagnosis and treatment stage and operation sequence. The assessment and aggregation module, based on the description of the sequence of operations in the treatment stage and the connection method of operations during the transition between treatment stages, and combined with the treatment stage attribution description in the minimally invasive treatment stage interpretation results, uniformly compares the lesion boundary evolution sequence, treatment stage evolution sequence, and operation behavior sequence in the complete minimally invasive treatment process, and correlates them on the same timeline. It clarifies the continuous connection between each treatment stage in the process, as well as the correspondence between each stage and the lesion boundary evolution sequence, and provides an overall expression of the treatment stage attribution descriptions formed throughout the treatment process, generating intelligent assessment results for minimally invasive gastrointestinal tumor treatment data.

[0022] The description of the continuous movement path of the lesion boundary includes the boundary space trajectory, continuous displacement segments, path stability identifiers, and temporal continuity attributes. The description of the evolution sequence of the lesion boundary specifically includes the sequence of directional changes, continuous advancement segments, turning node identifiers, and evolutionary order. The interpretation results of the minimally invasive diagnosis and treatment stages include the diagnosis and treatment stage type, stage interpretation label, stage sequence structure, and stage feature attribution. The description of the correspondence between the diagnosis and treatment stages and the operation sequence specifically includes the correspondence between stage operations, the arrangement of operation time sequence, the stage switching association identifier, and the operation connection structure. The intelligent evaluation results of the minimally invasive diagnosis and treatment data for gastrointestinal tumors include the boundary evolution consistency index, the completeness representation of stage evolution, the evaluation of the rationality of operation matching, and the overall diagnosis and treatment process evaluation expression.

[0023] Please see Figure 2 The lesion perception module includes an image acquisition submodule, an edge extraction submodule, and a trajectory recognition submodule; The image acquisition submodule acquires the sequence of images continuously output by the gastrointestinal endoscopy device, sorts the image frames according to the time sequence number generated by the device, calls the position identification information, color channel distribution and brightness coverage area of ​​the clear area in the sorted image, removes incomplete or blurry areas in the image frame, filters the continuous image frame segments that meet the image clarity requirements, and obtains the effective continuous image frame segments. Continuous analog signals from inside the gastrointestinal tract are acquired using a high-resolution CMOS or CCD image sensor integrated into the endoscope. These signals are then converted into a digital set of single-frame image tensors via an analog-to-digital converter (ADC) at a sampling rate of 60 frames per second. An FPGA chip reads the timestamp metadata (in milliseconds) generated by the hardware system from the header of each image tensor. The image tensors are then sorted in ascending order based on the timestamp values, generating an indexed set. ordered image sequence For each image tensor in the sequence First, convert it into a single-channel grayscale matrix, and define a size of... Given a pixel matrix, a convolution kernel based on the Laplacian operator is constructed. Convolution operations are performed on the pixel matrix to extract high-frequency texture features. The variance of the convolution output feature map is then calculated. As a rating of image sharpness, a sharpness threshold is set. The value is 100.0 (based on the 8-bit color depth standard). If the calculated value is... If the value is less than 100.0, the frame is determined to be a blurry frame caused by focus failure, and its status flag is set to 0. At the same time, the luminance channel V component of the image in the HSV color space is extracted, and the average luminance of all pixels is calculated. Set the effective brightness range as ,like If a frame falls outside this range, it is considered an exposure anomaly due to light source reflection or obstruction, and its status flag is set to 0. Next, the pixel histogram entropy values ​​for the three RGB channels are calculated. If the entropy value is lower than a preset monotonicity threshold of 1.5, it is considered an invalid image. The results of the above three logical judgments are then ANDed. Only when the sharpness, brightness, and color distribution all meet the set threshold conditions is the status flag of the frame's image tensor set to 1. If it is determined to be inconsistent or invalid, the frame is discarded and a frame loss log is recorded. For example, in processing the first... If the variance is calculated to be 85.0, which is less than the threshold of 100.0, then the frame is marked as invalid. After traversing the entire sequence, all image tensors with a state of 1 are extracted, the index mapping relationship is reconstructed, and a list of tensors after removing noise interference is formed to obtain the valid frame segments of the continuous image.

[0024] The edge extraction submodule, based on the visible area of ​​each image frame in the effective frame segment of a continuous image, calls the visual extension range, color change area and texture direction distribution of the gastrointestinal mucosal surface boundary within the image frame, and combines the closed structure features of the boundary area in the image to perform the edge marker position delineation operation, extract the boundary set that constitutes a complete closed shape in each frame, and obtain the suspicious tissue boundary area. Based on the visible regions of each image frame in a continuous effective image segment, the filtered image tensors are input to the encoder of a pre-trained deep convolutional neural network (such as a U-Net architecture based on ResNet as the backbone) deployed on a GPU cluster. The first layer of this network uses... The convolutional kernels extract features from the input image, generating a primary feature map containing 64 channels. Then, a max-pooling layer is used for downsampling to reduce dimensionality, extracting texture gradient features from the gastrointestinal mucosa surface. Dilated convolutions are used in the intermediate layers of the network to expand the receptive field, capturing large-scale color abrupt change regions, such as shifting RGB values ​​from... Mutation The region identified as a potential lesion area is then analyzed. The decoder uses bilinear interpolation upsampling to restore the feature map to the original image resolution. Deep semantic features and shallow detail features are fused through skip connections to output a probability distribution matrix of the same size as the original image. Each element The binarization threshold represents the confidence level that the pixel belongs to the edge of abnormal tissue. If the value is 0.85, elements in the matrix greater than 0.85 are set to 1, and the rest are set to 0, thus generating a binarized edge mask. For connected regions in the mask, a topological structure analysis algorithm is applied to check the 8-neighborhood connectivity of pixels. Isolated noise regions with fewer than 50 pixels (physical size approximately 0.5 mm²) are removed, and the set of pixels that can form a closed ring structure is retained. The difference between the texture entropy of the closed region and the texture entropy of the surrounding normal mucosal region is calculated. If the difference exceeds the set difference threshold of 0.3, the closed structure is confirmed as the target object. The set of pixel coordinates corresponding to the closed structure is used as the output object to obtain the suspicious tissue boundary region.

[0025] The trajectory recognition submodule, based on the sequential arrangement of suspicious tissue boundary regions in consecutive image frames, calls the image number and displacement direction annotation of the boundary position in adjacent frames, tracks the visual path shape formed by the boundary set in consecutive frames, identifies segments with consistent extension directions in the boundary path, and divides and merges them according to continuous extension segments to obtain a description of the continuous movement path of the lesion boundary. Based on the sequential arrangement of suspicious tissue boundary regions in consecutive image frames, the closed boundary region mask extracted from each frame is read, and the geometric centroid coordinates of the mask region are calculated. ,in Assign frame numbers to two adjacent frames. and Construct the displacement vector The shift vector is serialized and input into the input layer of a Long Short-Term Memory (LSTM) network, extracting data at a time step of [value missing]. Hidden layer state vector Calculate the cosine similarity between displacement vectors at adjacent time steps. To prevent division by zero errors caused by a zero modulus in a static state, a local minimum is added to the denominator. The calculation formula is as follows: ; The cosine of the angle between the motion directions at two consecutive moments is used to set a threshold for directional consistency. The calculated value is 0.9. If the value is greater than 0.9, it is determined that the lesion movement direction is consistent between the current frame and the next frame, and these two frames are grouped into the same movement segment set. For example, during frames 10 to 15, the centroid displacement vectors are respectively If the calculated pairwise similarity is above 0.95, then frames 10 to 15 are marked as the same continuous segment "advancing to the upper right". When a sudden change in the displacement vector of frame 16 is detected... When the similarity with frame 15 drops to -0.2, the current sequence is cut off at that point, a new path description segment is opened, the average direction vector of all the divided segments is extracted as the attribute label of the segment, and all segments are spliced ​​together according to the time axis to obtain the continuous movement path description of the lesion boundary.

[0026] The evolution characterization module includes a direction extraction submodule, a fragment segmentation submodule, and a sequence representation submodule; The direction extraction submodule obtains each edge movement path in the description of the continuous movement path of the lesion boundary, extracts the direction of advancement of each path in the continuous image frame, calls the start and end annotation positions of the direction of advancement, maps the direction of advancement content to the position index of each frame according to the image frame number, marks the advancement direction status of all paths in a unified format, and obtains the edge advancement direction annotation set. Obtain each edge movement path from the continuous movement path description of the lesion boundary, and parse the path set consisting of a series of discrete coordinate points from the path description data structure. Each element Corresponding to the Geometric center coordinates of the lesion edge in the frame image Set the time window size For 5 frames, for any 1st frame Frame, Selecting an interval The coordinate sequence within the interval is used to construct a local trajectory vector matrix. The least squares method is then used to linearly fit the point set within this interval. To avoid the problem of infinitely large slopes of vertical lines, the atan2 function is used to solve for the advancing angle of the fitted line. Calculate the unit direction vector of the segment based on the direction of the fitted straight line. The calculated angle values ​​are mapped to intervals. Within a degree, for example, for the coordinate sequence of frames 20 to 25. The calculated corresponding angle is 45 degrees, and the unit vector is... Iterate through the entire path set and index each frame of the image. Associate a corresponding direction vector A sequence of triplets containing frame number, coordinate values, and direction vectors is constructed. All calculated direction vectors are input into a fully connected neural network classifier. This classifier has eight pre-defined standard direction categories (e.g., north, northeast, east, etc.). The Softmax activation function outputs the probability distribution of the current frame's propagation direction belonging to each category. The category with the highest probability value is selected as the discretized direction label for that frame. If the highest probability value is lower than the confidence threshold of 0.6, it is marked as "no significant propagation" or "direction uncertain," and the label from the previous frame is used to fill in the gaps. For example, the direction vector of a certain frame... The probability of belonging to the "East" category determined by the network is 0.85, so it is labeled "East". These discretized labels are encapsulated together with the original continuous vector values ​​and stored in the tensor list according to the frame index order to obtain the edge advancement direction label set.

[0027] The segmentation submodule, based on the direction annotation content of each edge path in the edge advancement direction annotation set, calls the continuous frame advancement direction state sequence, identifies whether the direction state between adjacent frames is consistent, determines whether the frame number at the direction change position constitutes a segmentable condition, extracts the frame sequence interval corresponding to the direction consistent frame segment and the direction change frame segment, and obtains a list of continuous segments in the advancement direction. Based on the direction annotation content of each edge path in the edge propagation direction annotation set, read the direction vector of each frame. (Normalized) and its corresponding discretized label, setting the length of the sliding window for direction change detection. Given a value of 3, slide the window along the timeline and calculate the dot product of the direction vectors of two adjacent frames within the window. Since the vectors have been normalized, the dot product is the cosine of the included angle. If the dot product result is greater than the set consistency threshold... (The corresponding angle is within approximately 18 degrees), and the motion state between these two frames is determined to be stable. Continue sliding the window backward, and when a certain frame is encountered... and When the dot product is less than 0.95 and its corresponding discretized label changes (e.g., from "East" to "Southeast"), the tag frame is marked. To filter out transient jitter caused by noise and identify potential split points, a continuous verification mechanism is introduced to check the split points continuously. To determine if the frame's direction vector stably maintains its new direction state, calculate the average variance of the direction vector within 5 frames after the segmentation point. If the variance is less than 0.05, the segmentation point is considered valid; otherwise, it is treated as noise and ignored. Record the frame numbers of all confirmed valid segmentation points. Based on these segmentation points, the original complete frame sequence is divided into several sub-intervals. For example, if a sudden change in direction is detected at frame 45 and subsequent stability is achieved, then an interval is generated. and For each segmented sub-interval, the length of the frames it contains is counted. If the length is less than the minimum segment threshold of 10 frames, the short segment is merged with its preceding segment, the segmentation point set is corrected, and finally, a unique segment ID is assigned to each retained interval, and its starting frame number is recorded. With the termination frame number Get a list of consecutive segments in the direction of propulsion.

[0028] The sequential expression submodule calls the image sequence number and direction change annotation corresponding to each frame sequence interval in the continuous segment list of the advancement direction, performs sorting operation according to the frame sequence number, marks the start and end sequence positions of all advancement segments in the complete image sequence, integrates the sorted advancement segment set and adds direction change annotation information to obtain the lesion boundary evolution sequence description. By calling the image sequence number and orientation change label corresponding to each frame interval in the continuous segment list of the advancement direction, a doubly linked list data structure is constructed. Each advancement segment is inserted into the linked list as a node, and the data field of the node contains the starting frame number of the segment. Termination frame number Average direction vector and duration (Unit: Frames) Sort the linked list nodes in ascending order of their starting frame number, traverse the sorted linked list, and calculate adjacent nodes. and Directional turning angle between Using the inverse cosine function The solution is obtained, and the result is converted to radians and then to degrees. The calculated turning angle value is appended to the node connection edge. For example, the connection edge between segment A (eastward, frames 1-50) and segment B (southward, frames 51-100) is marked as "rotated 90 degrees clockwise". At the same time, a recurrent neural network (RNN) is used to encode this series of ordered segment nodes. The attribute vector of each node (including direction, duration, and turning angle) is sequentially input into the RNN unit to extract the global temporal feature vector of the entire evolution process. Finally, the information of each node in the linked list and its interrelationships are converted into a structured JSON format text description, such as "Sequence No. 1: Start frame 1, end frame 50, direction due east; Sequence No. 2: Start frame 51, end frame 100, direction due south, rotated 90 degrees clockwise relative to the previous segment". In this form, the entire evolution logic from the beginning to the end of the image is completely recorded to obtain the description of the evolution sequence of the lesion boundary.

[0029] Please see Figure 4 The stage judgment module includes a sequential parsing submodule, a feature correspondence submodule, and a stage attribution submodule; The sequential parsing submodule obtains the arrangement number of each advancing segment in the description of the evolution sequence of the lesion boundary, calls the start frame number, end frame number and advancement direction change mark of the advancing segment in the image sequence, performs sequential comparison of the connection mode between segments, identifies the combination mode of segments with jump, overlap or continuous advancement relationship, and obtains the connection status of the advancement sequence. Obtain the sequence number of each advancement segment in the description of lesion boundary evolution, parse the JSON data structure in the description file, and extract the collection of all segment objects. Each object Includes attributes Iterate through adjacent pairs of elements in the set. Calculate the starting frame of the next segment End frame of the previous segment The difference: Set a continuity threshold If it is 1, If the value equals 1, the two are considered to have a "smooth and continuous" relationship. If the value is greater than 1, it is determined to be a "jump interruption" relationship (meaning there is a missed detection or occlusion). If the value is less than or equal to 0, it is determined to be an "overlapping and redundant" relationship (possibly due to multi-threaded concurrency in the detection algorithm). For combinations of segments determined to be "smooth and continuous," the angle between their direction vectors is further calculated. ,like If the angle is less than 30 degrees, then these two segments are marked as a "combination extending in the same direction". If the angle is greater than or equal to 30 degrees, it is marked as "turning connection combination". For example, if segment A (frames 1-50) and segment B (frames 51-100) are directly connected and the directional angle is 90 degrees, it is marked as "continuous-turning". In the case of "jump interruption", the number of frames during the interruption is counted. If the length exceeds 10 frames, a placeholder node of "unknown / occluded" is inserted in the description. In the case of "overlapping redundancy", the union interval of the two segments is taken as the new valid interval, and the index reference of all subsequent segments is updated to generate a directed acyclic graph (DAG) composed of nodes and edge with attributes. The nodes represent segments and the edges represent connection types to obtain the connection status of the advancement sequence.

[0030] The feature-corresponding submodule, based on the combined performance of each advancement segment in the advancement sequence connection, calls the edge advancement morphology direction, path turning position and advancement direction mutation number within the corresponding frame range of the segment, compares with the stage standards of lesion expansion trend and morphological change type in the diagnosis and treatment interpretation guidelines, marks the mappable stage features, and obtains the advancement stage feature matching results; Based on the combined performance of each advancement segment in the advancement sequence connection, a pre-set diagnostic and treatment interpretation guide database stored in non-volatile memory is loaded. This database has been transformed into a high-dimensional feature vector space using graph embedding technology. This includes feature vector prototypes for standard stages such as "early infiltration," "lateral development," and "deep excavation." For the current combination of advancement segments to be analyzed, key feature parameters are extracted to construct a query vector. The characteristic parameters include: normalized average propulsion speed. (Calculated by dividing the path pixel length by the number of frames, unit: pixels / frame), frequency of directional changes (Number of directional changes per unit time, unit: Hz), and path curvature. These three scalar features are concatenated and input into a three-layer perceptron (MLP) network, which maps them to... Calculate the query vector in the same dimensional space. With each prototype vector in the database Euclidean distance Select the prototype with the smallest distance. As a matching object, set a matching distance threshold. If the minimum distance is 0.5, A distance less than 0.5 indicates a successful match. For example, if the distance between the feature vector calculated from a combination of segments and the prototype of "laterally developing tumor (LST)" is 0.3, then the combination of segments is labeled as "LST feature match." If the distance is greater than 0.5, it is labeled as "atypical feature." Simultaneously, an attention mechanism is used to calculate the contribution weight of each component in the query vector to the final matching result. If the frequency of directional abrupt changes... If the weight exceeds 0.7, the "high-frequency change of direction" label is added to the annotation. All fragment combination nodes are traversed, and the matched standard medical term labels are attached to the corresponding DAG node attributes to obtain the feature matching results of the advancement stage.

[0031] The stage attribution submodule calls up the items already marked in the feature matching results of the advancement stage, classifies the changes in the advancement path shown in the description of the evolution sequence of each lesion boundary, and matches the classified content with the treatment stage number item to identify the treatment stage type to which each segment belongs, and obtains the minimally invasive treatment stage interpretation results. By invoking the labeled information from the feature matching results of the advancement phase, a Long Short-Term Memory (LSTM) network classifier is constructed. The input layer of the network receives the segment sequence after feature matching and labeling. The input at each time step includes the segment ID, the matched feature label encoding (One-hot Encoding), and the corresponding confidence score. The hidden layer states of the network are also considered. Updated progressively with the sequence, incorporating contextual temporal information, the output layer is connected to a fully connected layer and a softmax activation function, corresponding to the output diagnosis and treatment stage category probability distribution. Preset diagnosis and treatment stage categories include... For each segment in the sequence ; The network outputs a probability vector belonging to each stage. Select the category with the highest probability value. As the stage to which this segment belongs, stage transition constraint logic rules are set (based on the principle of HMM transition matrix), such as "stage T2 cannot directly jump back to T1". If the network output violates this rule (such as the sequence showing T1->T2->T1), the Viterbi Algorithm is started to find the global optimal solution in the state transition path that meets the constraint conditions, correcting the abnormal stage labels, such as correcting T1->T2->T1 to T1->T2->T2. Finally, the corrected stage labels are arranged in chronological order to generate a structured description list in the form of "frame interval [1-100]: stage T1; frame interval [101-250]: stage T2", to obtain the minimally invasive diagnosis and treatment stage interpretation results.

[0032] Please see Figure 5 The diagnosis and treatment mapping module includes an operation extraction submodule, a location association submodule, and a sequence correspondence submodule; The operation extraction submodule obtains the treatment stage attribution description from the minimally invasive treatment stage interpretation results, collects operation behavior record entries generated synchronously during the minimally invasive treatment process, extracts the execution timestamp and action category identifier corresponding to each operation behavior, sorts the operation behaviors according to the timestamp content, and obtains the operation behavior sorting sequence. Retrieve the treatment stage attribution description from the minimally invasive treatment stage interpretation results. Read this description document, whose data structure is a series of tuples containing time intervals and stage labels. Simultaneously, connect to the digital communication interface of the minimally invasive surgical robot's control system or electrosurgical workstation to receive operation log data in real time in the form of a JSON stream. Parse each JSON log object and extract the value with the key name "timestamp" as the execution timestamp. Extract the string with the key "action_code" as the action category identifier. (e.g., "CUT-01" represents electrocautery cutting, "COAG-02" represents electrocoagulation hemostasis), construct a system containing... Unordered list of binary pairs To eliminate the out-of-order problem caused by network latency, the Quick Sort algorithm is used to sort the list. Reordering, using timestamps To compare key values, elements are adjusted in ascending order. For conflicting records with identical timestamps, the order is fine-tuned according to a preset action priority table (e.g., cutting > hemostasis > rinsing). For example, if "rinsing" and "hemostasis" both exist... If this occurs, "stop bleeding" will be prioritized over "rinse," and the sorted list will be traversed to check the time interval between adjacent operations. ,like If the interval is less than 50 milliseconds and the action category is the same, it is considered a duplicate record (such as key bounce). The first record is retained and subsequent duplicates are deleted, ultimately generating an index. Strictly ordered operation vector sequence Each element The complete record of the first The time and type of each effective operation are used to obtain the sorted sequence of operation behaviors.

[0033] The location association submodule, based on the time sequence number of each operation in the operation behavior sorting sequence, calls the image frame index number and treatment stage number corresponding to the operation behavior, performs pairing processing on the stage belonging to the operation occurrence time, classifies each operation behavior into the corresponding treatment stage, and obtains the list corresponding to the operation stage. Based on the time sequence number of each operation in the operation behavior sorting sequence, a time-frame index mapping function is established. Assuming the video recording is at a constant frame rate (CFR), the operation timestamp is calculated using linear interpolation. (Unit: seconds) Convert to the corresponding image frame index number ; The formula is ,in This is the start time of video recording. For frame rate (e.g., 60), this calculation is in units of... (Dimensionless index), traversal operation sequence Each item in Calculate its corresponding Then, a binary search is performed in the attribution description of the diagnosis and treatment stage to retrieve the frame index. Which stage range does it fall into? Once a matching interval is found, the corresponding treatment stage number is read. (e.g., "Stage-T2"), append the stage ID as a new attribute field to the operation object. Above, forming an extended quadruple. If the search result is empty (i.e. the operation occurred in an undefined blank area), then its stage ID is marked as "Unknown". The number of operations contained in each stage is counted, and a hash table (Hash Map) is constructed with the stage ID as the key and the list of operation objects as the value. For example, the key "Stage-T2" corresponds to a list of values. This process completes the mapping transformation from the time dimension to the semantic stage dimension and obtains the list corresponding to the operation stage.

[0034] The sequence correspondence submodule calls the stage affiliation information of all operation behaviors in the operation stage correspondence list, extracts the operation arrangement order and stage transition point content in the stage transition frame interval, performs sequential processing on the order of operation execution and the stage transition connection method within the stage, and obtains the description of the correspondence between the diagnosis and treatment stage and the operation sequence. The system retrieves the stage affiliation information of all operations in the corresponding list of operation stages, iterates through the hash table, and extracts the operation subsequences within each stage according to the logical order of stage IDs (e.g., T1->T2->T3). For each stage transition point (e.g., the moment T1 ends and T2 begins), it extracts the operation records within 5 seconds before and after that moment, constructing a "transition period operation window." The system analyzes the distribution changes of operation types within the window, calculates the rate of change (unit: percentage / second) of the proportion of "cutting" operations and the proportion of "hemostasis" operations. If the proportion of cutting operations decreases by more than 30% and the proportion of hemostasis operations increases by more than 30% at the transition point, then the transition point is marked as a "high-risk transition period." The system uses the self-attention mechanism of the Transformer model to encode the features of the operation subsequences within each stage, learns the contextual dependencies between operation behaviors, and generates semantic vectors that represent the operation patterns of that stage. These semantic vectors are concatenated with the original operation sequence text to generate a natural language description, in the format: "Stage T1 (mucosal labeling period): [labeling, [Marking, rinsing] operations; Transition point T1-T2: seamless connection; Stage T2 (circumferential incision period): [Pre-incision, hemostasis, dissection] operations were performed in sequence at an operation density of 12 times per minute. This description not only lists the operations but also clarifies the coupling logic between the operation flow and the stage flow, obtaining a description of the correspondence between the diagnosis and treatment stages and the operation sequence.

[0035] Please see Figure 6 The evaluation and aggregation module includes a sequential fusion submodule, a stage comparison submodule, and a process merging submodule; The sequential fusion submodule obtains the operation unfolding order and stage connection method of each diagnosis and treatment stage in the description of the corresponding diagnosis and treatment stage and operation sequence. It calls the image frame number range where the operation behavior sorting sequence and the advancement segment are located, performs unified alignment processing according to the arrangement order of each number on the time axis, establishes the fusion information between the operation behavior sequence and the stage evolution sequence, and obtains the sequential fusion result of the diagnosis and treatment process. Obtain the operational sequence and stage connection method of each treatment stage in the description of the correspondence between treatment stages and operation sequence, and extract the three types of time-series information from the descriptive data: (Operation sequence, including timestamps and actions) (Phase sequence, including start / end frame number and phase label) (The image frame sequence, including frame number and resolution marker) is loaded into high-performance computing memory to construct a globally unified timeline. Using the smallest time unit (1 millisecond) as the scale, the data points of the above three sequences are mapped to using an interpolation algorithm. Above, for each time point Create a state vector containing multidimensional attributes: If no operation occurs at a certain time, then Set to 0 (no operation), traverse the entire timeline, and check for any "time misalignment" anomalies, i.e., when... When a change occurs (e.g., from T1 to T2), have all critical operations from the previous stage (e.g., "mark as complete") been completed? Calculate the stage transition time. With the last critical operation moment Time difference ,like If the value is negative (meaning the stage has changed but the operation is not yet complete, usually caused by data transmission delay), then this time period is marked as a "timing conflict interval." Dynamic programming is used to correct the conflict interval by adjusting the frame index at the stage boundary. Returning to the non-negative interval, the corrected state vector sequence forms a well-aligned multimodal data stream. For example, the original state vector sequence... The phase transition that was scheduled to occur at that time has been postponed to , to wrap The final cut operation at time t will ultimately generate a sequence of length t. Fusion matrix (total duration in milliseconds) Each row of this matrix represents a full-dimensional state snapshot at a millisecond moment, obtaining the sequential fusion results of the diagnosis and treatment process.

[0036] The stage comparison submodule, based on the time series fusion results of the operation behavior and stage sequence in the fusion results of the diagnosis and treatment process sequence, calls the list of stage numbers already assigned in the minimally invasive diagnosis and treatment stage interpretation results, compares the stage assignment number with the time frame index in the advancement path sequence, identifies the mapping between the lesion boundary change segment and the corresponding stage, and obtains the correspondence between stage and boundary evolution. Based on the time-series fusion results of operational behaviors and stage sequences in the fusion results of the diagnosis and treatment process sequence, a fusion matrix is ​​extracted. In column sum (Path Direction) column, construct a list of "stage-evolution" association pairs, for each independent diagnosis and treatment stage interval (e.g. Extract all path direction vector sequences within this interval. Calculate the average directional consistency index of the sequence. ,in For a unit vector, this formula is the average composition length in cyclic statistics, and its value is in... Between, if If the value is greater than the threshold of 0.8, the lesion boundary evolution in this stage is judged as "unidirectional orderly progression"; if it is less than 0.5, it is judged as "multidirectional chaotic diffusion". At the same time, the number of directional abrupt changes in this stage is counted. Calculate mutation density (Unit: times / frame) Based on a pre-defined medical rule base, verify whether the current stage label matches the evolutionary characteristics. For example, for "Deep Mining Stage (Stage-T3)," the rule requires... It should be greater than 0.1 and Less than 0.6 (reflecting multi-directional adjustments under complex operations), if the actual calculation is and If the error message is not found, the message "Stage label does not match evolution feature" will be displayed. Calculate the matching confidence score Intervals with a confidence level below 0.6 are labeled as "suspicious stages." A Transformer-based sequence labeling model is then used to input... The sequence is predicted to have the most likely theoretical stage label. Cross-validation is performed with the original label to generate a detailed mapping table containing "original stage ID, evolution feature index, matching confidence, and suggested correction label" to obtain the correspondence between stage and boundary evolution.

[0037] The process merging submodule calls the time distribution content of each stage number and boundary advancement segment in the correspondence between stage and boundary evolution, and unifies the stage attribution description, stage number sorting, operation behavior sequence and image sequence into the same sequence node. All nodes are merged and integrated according to the numbering order to obtain intelligent evaluation results of minimally invasive diagnosis and treatment data for gastrointestinal tumors. The system retrieves the stage number and time distribution of each boundary advancement segment from the correspondence between the calling stages and boundary evolution. It initializes an empty XML structured document object, creates a root node named "Evaluation_Report," iterates through all generated mapping entries and fusion matrix data, and sequentially creates "Process_Node" child nodes in chronological order. Each child node encapsulates the following core data fields: (Time range) (Stage ID and confidence level) (A list of actions performed during this period, such as "cutting x3 -> stopping bleeding x1") (The evolutionary characteristics of the lesion boundary, such as "progressing deeper with a stable direction") (Operational quality score), where the quality score is calculated using a weighted formula: ; in These are the weights for stage matching degree, operational risk coefficient (derived inversely from the proportion of hemostasis operations, ranging from [0,1]), and smoothness, respectively. , For risk coefficient, For smooth operation; For example, a node's score is The score is dimensionless. The calculated score will be filled into the node attributes. Using the Natural Language Generation (NLG) module, this structured data will be transformed into a coherent medical assessment text, such as "In Stage-T2, the operation fluency score is 0.87, and the evolution of the lesion boundary is as expected, but an atypical directional mutation occurs at 3 minutes and 20 seconds. Review is recommended." All nodes and text will be summarized and packaged to obtain the intelligent assessment results of minimally invasive diagnosis and treatment data for gastrointestinal tumors.

[0038] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A deep learning-based intelligent assessment system for minimally invasive diagnosis and treatment data of gastrointestinal tumors, characterized in that, The system includes: The lesion perception module acquires continuous gastrointestinal images, locates the gastrointestinal mucosal region, identifies the edge of abnormal tissue, tracks the change of edge position, analyzes the movement path in adjacent images, and performs segmented description of continuous moving edges to form a continuous movement path description of the lesion boundary. The evolution characterization module extracts the direction of advancement based on each edge movement path in the continuous movement path description of the lesion boundary, compares the direction changes segment by segment, distinguishes between advancement segments with consistent direction and those with changing direction, and expresses them in the order of the images to obtain a description of the evolution sequence of the lesion boundary. The stage interpretation module analyzes the boundary change characteristics item by item based on the order of each advancement segment in the description of the lesion boundary evolution sequence, compares with the diagnosis and treatment interpretation guidelines, and combines the connection method of the advancement segment and the edge advancement pattern to determine the corresponding diagnosis and treatment stage type, gives an explanation of the diagnosis and treatment stage attribution, and forms a minimally invasive diagnosis and treatment stage interpretation result. The diagnosis and treatment mapping module, based on the diagnosis and treatment stage attribution description in the minimally invasive diagnosis and treatment stage interpretation result, obtains the timing information of the operation behavior, identifies the diagnosis and treatment stage position corresponding to the operation behavior, clarifies the order of operation behavior and the transition method within the stage, and forms a description of the correspondence between the diagnosis and treatment stage and the operation sequence.

2. The intelligent evaluation system for minimally invasive diagnosis and treatment data of gastrointestinal tumors based on deep learning according to claim 1, characterized in that: The description of the continuous movement path of the lesion boundary includes the boundary space trajectory, continuous displacement segments, path stability identifier, and temporal continuity attribute. The description of the evolution sequence of the lesion boundary specifically includes the direction change sequence, continuous advancement segments, turning node identifier, and evolution sequence relationship. The interpretation result of the minimally invasive diagnosis and treatment stage includes the diagnosis and treatment stage type, stage interpretation label, stage sequence structure, and stage feature attribution. The description of the correspondence between the diagnosis and treatment stage and the operation sequence specifically includes the stage operation correspondence relationship, operation time sequence arrangement, stage switching association identifier, and operation connection structure.

3. The intelligent evaluation system for minimally invasive diagnosis and treatment data of gastrointestinal tumors based on deep learning according to claim 1, characterized in that: The lesion perception module includes an image acquisition submodule, an edge extraction submodule, and a trajectory recognition submodule; The image acquisition submodule acquires the sequence of images continuously output by the gastrointestinal endoscopy device, sorts the image frames according to the time sequence number generated by the device, calls the position identification information, color channel distribution and brightness coverage area of ​​the clear area in the sorted image, removes incomplete or blurry areas in the image frame, filters the continuous image frame segments that meet the image clarity requirements, and obtains the effective continuous image frame segments. The edge extraction submodule, based on the visible area of ​​each image frame in the effective frame segment of the continuous image, calls the visual extension range, color change area and texture direction distribution of the gastrointestinal mucosal surface boundary within the image frame, and combines the closed structure features of the boundary area in the image to perform the edge marker position delineation operation, extract the boundary set that constitutes a complete closed shape in each frame, and obtain the suspicious tissue boundary area. The trajectory recognition submodule, based on the sequential arrangement of the suspected tissue boundary regions in consecutive image frames, calls the image number and displacement direction annotation of the boundary position in adjacent frames, tracks the visual path shape formed by the boundary set in consecutive frames, identifies segments with consistent extension directions in the boundary path, and divides and merges them according to continuous extension segments to obtain a description of the continuous movement path of the lesion boundary.

4. The intelligent evaluation system for minimally invasive diagnosis and treatment data of gastrointestinal tumors based on deep learning according to claim 1, characterized in that: The evolution characterization module includes a direction extraction submodule, a fragment segmentation submodule, and a sequence representation submodule; The direction extraction submodule obtains each edge movement path in the description of the continuous movement path of the lesion boundary, extracts the direction of advancement of each path in the continuous image frame, calls the start and end annotation positions of the direction of advancement, maps the direction of advancement to the position index of each frame according to the image frame number, marks the advancement direction status of all paths in a unified format, and obtains the edge advancement direction annotation set. The segmentation submodule, based on the direction annotation content of each edge path in the edge advancement direction annotation set, calls the continuous frame advancement direction state sequence, identifies whether the direction state between adjacent frames is consistent, determines whether the frame number at the direction change position constitutes a segmentable condition, extracts the frame sequence interval corresponding to the direction consistent frame segment and the direction change frame segment, and obtains a list of continuous advancement direction segments. The sequential expression submodule calls the image sequence number and direction change annotation corresponding to each frame sequence interval in the continuous segment list of the advancement direction, performs sorting operation according to the frame sequence number, marks the start and end sequence positions of all advancement segments in the complete image sequence, integrates the sorted advancement segment set and adds direction change annotation information to obtain the lesion boundary evolution sequence description.

5. The intelligent evaluation system for minimally invasive diagnosis and treatment data of gastrointestinal tumors based on deep learning according to claim 1, characterized in that: The stage judgment module includes a sequential parsing submodule, a feature correspondence submodule, and a stage attribution submodule; The sequential parsing submodule obtains the arrangement number of each advancing segment in the description of the evolution sequence of the lesion boundary, calls the start frame number, end frame number and advancing direction change mark of the advancing segment in the image sequence, performs sequential comparison of the connection mode between segments, identifies the combination mode of segments with jump, overlap or continuous advancing relationship, and obtains the advancing sequence connection status. The feature-corresponding submodule, based on the combined performance of each advancement segment in the advancement sequence connection, calls the edge advancement morphology direction, path turning position and advancement direction mutation number within the corresponding frame range of the segment, compares with the stage standards of lesion expansion trend and morphological change type in the diagnosis and treatment interpretation guidelines, marks the mappable stage features, and obtains the advancement stage feature matching results; The stage attribution submodule calls the items already marked in the feature matching results of the advancement stage, classifies the changes in the advancement path shown in the description of the evolution sequence of each lesion boundary, and matches the classified content with the treatment stage number item to identify the treatment stage type to which each segment belongs, and obtains the minimally invasive treatment stage interpretation result.

6. The intelligent evaluation system for minimally invasive diagnosis and treatment data of gastrointestinal tumors based on deep learning according to claim 1, characterized in that: The diagnostic mapping module includes an operation extraction submodule, a location association submodule, and a sequence correspondence submodule; The operation extraction submodule obtains the treatment stage attribution description in the minimally invasive treatment stage interpretation result, collects the operation behavior record entries generated synchronously during the minimally invasive treatment process, extracts the execution timestamp and action category identifier corresponding to each operation behavior, sorts the operation behaviors according to the timestamp content, and obtains the operation behavior sorting sequence. The location association submodule, based on the time sequence number of each operation in the operation behavior sorting sequence, calls the image frame index number and treatment stage number corresponding to the operation behavior, performs pairing processing on the stage belonging to the operation occurrence time, classifies each operation behavior into the corresponding treatment stage, and obtains the list corresponding to the operation stage. The sequence correspondence submodule calls the stage affiliation information of all operation behaviors in the operation stage correspondence list, extracts the operation arrangement order and stage transition point content in the stage transition frame interval, performs sequential processing on the order of operation execution and the stage transition connection method within the stage, and obtains the description of the correspondence between the diagnosis and treatment stage and the operation sequence.

7. The intelligent evaluation system for minimally invasive diagnosis and treatment data of gastrointestinal tumors based on deep learning according to claim 1, characterized in that: The system also includes: The evaluation and aggregation module, based on the sequence of operational behaviors and the operational connection method during the transition between treatment stages in the description of the corresponding treatment stages and operation sequences, and combined with the treatment stage attribution description in the minimally invasive treatment stage interpretation results, associates the lesion boundary evolution sequence, treatment stage evolution sequence, and operation behavior sequence under a unified time axis, as well as the correspondence between them and the lesion boundary evolution sequence, and expresses the treatment stage attribution description as a whole, generating intelligent evaluation results of minimally invasive treatment data for gastrointestinal tumors; The intelligent evaluation results of the minimally invasive diagnosis and treatment data for gastrointestinal tumors include boundary evolution consistency indicators, stage evolution integrity characterization, operation matching rationality evaluation, and overall diagnosis and treatment process evaluation.

8. The intelligent evaluation system for minimally invasive diagnosis and treatment data of gastrointestinal tumors based on deep learning according to claim 7, characterized in that: The evaluation aggregation module includes a sequential fusion submodule, a stage comparison submodule, and a process merging submodule; The sequential fusion submodule obtains the operation unfolding order and stage connection method of each diagnosis and treatment stage in the description of the correspondence between the diagnosis and treatment stage and the operation sequence, calls the image frame number range where the operation behavior sorting sequence and the advancement segment are located, performs unified alignment processing according to the arrangement order of each number on the time axis, establishes fusion information between the operation behavior sequence and the stage evolution sequence, and obtains the sequential fusion result of the diagnosis and treatment process. The stage comparison submodule, based on the time series fusion result of the operation behavior and stage sequence in the fusion result of the diagnosis and treatment process sequence, calls the list of stage numbers already assigned in the minimally invasive diagnosis and treatment stage interpretation result, compares the stage assignment number with the time frame index in the advancement path sequence, identifies the mapping between the lesion boundary change segment and the corresponding stage, and obtains the correspondence between stage and boundary evolution. The process merging submodule calls the time distribution content of each stage number and boundary advancement segment in the corresponding relationship between the stages and boundaries. It unifies the stage attribution description, stage number sorting, operation behavior sequence and image sequence into the same sequence node, merges and integrates all nodes according to the numbering order, and obtains intelligent evaluation results of minimally invasive diagnosis and treatment data for gastrointestinal tumors.

Images