Image recognition system for auxiliary warning in operation

Abstract

The invention discloses an image recognition system for auxiliary warning in an operation, which relates to the technical field of image recognition and comprises a time sequence construction module, a distortion fragment extraction module, a refresh rhythm smoothing module, a delay identifier removal module and a visual buffer regulation and control module. The time sequence construction module is used for collecting continuous picture data of the lens in the rapid movement and angle sudden change stage in the operation, synchronously recording the timestamp, the illumination intensity and the visual angle variation of each frame of picture, and establishing a time sequence basis for dynamic picture recognition. According to the method, the multi-dimensional time sequence is constructed, and the picture refreshing and delay are regulated and controlled, so that the time and the space of the intra-operative picture are kept consistent when the lens quickly moves or the angle is changed, and the judgment is prevented from being interfered by mark dislocation. And meanwhile, by matching the refresh frequency with the lens rhythm, synchronous output of the recognition prompt and the real picture is realized, so that the visual stability and the operation safety in a high-precision operation stage are improved.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and more specifically to an image recognition system for assistive warnings during surgery. Background Technology

[0002] The intraoperative image recognition system is a surgical aid tool based on artificial intelligence visual algorithms, image segmentation technology, and real-time data processing. Connecting to the video output of a laparoscopy or surgical robot, the system receives high-resolution image streams from the surgical site in real time and uses a deep learning model to automatically identify and dynamically annotate key anatomical structures, vascular branches, neural pathways, and lesion areas in the images. During the identification process, the system provides real-time warnings for potentially high-risk areas or critical tissues through color markings, edge contours, or pop-up prompts, helping surgeons receive visual cues before the procedure and reducing the occurrence of risks such as accidental cutting, accidental contact, and bleeding. Simultaneously, the system can self-learn and match scenes based on dynamic changes in the surgical environment, combining preoperative CT / MRI data for structural-level comparison, thereby providing surgeons with precise visual navigation and decision support, significantly improving surgical safety and success rates.

[0003] The existing technology has the following shortcomings: In dynamic scenarios where the camera moves rapidly or the angle changes suddenly during surgery, the image recognition system's model often experiences recognition drift due to instantaneous changes in perspective. This causes the system to fail to update image data in time, resulting in the delayed overlay of structures already recognized in the previous frame onto the new real-time image. Such delays can cause the system to generate markers or warnings at incorrect locations, creating a "misalignment warning" phenomenon. When surgeons rely on the system for visual judgment, they are highly susceptible to mistaking safe operating areas for high-risk areas, leading to hesitation, path deviation, or even accidental damage to critical structures. If this problem occurs during high-precision procedures (such as gallbladder triangle dissection or vascular ligation), it can easily lead to serious consequences such as bleeding and duct damage, affecting intraoperative safety and postoperative recovery.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide an image recognition system for assisting in surgical warnings, in order to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: an image recognition system for assistive warnings during surgery, comprising a time series construction module, a distortion segment extraction module, a refresh rhythm smoothing module, a delay marker removal module, and a visual buffer control module; Time series construction module: Collects continuous image data of the intraoperative camera during rapid movement and sudden angle changes, and records the timestamp, light intensity and angle change of each frame in sync to establish the time series basis for dynamic image recognition, which is used as a unified reference for subsequent image stability. Distortion segment extraction module: Based on the time series, it continuously calculates the viewpoint offset between adjacent frames, locates areas of sudden illumination changes and motion blur areas, and extracts distortion-sensitive segments from the time series to provide a dynamic correction basis for adaptive adjustment of the screen refresh delay; Refresh Rhythm Smoothing Module: Based on the location results of distortion-sensitive segments, differential smoothing processing is performed on the refresh rhythm of continuous image data to transform the inter-frame delay in the time series into a smooth transition sequence, so as to maintain the continuous stability of image recognition during the change of viewing angle. Delay marker removal module: Based on the dynamic characteristics of the smooth transition sequence, it performs instantaneous removal processing on the delay superposition information in continuous images, and uses a time series weight allocation mechanism to remove expired markers in order to avoid interference from old frame residues and provide real-time identification and prompts. Visual buffer control module: Combining the time sequence state after delay and clearing, it performs visual buffer control on the output process of continuous images, keeping the refresh rate synchronized with the camera movement rhythm, thereby ensuring real-time accuracy of dynamic image recognition and stable prompts.

[0007] Preferably, the process of acquiring continuous image data from the intraoperative camera during rapid movement and sudden angle changes includes the following steps: The video stream from the intraoperative camera during the surgery is continuously accessed to obtain a complete sequence of image frames. At the moment of acquisition, a time recording device is used to assign a unique timestamp to each frame to identify the specific position of the frame in the video sequence. Under the condition of obtaining continuous time recording, the illumination distribution of each frame is synchronously measured. By sampling the brightness change range of multiple areas of the image, the illumination intensity value is recorded and stored in correspondence with the timestamp. Under the premise that the timestamp and light intensity are recorded synchronously, the changes in the position of the lens in the continuous images are analyzed frame by frame, and the rotation direction and angle change range of the lens in space are recorded to obtain the change in viewpoint and store it together with the aforementioned information. The recorded results of timestamps, light intensity, and changes in viewing angle are integrated to form a multi-dimensional time-series data set containing time, brightness, and spatial information, which is used for subsequent image stability processing and recognition control.

[0008] Preferably, in the frame-by-frame analysis of lens position changes, the rotation range of the lens in the horizontal and vertical directions is determined by comparing the positional differences of fixed feature areas in adjacent frames. The obtained angle change is recorded together with the corresponding timestamp and light intensity to form temporal information reflecting the continuity of the lens motion trajectory, which is used to improve the recognition stability of the image during rapid movement and angle change stages.

[0009] Preferably, the process of continuously calculating the viewpoint shift between adjacent frames in a time series and extracting distortion-sensitive segments includes the following steps: Based on the timestamp, two adjacent frames in the time series are extracted sequentially. The relative positions of the edge shape, contour direction and fixed feature area of ​​the image are compared. The offset values ​​of the lens in the horizontal and vertical directions are recorded and stored as a viewpoint offset recording sequence. Based on the viewpoint offset recording sequence, correlation analysis is performed on the illumination distribution in continuous images. By comparing the brightness change range of adjacent frames, the location and range of the illumination change area are determined and marked. Once the location of the area with a sudden change in illumination is completed, the motion blur area caused by the camera movement is identified by comparing the edge sharpness and texture continuity of adjacent frames and marked with a timestamp. Based on the combined results of viewing angle shift recording, illumination change areas, and motion blur areas, consecutive frames with rapid viewing angle changes, large illumination changes, or motion blur areas exceeding a set range are marked as distortion-sensitive segments for reference in screen refresh delay correction.

[0010] Preferably, in the process of marking distortion-sensitive segments, the timestamp is used as an index to synchronously mark consecutive frames where the rate of change of the viewing angle exceeds the average value and the amplitude of change of illumination exceeds the set range. The key time period is determined by combining the distribution information of the motion blur region, so that the extracted distortion-sensitive segments maintain continuity in time and contain comprehensive data of the lens movement direction and illumination change trend in space, which is used for adaptive correction of screen refresh delay.

[0011] Preferably, the process of performing differential smoothing on the refresh rate of continuous image data based on the location results of the distortion-sensitive segments includes the following steps: Based on the extracted distortion-sensitive segments, the delay features of each frame in the time series are calibrated segment by segment. The inter-frame delay change recording curve is formed by comparing timestamps, and the time range of the distortion-sensitive segments is used as the boundary to mark the time period that needs to be smoothed. Under the condition of obtaining the inter-frame delay change curve, differential analysis is performed on continuous frame data to determine the direction and magnitude of refresh rate change, and rhythm optimization is performed in combination with the positioning information of distortion-sensitive segments. Based on the differential analysis results of inter-frame delay, the refresh interval of adjacent frames is proportionally coordinated, the time interval of frames with larger delay is gradually shortened, and the interval of frames with excessively fast refresh is appropriately extended. The smoothing results are integrated into a new time series state, forming a smooth transition sequence, which is then used as a unified time sequence to support the continuous and stable recognition of images.

[0012] Preferably, when coordinating the refresh interval of adjacent frames proportionally, the time interval of delayed frame segments is gradually shortened in sequence with reference to the inter-frame delay change recording curve, and the interval of frame segments with excessively fast refresh rates is extended proportionally. In the smoothing process, the viewpoint offset recording and illumination change recording are combined for synchronous adjustment, so that the smooth transition sequence maintains continuity in the time dimension and maintains a stable flow state when the screen is refreshed.

[0013] Preferably, the process of performing instantaneous clearing processing on delayed overlay information in consecutive frames based on the dynamic characteristics of the smooth transition sequence and removing expired markers using a time-series weighting mechanism includes the following steps: Under the condition of obtaining a smooth transition sequence, the time dynamic features of each frame in the sequence are identified, the time interval between frames is read by using the timestamp as an index and the delay change trend is analyzed to determine the dynamic state of the frame. Under the premise of completing the time dynamic feature recognition, the delay superposition information in the continuous picture is analyzed frame by frame. The expired mark is identified by comparing the prompt information of adjacent frames with the mark position and corresponding to the time period. Instantaneous clearing is performed according to the inter-frame order of the time series, clearing the identified expired markers in chronological order and combining the dynamic features of the smooth transition sequence to ensure that clearing and refreshing are synchronized; A weight allocation mechanism is established by combining the overall dynamic characteristics of the time series. The identification information of each frame is redistributed over time, and expired frames are removed according to the weight difference to keep the identification prompts updated in real time.

[0014] Preferably, when redistributing the identification information of each frame over time, the weight value of each frame is determined by comprehensively calculating the timestamp, refresh interval and clearing priority. When a new frame enters the time sequence, the weight of the old frame is gradually reduced according to the weight decay law. When the weight value is lower than the set threshold, the corresponding identifier is automatically removed to maintain the continuous update of the identification prompt in the time dimension and the real-time correspondence with the screen content.

[0015] Preferably, the process of performing visual buffering control on the output of consecutive images in combination with the time-series state after delay clearing includes the following steps: The time series status is read and analyzed, and the time interval, illumination change and viewing angle change rate between each frame are obtained according to the timestamp order. The temporal distribution characteristics that reflect the dynamic status of the time series are constructed. Based on the changing trend of the inter-frame time interval and the rhythm of camera movement, the buffer control range for continuous image output is defined, and the continuous frames in the time series are arranged in chronological order within the buffer range; Based on the timestamps and the rate of change of viewpoint in the time series, the output refresh rate of the continuous images is adjusted frame by frame to keep the output rhythm in line with the camera movement. Under the condition of completing the refresh rate adjustment, the output frames are prioritized by combining the weight allocation mechanism of time series in order to maintain the stability and time consistency of prompts during continuous screen output.

[0016] The technical effects and advantages provided by the present invention in the above technical solution are as follows: This invention constructs a dynamic time series with timestamps, illumination intensity, and changes in viewing angle as unified references. Based on this, it smoothly controls and delays the clearing of the screen refresh rate, ensuring that continuous images maintain temporal consistency and spatial coherence even during rapid camera movement or sudden angle changes. The markings in the image and the actual anatomical locations are updated synchronously with camera changes, effectively avoiding misalignment caused by residual information from old frames. This provides the surgeon with stable and continuous visual guidance during dynamic operations, thereby reducing judgment interference caused by screen jumps or marker drift.

[0017] This invention, based on achieving delayed overlay removal, matches and adjusts the screen output refresh rate with the camera movement rhythm, enabling recognition prompts in dynamic scenes to be presented synchronously with the actual surgical footage. This method continuously outputs stable and reliable visual prompts during high-precision operations, reducing operational hesitation and path deviations caused by screen lag or unstable prompts. This helps surgeons maintain operational continuity and improves overall surgical safety and success rates. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0019] Figure 1 This is a schematic diagram of a module of an image recognition system for assisting in surgical warnings according to the present invention. Detailed Implementation

[0020] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the description of this disclosure will be more complete and fully convey the concept of the exemplary embodiments to those skilled in the art.

[0021] This invention provides, for example Figure 1 The image recognition system for assistive warnings during surgery includes a time series construction module, a distortion segment extraction module, a refresh rate smoothing module, a delay marker removal module, and a visual buffer control module. Time series construction module: Collects continuous image data of the intraoperative camera during rapid movement and sudden angle changes, and records the timestamp, light intensity and angle change of each frame in sync to establish the time series basis for dynamic image recognition, which is used as a unified reference for subsequent image stability. The specific implementation method for this step is as follows: First, the intraoperative camera continuously captures the video stream throughout the entire surgical procedure, acquiring a complete sequence of image frames. During this process, each frame output by the camera is received in real time and assigned a unique timestamp at the moment of acquisition by a time-recording device, identifying the frame's specific position within the overall sequence. The timestamp recording is synchronized with the camera's frame rate, ensuring that each frame has a corresponding time record and preventing frame loss or timing misalignment during rapid camera rotation or sudden changes in direction. The timestamps are recorded with millisecond-level precision, allowing for the tracking of time intervals between any consecutive frames. Whenever the camera moves, the image acquisition unit continuously captures complete frame data, arranging them sequentially to form a time-uninterrupted frame sequence. Simultaneously, during acquisition, a brightness measurement device extracts the overall brightness value of each frame to reflect the lighting conditions of the surgical environment. This synchronous recording method ensures consistency in both time and brightness dimensions during image acquisition, enabling subsequent light intensity analysis and image stability assessment to rely on a precise time reference.

[0022] Secondly, based on the continuous time recording, the illumination distribution of each frame is simultaneously measured. By sampling the brightness variation range of multiple areas in the image, the illumination intensity value at the current moment is recorded. The illumination intensity value is stored in numerical form corresponding to a timestamp, forming a set of illumination change sequences. This sequence is used to reflect the brightness fluctuations caused by the surgical light's illumination angle, tissue reflection, and instrument reflection during rapid camera movement. At this stage, the acquisition process simultaneously records the overall brightness level and local brightness changes of the image. For example, when the laparoscopic lens moves from above to the side, the change in illumination angle causes the brightness in the central area of ​​the image to decrease, while the brightness in the edge area increases. These changes are recorded in real time and matched with the timestamp of the corresponding frame. In this way, each frame has a precise time stamp and illumination feature record, allowing the brightness change trajectory of the surgical scene to be continuously reflected in the time dimension, thus maintaining the temporal consistency of illumination information in scenes with frequent changes in camera angle.

[0023] After the timestamp and illumination intensity are recorded synchronously, the changes in the camera's position in the continuous images are analyzed frame by frame to obtain the changes in viewing angle. This process compares the image feature regions between adjacent frames, recording the camera's rotation direction and angular range in space, thus obtaining the camera's trajectory along the time axis. The changes in viewing angle for each frame are stored together with its corresponding timestamp and illumination intensity record, forming a three-dimensional information unit for that frame. In this way, each frame not only records the time but also reflects the illumination conditions and the camera's spatial orientation at that time. When the camera moves rapidly, the changes in viewing angle between adjacent frames increase, while when the camera is stationary, the changes remain small. This continuous numerical change accurately describes the coherence and directionality of the camera's movement. For example, when the camera moves from the bottom of the gallbladder to the porta hepatis during surgery, the recorded changes in viewing angle can fully reflect the direction and speed of the camera's movement path, forming a continuous and analyzable dynamic change sequence. In this way, each frame establishes a correspondence in the three dimensions of time, illumination, and viewing angle, providing accurate basic data for subsequent image stabilization processing.

[0024] Finally, the recorded timestamps, illumination intensity, and angle of view changes are integrated to construct a complete time series foundation. This time series uses timestamps as the primary index and illumination intensity and angle of view changes as secondary parameters, forming a multi-dimensional time-series data set. During the integration process, all frames are arranged sequentially in chronological order, maintaining a continuous transition of illumination and angle information between adjacent frames to ensure that the data in the three dimensions of time, brightness, and space remain synchronized overall. Through this integration method, the resulting time series foundation not only accurately describes the image changes during rapid movement and sudden angle changes of the surgical lens but also serves as a unified reference standard in subsequent recognition processing, providing a basis for image refresh delay correction and recognition stability control. The integrated time series foundation can be continuously invoked in subsequent steps to achieve continuous analysis of dynamic images and real-time correction of delay superposition.

[0025] Through the implementation of the above steps, the entire image acquisition process achieves synchronous recording and integration across three dimensions: time, illumination, and viewing angle. The introduction of timestamps ensures strict temporal continuity of the frame sequence, the measurement of illumination intensity ensures the temporal correspondence of brightness changes, and the extraction of viewing angle changes enables dynamic tracking in the spatial dimension. The temporal sequence foundation constructed by these three elements ensures that image acquisition remains continuous, stable, and complete even when the intraoperative camera moves rapidly or undergoes sudden angle changes. This method, by synchronously recording and temporally integrating multi-dimensional information, lays the foundation for the stability of subsequent image recognition and effectively avoids image drift and recognition delays caused by drastic camera movement or sudden illumination changes, thereby ensuring the continuity and accuracy of subsequent recognition processing.

[0026] Distortion segment extraction module: Based on the time series, it continuously calculates the viewpoint offset between adjacent frames, locates areas of sudden illumination changes and motion blur areas, and extracts distortion-sensitive segments from the time series to provide a dynamic correction basis for adaptive adjustment of the screen refresh delay; The specific implementation method for this step is as follows: Based on the established time series, the spatial correspondence between adjacent frames in a continuous sequence of images is continuously calculated. Using the timestamp as the primary index, adjacent frames in the time series are sequentially retrieved. By comparing the edge morphology, contour direction, and relative positional relationships of fixed feature regions within the image, the direction and magnitude of the camera's perspective change within that time period are determined. During this process, the positional offset of stable reference areas (such as the surgical field boundary or the area where the instrument handle is located) is compared frame by frame, recording the camera's horizontal and vertical offset values. These offset values ​​are then bound to the corresponding frame's timestamp information and stored, forming a continuous sequence of perspective offset records. This sequence reflects the continuous trajectory of perspective changes during dynamic camera movement, providing a spatial reference for subsequent illumination changes and motion blur localization. During rapid camera rotation or transitions from close-up to distant views, the continuous recording of perspective offsets can fully depict the dynamic migration process of the camera's field of view, thus maintaining data continuity in the spatial dimension.

[0027] Based on the obtained viewpoint shift recording sequence, correlation analysis is performed on the changes in illumination distribution in each frame to determine the location and range of illumination abrupt change regions. By comparing the brightness distribution of the same area in two consecutive frames point by point, areas where the brightness change exceeds a set threshold within a short period of time are identified and marked as illumination abrupt change regions. In this process, combined with the spatial localization results of the viewpoint shift sequence, it is possible to distinguish between brightness changes caused by changes in lens angle and brightness abrupt changes caused by changes in light source illumination. For example, when the lens quickly shifts from a normal viewing angle to an oblique viewing angle, the reflected light on some tissue surfaces will instantly increase, while other areas will darken. By combining the viewpoint shift information, it can be determined that this brightness change is an illumination abrupt change caused by the angle rotation rather than a change in image content. The localization of illumination abrupt change regions includes not only areas of increased brightness but also areas of decreased brightness, thus ensuring bidirectional recording of illumination changes, so that each frame in the time series can reflect the dynamic response characteristics of the image in the illumination dimension.

[0028] After locating the areas of sudden illumination changes, motion blur regions caused by rapid camera movement or abrupt angle changes in consecutive frames are identified. By comparing edge sharpness and texture continuity in adjacent frames, areas where detail contours diffuse or become blurred are detected. Motion blur regions typically appear when the camera rotation speed exceeds the frame refresh rate. These regions can be linked to corresponding time periods using timestamp information for unified analysis in the time series. Combining viewpoint shift records and the spatial location of areas of sudden illumination changes during detection effectively distinguishes blur caused by camera rotation speed from image distortion caused by sudden brightness changes. For example, when the camera moves rapidly past highly reflective materials, the edges of the image may lose some texture details due to excessive brightness. This can be cross-referenced with records of sudden illumination changes to avoid misclassifying it as motion blur. Through this continuous spatial-illumination correspondence, the true motion-induced blur regions can be accurately extracted while maintaining the continuity of this information with the time series.

[0029] Finally, based on the results of recording viewpoint shifts, identifying areas of sudden illumination changes, and recognizing motion blur areas, these data are integrated to extract distortion-sensitive segments from the time series. The extraction process uses timestamps as indexes, marking consecutive frames within the same time period that exhibit a higher-than-average rate of viewpoint change, a large amplitude of illumination change, or a motion blur area exceeding a set range as distortion-sensitive segments. Each distortion-sensitive segment is temporally continuous and spatially contains comprehensive information on the direction of lens movement, illumination change trends, and image blur distribution. The extracted distortion-sensitive segments can fully reflect the key time periods during surgical lens movement where image recognition drift or delay superposition is likely to occur. These segments are used for adaptive correction of subsequent image refresh delays, providing a reference for stable image recognition. Since each segment carries a timestamp, illumination distribution characteristics, and viewpoint shift trajectory, this information can be used in subsequent refresh control to determine which frames belong to the dynamic transition phase and which belong to the stable phase, thus providing data support for adjusting the refresh rate.

[0030] Through the above steps, multi-dimensional analysis of dynamic images can be performed based on time series data, enabling continuous calculation of viewpoint shifts between adjacent images and simultaneous location of areas of abrupt illumination changes and motion blur in both time and space. This method integrates information on changes in time, illumination, and viewpoint to establish a multi-dimensional continuous response relationship in dynamic images, allowing distortion-sensitive segments extracted from the time series to fully reflect the true process of image changes. The extraction of these segments provides sufficient data for subsequent image refresh delay correction, ensuring continuous and stable image recognition even under rapid camera movement or sudden angle changes. This avoids misalignment or delay accumulation in recognition results due to distortion accumulation, thereby improving the real-time performance and accuracy of intraoperative image recognition.

[0031] Refresh Rhythm Smoothing Module: Based on the location results of distortion-sensitive segments, differential smoothing processing is performed on the refresh rhythm of continuous image data to transform the inter-frame delay in the time series into a smooth transition sequence, so as to maintain the continuous stability of image recognition during the change of viewing angle. The specific implementation method for this step is as follows: Based on the extracted distortion-sensitive segments, the delay characteristics of each frame in the time series are calibrated segment by segment. By reading the start frame, end frame, and inter-frame time interval information of each distortion-sensitive segment, the fluctuation of the screen refresh delay within that time period is determined. In this process, using the timestamp as a reference, the delay level of each frame is compared with the adjacent frames before and after it, forming a continuous recording curve of inter-frame delay changes. This curve reflects the temporal characteristics of the delay distribution when the screen is dynamically changing, that is, the delay increases when the camera moves faster and decreases when the camera is stationary or adjusting slowly. Through this frame-by-frame comparison method, areas with uneven refresh rates in the time series can be clearly identified, and these areas are marked as periods requiring smoothing, using the time range of the distortion-sensitive segments as boundaries. In this way, the non-uniform inter-frame intervals in the time series are quantified, forming basic rhythm data that can be used for subsequent adjustments.

[0032] After obtaining the inter-frame delay variation curve, differential analysis is performed on the refresh rhythm of continuous frame data to determine the direction and magnitude of the refresh rate change. Specifically, for each frame, the time interval change trend relative to the previous frame is calculated, and the time difference of several consecutive frames is analyzed as a whole change sequence. During rapid camera movement, the time interval change often exhibits irregular fluctuations, manifesting as some frames refreshing later or earlier, resulting in abrupt changes in the playback rhythm. By analyzing these uneven intervals, it is possible to identify which frame segments have excessively fast or slow refresh rhythm deviations, and determine corresponding smoothing adjustment strategies based on the direction of delay change. Combining this analysis result with the location information of distortion-sensitive segments, the refresh rhythm can be locally optimized within a specific time range, allowing the refresh rate to transition gradually rather than abruptly. This avoids frame skipping or ghosting during high-speed perspective changes and maintains the temporal continuity between consecutive frames, providing a precise basis for subsequent smoothing processing.

[0033] Subsequently, based on the differential analysis results of inter-frame delay, smoothing processing is performed on the continuous image data to achieve a smooth transition in refresh rate. In this stage, targeting the frame intervals of the aforementioned distortion-sensitive segments, the distribution of the images in the time series is rhythmically adjusted. By proportionally coordinating the refresh intervals of adjacent frames, the time intervals of frames with larger delays are gradually shortened, while the intervals of frames with excessively fast refreshes are appropriately extended, resulting in a gradual transition in refresh rate between consecutive frames. This smoothing process spans the entire time range of the distortion-sensitive segments, allowing the image refresh rate to transition from a fluctuating state to a stable state. During this process, combining viewpoint shift recording and illumination change recording further ensures that the smoothed frame sequence maintains a visually natural continuity. When the camera angle changes, the smoothed image refresh can switch at a speed close to human visual perception, thereby reducing the impact of camera angle changes on recognition stability. Through this differential smoothing processing method, the inter-frame delay in the time series no longer exhibits abrupt changes but rather forms a continuous and smooth trend, providing a dynamic basis for temporal consistency in subsequent recognition.

[0034] After differential smoothing, the smoothing results are integrated into a new time series state, forming a smooth transition sequence. This smooth transition sequence, based on adjusted time intervals, reorders consecutive frames according to a new refresh rate, ensuring a constant temporal connection between each frame and its preceding and following frames. In this way, frames with previously abrupt delays are redistributed with refresh rates, resulting in a continuous and stable flow of the image over time. The smooth transition sequence not only corrects inter-frame temporal shifts but also improves the overall smoothness of image changes. When the camera rapidly shifts from one angle to another during surgery, the image display no longer jumps or lags but transitions continuously, ensuring the recognition model can stably analyze consecutive frames. This smooth transition sequence serves as a unified temporal baseline in subsequent recognition stages, supporting the continuity of image recognition during viewpoint changes and ensuring stability and reliability in dynamic scenes.

[0035] By continuously executing the above steps, the refresh rate of continuous image data can be effectively adjusted based on the location results of the distortion-sensitive segments, smoothing out inter-frame delays in the time series and forming a stable, smooth transition sequence. The entire process achieves end-to-end connectivity from distortion segment recognition to time series smoothing through dynamic correlation of time, illumination, and viewpoint information, ensuring continuous and stable image recognition even in scenarios with rapid camera movement, changes in illumination, or sudden angle shifts. This implementation method achieves adaptive control of the refresh rate through differential smoothing, providing a stable data foundation for subsequent delay removal and visual buffering, thereby improving the reliability and stability of intraoperative image recognition in complex dynamic environments.

[0036] Delay marker removal module: Based on the dynamic characteristics of the smooth transition sequence, it performs instantaneous removal processing on the delay superposition information in continuous images, and uses a time series weight allocation mechanism to remove expired markers in order to avoid interference from old frame residues and provide real-time identification and prompts. The specific implementation method for this step is as follows: After obtaining the smooth transition sequence, the temporal dynamic features of each frame in the sequence are identified. Using the timestamp as the primary index, the time interval distribution of each frame in the smooth transition sequence is read, and the trend of inter-frame delay changes is analyzed. During continuous frame refresh, there may be slight differences in delay between different frames. If these delay differences are not processed, they can easily cause time misalignment of prompts during the image overlay stage. By analyzing the dynamic features of the smooth transition sequence, the relative position and refresh weight of each frame in the time series can be determined. For example, during the phase of rapid changes in camera angle, the inter-frame time interval shortens, while during the stable phase, the inter-frame interval tends to be balanced. Through this continuous time analysis method, it is possible to accurately determine which frames are in the dynamic transition state and which frames are in the stable state, thus providing a reference range for the subsequent removal of delay overlay information. This step establishes a frame-level temporal dynamic identification mechanism, enabling subsequent removal operations to be based on a specific time range, ensuring that the processing of each frame corresponds to its actual dynamic state in the time series.

[0037] After completing the temporal dynamic feature recognition, frame-by-frame analysis and location of potentially delayed overlay information in continuous images are performed. By comparing the prompts, markers, and outline positions of adjacent frames in a smooth transition sequence, prompts that appeared in the previous frame but are still retained in the current frame are identified. These prompts are usually expired markers, originating from the previous frame's recognition results not being replaced in time during refresh, thus being incorrectly overlaid onto the current frame. At this stage, by comparing the overlapping areas of markers between the current and previous frames, the range of delayed overlay information can be determined, and it can be mapped to a specific time period using a timestamp as a reference. For example, when the camera rapidly shifts from the bottom of the gallbladder to the porta hepatis during surgery, if the vascular direction markers in the previous frame are not cleared in time, they will be mistakenly overlaid onto the new frame position. By continuously comparing the changes in marker positions and image content between frames, the image areas where these expired markers are located can be accurately identified, providing a clear target for subsequent removal operations, thereby avoiding the remnants of old frame information in new images.

[0038] After the delayed overlay information is located, an instantaneous clearing process is performed based on the frame order and refresh rhythm of the time series. During this process, the expired identifier information is cleared sequentially according to the time sequence of the frames, with the time series as the main axis. The clearing process is dynamically adjusted based on the inter-frame time interval and weight differences: frames with longer delays have higher clearing priority; frames with shorter refresh intervals have relatively lower clearing priority. This time-series-based clearing strategy enables the immediate removal of delayed overlay information, preventing expired identifiers from causing visual interference in new frames. Simultaneously, combining the dynamic characteristics of a smooth transition sequence during the clearing process ensures that the clearing operation is synchronized with the screen refresh, without affecting the continuity of the continuous display. For example, when the camera angle continuously changes, the clearing process can remove old identifiers before the current frame refresh is complete, ensuring that only the prompt information corresponding to the content of that frame is retained when the next frame is displayed, thus achieving time consistency between the screen content and the prompt information.

[0039] After the instantaneous clearing process is completed, a weight allocation mechanism is established based on the overall dynamic characteristics of the time series to redistribute and manage the timeliness of the identification information in continuous frames. By comprehensively calculating the timestamps, refresh intervals, and clearing priorities of each frame, the weight value of each frame in the time series is determined. Frames with higher weight values ​​are the main frames within the current time window, and their identification information will be retained first; frames with lower weight values ​​are determined to be expired frames, and their residual identification information will be completely removed. Through this weight allocation mechanism, the existence period of the frame identification in the time series can be dynamically adjusted, ensuring that each frame retains only the prompt information corresponding to its refresh time. When a new frame enters the time series, the weight value of the old frame will gradually decrease over time and eventually be removed from the recognition range, thus ensuring that the entire frame recognition result remains updated in real time. In this process, combined with the refresh rhythm of the smooth transition sequence, the time weight and frame update can be synchronized, ensuring consistency between clearing and retention operations in the time dimension, further guaranteeing the accuracy and real-time nature of the recognition prompts.

[0040] Through the above steps, delayed and superimposed information can be instantly cleared and weighted based on a smooth transition sequence. The entire process, through analysis of inter-frame delays in the time series, location of expired markers, execution of instantaneous clearing, and dynamic control of weight allocation, ensures that the prompts in continuous images remain synchronized with the actual image content in time. This implementation effectively solves the problem of misaligned prompts caused by residual old frame markers, making the image recognition results more consistent with the actual intraoperative scene and avoiding visual misleading caused by information delays or overlaps. Through this clearing and weight allocation process, image refresh and recognition prompts achieve temporal consistency and spatial coherence, providing a high-precision time series basis for subsequent visual buffering adjustments, thereby further ensuring the real-time performance and stability of recognition prompts during surgery.

[0041] Visual buffer control module: Combining the time sequence state after delay and clearing, it performs visual buffer control on the output process of continuous images, keeping the refresh rate synchronized with the camera movement rhythm, thereby ensuring real-time accuracy of dynamic image recognition and stable prompts; The specific implementation method for this step is as follows: After delay removal, the obtained time series state is read and analyzed to identify the temporal balance and inter-frame stability of the current sequence. The removed frame sequence is read sequentially by timestamp, and the time interval, illumination variation, and viewing angle change rate between each frame and its adjacent frames are analyzed to construct a temporal distribution feature table reflecting the overall dynamic state of the time series. This feature table displays the refresh stability and rhythmic change trend of the image over time. For example, when the camera rapidly changes direction during surgery, the variation in inter-frame intervals in the time series increases, while during periods of stable camera movement, the inter-frame intervals tend to be consistent. This continuous analysis method determines the refresh state at different stages of the time series, providing a basis for subsequent visual buffering adjustments. The analysis results at this stage can clearly identify changes in the image rhythm during camera acceleration, deceleration, or stillness, thus enabling targeted adjustments during the output stage.

[0042] After obtaining the dynamic state information of the time series, a buffer control range is established for the continuous image output process. Using the time series as the core reference, the time boundaries between visual buffers are defined according to the changing trends of inter-frame time intervals and camera movement rhythm. This buffer is used to store several consecutive frames of data before image output to prevent output discontinuity caused by frame unevenness due to camera movement. During this process, consecutive frames after delay removal in the time series are arranged sequentially according to time, allowing continuous reading within the buffer for the output process. In this way, image output no longer directly depends on real-time captured frames, but is dynamically scheduled based on the time series state within the buffer, thus maintaining output continuity even during rapid camera movement. When camera movement accelerates, the output rhythm of the buffer automatically adapts to a higher refresh rate; when camera movement decelerates, the output rhythm smoothly transitions to a stable frequency. This time buffering mechanism effectively suppresses instantaneous jitter in the image during sudden changes in perspective, ensuring visual continuity of the output.

[0043] After establishing the buffer zone, the output refresh rate is synchronously adjusted according to the camera movement rhythm. In this stage, the output frequency is adjusted frame-by-frame using the timestamp of the time series and the rate of change of viewpoint as core parameters, ensuring the output rhythm matches the camera movement. When the camera is in a rapid panning or rotating phase, the output frequency of consecutive frames is increased accordingly to ensure that each frame covers the complete changes in the camera movement; when the camera movement gradually slows down or stops, the output frequency decreases to reduce the latency caused by repeated display of images. This synchronous adjustment based on time series and camera rhythm ensures that the image refresh rate always matches the camera movement speed in any surgical scenario, keeping the content seen by the surgeon synchronized with the actual surgical scene. Simultaneously, the process also smoothly controls the output interval, making the time transition between consecutive frames natural, thus avoiding screen flickering or stuttering. The entire refresh rate adjustment process forms a closed loop in the time dimension, enabling the output image to have adaptive dynamic rhythm adjustment capabilities.

[0044] After completing the output refresh synchronization control, visual buffer optimization is performed on the entire screen output process to ensure the stability of screen recognition prompts. In this stage, the temporal distribution of output frames is compared in real time with the frame sequence state within the buffer, and the priority of output frames is dynamically adjusted according to the time series weight allocation mechanism. For frames with higher weight in the time series, i.e., key frames within the current time window, their output priority is increased to ensure that the recognition results of these frames can be displayed immediately; for frames with lower weight, they are processed through delayed output or merged display to reduce screen content redundancy. Through this time-weighted output scheduling mechanism, the real-time accuracy of the output content can be further improved while maintaining refresh continuity. In the event of rapid camera movement or sudden angle changes, the buffer can automatically update the sorting of output frames through time weighting, ensuring that the screen always presents content consistent with the current surgical perspective. The final visual buffer control state maintains the continuity of output and the stability of prompts in both time and space dimensions, providing the surgeon with continuous and consistent visual feedback during the operation.

[0045] By implementing the above sequential steps, the output process of continuous images can be effectively visually buffered and controlled by combining the time-series state after delay clearing, and the refresh rate of the image can be kept synchronized with the camera movement rhythm. The entire process achieves adaptive adjustment of the output rhythm in the time dimension and ensures synchronous changes in the image and camera perspective in the spatial dimension, thereby maintaining real-time accuracy and stable output of recognition prompts in dynamic surgical scenarios. This implementation method, through the synergistic effect of time-series analysis, buffer construction, refresh synchronization control, and weighted output optimization, eliminates the problem of image discontinuity caused by uneven camera movement or accumulated delays, ensuring that the image recognition results remain consistent with the actual visual scene throughout the entire surgical process, providing a stable technical foundation for the safety and reliability of visual aids during surgery.

[0046] Beneficial effect 1: This invention constructs a dynamic time series with timestamps, illumination intensity, and changes in viewing angle as unified references. Based on this, it smoothly controls and delays the clearing of the screen refresh rate, ensuring that continuous images maintain temporal consistency and spatial coherence even during rapid camera movement or sudden angle changes. The markings in the image and the actual anatomical locations are updated synchronously with camera changes, effectively avoiding misalignment caused by residual information from old frames. This provides the surgeon with stable and continuous visual guidance during dynamic operations, thereby reducing judgment interference caused by screen jumps or marker drift.

[0047] Benefit 2: This invention, based on achieving delayed overlay removal, matches and adjusts the screen output refresh rate with the camera movement rhythm, enabling recognition prompts in dynamic scenes to be presented synchronously with the actual surgical footage. This method continuously outputs stable and reliable visual prompts during high-precision operations, reducing operational hesitation and path deviations caused by screen lag or unstable prompts. This helps surgeons maintain operational continuity and improves overall surgical safety and success rates.

[0048] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. An image recognition system for assisting in surgical warnings, characterized in that, It includes a time series construction module, a distortion segment extraction module, a refresh rate smoothing module, a delay marker removal module, and a visual buffer control module; Time series construction module: Collects continuous image data of the intraoperative camera during rapid movement and sudden angle changes, and records the timestamp, light intensity and angle change of each frame in sync to establish the time series basis for dynamic image recognition; Distortion segment extraction module: Based on the time series, it continuously calculates the viewpoint offset between adjacent frames, locates areas of sudden illumination changes and motion blur areas, extracts distortion-sensitive segments from the time series, and provides a dynamic correction basis for adaptive adjustment of screen refresh delay; Refresh Rhythm Smoothing Module: Based on the location results of distortion-sensitive segments, differential smoothing processing is performed on the refresh rhythm of continuous frame data to transform the inter-frame delay in the time series into a smooth transition sequence. Delay flag removal module: Based on the dynamic characteristics of the smooth transition sequence, it performs instantaneous removal processing on the delay superposition information in continuous frames, and uses a time series weight allocation mechanism to remove expired flags; Visual buffer control module: Based on the time sequence state after delay clearing, it performs visual buffer control on the output process of continuous images to keep the refresh rate synchronized with the camera movement rhythm.

2. The image recognition system for intraoperative auxiliary warning according to claim 1, characterized in that, The process of acquiring continuous image data from the intraoperative camera during rapid movement and sudden angle changes includes the following steps: The video stream from the intraoperative camera during the surgery is continuously accessed to acquire a complete sequence of image frames. At the moment of acquisition, a time recording device is used to assign a unique timestamp to each frame to identify the specific position of the frame in the video sequence. Under the condition of obtaining continuous time recording, the illumination distribution of each frame is synchronously measured. By sampling the brightness change range of multiple areas of the image, the illumination intensity value is recorded and stored in correspondence with the timestamp. With the timestamp and light intensity recorded synchronously, the changes in the position of the camera in the continuous images are analyzed frame by frame, and the rotation direction and angle change range of the camera in space are recorded. The recording results of timestamps, light intensity, and changes in viewing angle are integrated to form a multi-dimensional time-series data set containing time, brightness, and spatial information.

3. The image recognition system for intraoperative auxiliary warning according to claim 2, characterized in that, In the frame-by-frame analysis of changes in camera position, the range of camera rotation in the horizontal and vertical directions is determined by comparing the positional differences of fixed feature regions in adjacent frames. The resulting changes in viewing angle are recorded together with the corresponding timestamps and illumination intensity to form temporal information reflecting the continuity of the camera's motion trajectory.

4. The image recognition system for intraoperative auxiliary warning according to claim 2, characterized in that, The process of continuously calculating the viewpoint shift between adjacent frames in a time series and extracting distortion-sensitive segments includes the following steps: Based on the timestamp, two adjacent frames in the time series are extracted sequentially. The relative positions of the edge shape, contour direction and fixed feature area of ​​the image are compared. The offset values ​​of the lens in the horizontal and vertical directions are recorded and stored as a viewpoint offset recording sequence. Based on the viewpoint offset recording sequence, correlation analysis is performed on the illumination distribution in continuous images. By comparing the brightness change range of adjacent frames, the location and range of the illumination change area are determined and marked. Once the location of the area with a sudden change in illumination is completed, the motion blur area caused by the camera movement is identified by comparing the edge sharpness and texture continuity of adjacent frames and marked with a timestamp. By combining the results of viewing angle shift recording, illumination change areas, and motion blur areas, consecutive frames with fast viewing angle change rates, large illumination change amplitudes, or motion blur areas exceeding the set range are marked as distortion-sensitive segments for reference in screen refresh delay correction.

5. The image recognition system for intraoperative auxiliary warning according to claim 4, characterized in that, During the marking of distortion-sensitive segments, the timestamp is used as an index to synchronously mark consecutive frames where the rate of change of view exceeds the average value and the amplitude of change of illumination exceeds the set range. The key time period is determined by combining the distribution information of motion blur region, so that the extracted distortion-sensitive segments maintain continuity in time and contain comprehensive data of lens movement direction and illumination change trend in space.

6. The image recognition system for intraoperative auxiliary warning according to claim 4, characterized in that, The process of performing differential smoothing on the refresh rate of continuous image data based on the location results of distortion-sensitive segments includes the following steps: Based on the extracted distortion-sensitive segments, the delay features of each frame in the time series are calibrated segment by segment. The inter-frame delay change recording curve is formed by comparing timestamps, and the time range of the distortion-sensitive segments is used as the boundary to mark the time period that needs to be smoothed. Under the condition of obtaining the inter-frame delay change curve, differential analysis is performed on continuous frame data to determine the direction and magnitude of refresh rate change, and rhythm optimization is performed in combination with the positioning information of distortion-sensitive segments. Based on the differential analysis results of inter-frame delay, the refresh interval of adjacent frames is proportionally coordinated, gradually shortening the time interval of frames with large delay and appropriately extending the time interval of frames with fast refresh. The smoothing results are integrated into a new time series state, forming a smooth transition sequence, which is then used as a unified time series.

7. The image recognition system for intraoperative auxiliary warning according to claim 6, characterized in that, When coordinating the refresh interval of adjacent frames proportionally, the time interval of delayed frames is gradually shortened in sequence, and the interval of frames with faster refresh rates is extended proportionally, with the adjustment made synchronously in conjunction with the viewpoint offset record and the illumination change record during the smoothing process.

8. The image recognition system for intraoperative auxiliary warning according to claim 6, characterized in that, The process of instantaneously clearing delayed overlay information in consecutive frames based on the dynamic characteristics of the smooth transition sequence and removing expired markers using a time-series weighting mechanism includes the following steps: Under the condition of obtaining a smooth transition sequence, the time dynamic features of each frame in the sequence are identified, the time interval between frames is read by using the timestamp as an index and the delay change trend is analyzed to determine the dynamic state of the frame. Under the premise of completing the time dynamic feature recognition, the delay superposition information in the continuous picture is analyzed frame by frame. The expired mark is identified by comparing the prompt information of adjacent frames with the mark position and corresponding to the time period. Instantaneous clearing is performed according to the inter-frame order of the time series, clearing the identified expired markers in chronological order and combining the dynamic features of the smooth transition sequence to ensure that clearing and refreshing are synchronized; A weight allocation mechanism is established by combining the overall dynamic characteristics of the time series. The identification information of each frame is redistributed over time, and expired frames are removed according to the weight difference to keep the identification prompts updated in real time.

9. The image recognition system for intraoperative auxiliary warning according to claim 8, characterized in that, When redistributing the identification information of each frame over time, the weight value of each frame is determined by comprehensively calculating the timestamp, refresh interval and clearing priority. When a new frame enters the time sequence, the weight of the old frame is gradually reduced according to the weight decay law. When the weight value is lower than the set threshold, the corresponding identifier is automatically removed.

10. The image recognition system for intraoperative auxiliary warning according to claim 8, characterized in that, The process of performing visual buffering control on the output of consecutive images based on the time-series state after delay clearing includes the following steps: The time series status is read and analyzed, and the time interval, illumination change and viewing angle change rate between each frame are obtained according to the timestamp order. The temporal distribution characteristics that reflect the dynamic status of the time series are constructed. Based on the changing trends of the inter-frame time interval and the rhythm of camera movement, the buffer control range for continuous image output is defined, and the continuous frames in the time series are arranged in chronological order within the buffer range; Based on the timestamps and the rate of change of viewpoint in the time series, the output refresh rate of continuous images is adjusted frame by frame; Under the condition of completing the refresh rate regulation, the priority of the output frames is adjusted by combining the weight allocation mechanism of the time series.

Images