Oral cavity image real-time defogging method based on dynamic light flow sensing

Abstract

The invention discloses an oral cavity image real-time defogging method based on dynamic light flow sensing, which belongs to the technical field of image processing, and comprises the steps of constructing a multi-modal time sequence input feature tensor, performing defogging modeling and reasoning by a Fog-Lite model, and performing visual display and identification assistance by an output module. The technical problem of accurately identifying the fog area in the oral cavity or airway image by combining light stream information and reflective point track guidance is solved. The method has high reasoning efficiency, and can be applied to real-time image processing scenes in actual oral diagnosis and treatment, anesthesia video laryngoscope operation or fiberoptic bronchoscope operation. Effective modeling of dynamic changes in continuous images is achieved, the method is particularly suitable for dynamic interference caused by movement, intubation and other actions of a laryngoscope lens or a fiber bronchoscope, and the defogging precision is improved. And false detection is effectively reduced. The visual identification degree and the clinical identification auxiliary effect of an oral cavity endoscope, an anesthesia video laryngoscope or a fiber bronchoscope image are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to a real-time defogging method for oral images based on dynamic optical flow perception. Background Art

[0002] Oral imaging technology is widely used in the diagnosis of oral diseases and surgical navigation. Intraoral cameras, as the main imaging device, usually work in a narrow, humid, and highly reflective oral environment. Similarly, in airway management, anesthesia video laryngoscopes or fiber bronchoscopes also need to provide a clear field of view in a similar narrow, humid airway environment with secretions (such as saliva and blood), and face dynamic changes during the operation (such as the movement of the laryngoscope blade or fiber bronchoscope, and tracheal intubation). However, in actual applications, the images obtained by these devices (such as intraoral cameras, anesthesia video laryngoscopes or fiber bronchoscopes) are easily affected by fogging, water vapor condensation, and reflections, resulting in a decrease in image quality. In order to improve image clarity, existing technologies mostly use traditional image enhancement methods (such as histogram equalization, Retinex, guided filtering, etc.) or static image defogging networks based on deep learning.

[0003] However, the existing methods have the following main problems:

[0004] Most methods process single-frame images, ignoring the dynamic changes between consecutive frames. This makes it difficult to capture the true motion trend in the image, resulting in insufficient recovery of dynamic fog areas. This is particularly evident in anesthesia video laryngoscope or fiber bronchoscope applications where dynamic operations are frequent.

[0005] Traditional attention mechanisms cannot accurately model local reflective points and abnormal lighting areas, and are prone to misjudgment.

[0006] Some dehazing methods based on deep neural networks have large models and are difficult to deploy in resource-constrained oral or airway diagnostic and treatment equipment (such as portable oral endoscopes, anesthesia video laryngoscopes, or fiber bronchoscope workstations), making real-time processing difficult to achieve. Summary of the Invention

[0007] The purpose of the present invention is to provide a real-time defogging method for oral images based on dynamic optical flow perception, which solves the technical problem of accurately identifying foggy areas in oral or airway images by combining optical flow information with reflection point trajectory guidance during dynamic imaging.

[0008] To achieve the above object, the present invention adopts the following technical solutions:

[0009] A real-time defogging method for oral images based on dynamic optical flow perception includes the following steps:

[0010] Step 1: The image acquisition module continuously acquires three frames of oral images, packages these three frames, and sends them to the image processing unit. The image processing unit uses a lightweight optical flow network to estimate the motion vector between frames and extracts the trajectory of the reflective points by detecting the brightness stable region. At the same time, the three frames, the optical flow map, and the reflective guidance map are spliced ​​together to obtain the input feature tensor X.

[0011] Step 2: Establish a model module in the image processing unit and construct a Fog-Lite model to perform temporal modeling and guided image defogging on the input feature tensor; the Fog-Lite model integrates temporal modeling, optical flow information, and reflective point guided attention mechanism to accurately segment and enhance the fog area in the image, and finally generates and outputs the fog area mask M f and clear image I * The Fog-Lite model is specifically shown in the following formula:

[0012] M f ,I * =F Fog-Lite (X);

[0013] F Fog-Lite (X) = D°A°E°T(X);

[0014] T(X) represents the temporal feature embedding module, which is used to perform inter-frame modeling on the input feature tensor X. It mainly converts the image into a token sequence suitable for the Transformer through image patch partitioning and vectorized mapping methods. Its core is the image patch partitioning and linear mapping operations, the purpose of which is to convert the image tensor into a token sequence for processing by the Transformer model.

[0015] E represents the encoding module, which is a multi-layer attention structure based on the Transformer architecture and is used to perform context modeling and feature extraction on the token sequence; A represents the guided attention mechanism, which specifically includes a dynamic position bias method constructed using the trajectory of reflective points to guide the multi-head attention mechanism to focus on abnormal lighting areas and foggy areas; D represents the decoding module, which consists of a fog mask decoder and an image reconstruction module to generate fog mask images M respectively. f and clear image I * ;° represents the composite operation of functions, specifically applying multiple functions in sequence;

[0016] Step 3: Create an output module in the image processing unit, which calls the fog mask image M output by the Fog-Lite model. f and clear image I *, and highlight the boundary of the fog area, while visually analyzing the movement trajectory of the reflective points and the fog area to form a dynamic recognition auxiliary interface and display it on the display screen.

[0017] Preferably, the image acquisition module is an intraoral camera, an anesthesia video laryngoscope or an optical imaging probe of a fiber bronchoscope; and the image processing unit is an industrial computer.

[0018] Preferably, when executing step 1, the following steps are specifically included:

[0019] Step 1-1: The image acquisition module continuously acquires three frames of oral cavity images, which are represented as I t -1,I t ,I t+1 , where t represents the frame number and I is the image frame;

[0020] Normalize the three frames of images to generate a unified preset input size H×W, where H is the image height and W is the image width;

[0021] Stitching to form a time channel image: X c ∈R H×W×9 ; Among them, X c Time stitching image, R represents a set of real numbers;

[0022] Step 1-2: Use the lightweight optical flow model, namely the PWCNet model, to calculate the optical flow map F between the current frames t , that is, the motion vector between frames, detects the trajectory of the reflective point P according to the change of the high brightness area t ;

[0023] F t =LightFlow(I t-1 ,I t );

[0024] P t =DetectReflectedPoints(I t );

[0025] S t =Trajectory(P t-1 ,P t ,P t+1 );

[0026] Among them, F t is the optical flow map, which represents the pixel motion vector map from the t-1th frame to the tth frame, and contains two channels: horizontal and vertical;

[0027] P t is the coordinate set of the reflection points detected in the t-th frame; S tis the spatial trajectory path formed by the reflection point in the three frames of images;

[0028] According to S t Constructing reflective guide map G t , specifically, converting the position of the reflection point into an image mask;

[0029] Step 1-3: Splice the temporal image X c , optical flow graph F t , reflective guide map G t Combined into the input feature tensor X∈R H ×W×12 :

[0030] X=Concat(X c ,F t ,G t )∈R H×W×12 ;

[0031] Among them, Concat(·) represents splicing, and the optical flow map F t It represents the pixel motion vector graph from the t-1th frame to the tth frame, which contains two channels, representing the horizontal and vertical displacement of each pixel respectively. t [:,:,0] is the horizontal optical flow channel, F t [:,:,1] is the vertical optical flow channel, which is a tensor F t ∈R H×W×2 .

[0032] Preferably, when executing step 2, the following steps are specifically included:

[0033] Step 2-1: Divide the input feature tensor X into multiple small blocks, namely patches;

[0034] Then map each small patch into a token vector of fixed dimension and construct a token sequence:

[0035] T = Embed(X);

[0036] Where T is the Token sequence, which serves as the input of the Transformer encoder; Embed(X) represents the image embedding function, including patch partitioning and linear mapping algorithms;

[0037] Input the token sequence T into the Transformer encoding module E to obtain the context-enhanced token representation:

[0038] T′=E(T);

[0039] Where E(T) represents the Transformer encoding module, which includes multiple layers of multi-head attention and feedforward networks; T′ is the token sequence after context enhancement;

[0040] Step 2-2: Follow the reflective guide G t Construct dynamic position bias matrix B pt , guiding the Transformer encoding module to focus on the local fog area:

[0041]

[0042] Among them, Q, K, and V are respectively the Query matrix, the Key matrix, and the Value matrix, all of which are generated by T′ mapping; * indicates transposition; d k Indicates the dimension of the Key vector, used for normalization;

[0043] Attention(·) outputs the guided feature map, denoted as TA;

[0044] Step 2-3: Decode the attention feature map into the fog area mask map M through the MLP decoder f , combined with the original image to generate the final clear image I * ;

[0045] D={D mask (TA),D image (TA)};

[0046] M f =D mask (TA);

[0047] I * =D image (TA,I t );

[0048] Among them, D mask is a mask decoder used to generate fog area mask; D image It is an image restoration module used to restore the features to the original image I t Fusion generates clear images;

[0049] D mask and D image Constitutes the decoding module D.

[0050] Preferably, when executing step 3, the following steps are specifically included:

[0051] Step 3-1: Generate the fog mask M f and clear image I * Perform fusion display, generate visualization results, and display them on the display screen;

[0052] Step 3-2: Transform the reflection point trajectory P in the continuous frames t Dynamically superimposed on the output image, it assists in judging the dynamic behavior of fog interference.

[0053] The present invention discloses a real-time defogging method for oral images based on dynamic optical flow perception, which solves the technical problem of accurately identifying foggy areas in oral or airway (such as larynx) images by combining optical flow information with the guidance of reflective point trajectories during dynamic imaging. The present invention adopts lightweight optical flow estimation and Transformer structure, has high inference efficiency, and can be applied to real-time image processing scenarios in actual oral diagnosis and treatment, anesthesia video laryngoscope or fiber bronchoscope operations. Through three-frame splicing and optical flow estimation, effective modeling of dynamic changes in continuous images is achieved, especially adapting to dynamic interference caused by the movement of laryngoscope blades or fiber bronchoscopes, intubation and other actions, thereby improving the defogging accuracy. Dynamic position bias is constructed using reflective point trajectories to guide the attention mechanism to focus on abnormal lighting and fogged areas, effectively reducing false detections. The final output image is significantly superior to existing methods in terms of contrast, texture details and visible edges, significantly improving the visual recognition and clinical recognition auxiliary effect of oral endoscopes and anesthesia video laryngoscope or fiber bronchoscope images. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is the main flow chart of the present invention;

[0055] Figure 2 It is a sub-flow chart of step 1 of the present invention;

[0056] Figure 3 It is a sub-flowchart of step 2 of the present invention. DETAILED DESCRIPTION

[0057] Depend on Figure 1-Figure 3 The method for real-time defogging of oral images based on dynamic optical flow perception includes the following steps:

[0058] Step 1: The image acquisition module continuously acquires three frames of oral images, packages these three frames, and sends them to the image processing unit. The image processing unit uses a lightweight optical flow network to estimate the motion vector between frames and extracts the trajectory of the reflective points by detecting the brightness stable region. At the same time, the three frames, the optical flow map, and the reflective guidance map are spliced ​​together to obtain the input feature tensor X.

[0059] When executing step 1, the specific steps include:

[0060] Step 1-1: The image acquisition module continuously acquires three frames of oral cavity images, which are represented as I t -1,I t ,I t+1, where t represents the frame number and I is the image frame;

[0061] Normalize the three frames of images to generate a unified preset input size H×W, where H is the image height and W is the image width;

[0062] Stitching to form a time channel image: X c ∈R H×W×9 ; Among them, X c Temporal stitching image, R represents a real number set; 9 represents 9 channels, that is, RGB channels (3 channels) × 3 of 3 frames of image;

[0063] In this embodiment, when splicing to form a time channel image, three consecutive frames of RGB images are spliced ​​in the channel dimension direction, thereby forming an image tensor with 9 channels.

[0064] Three consecutive images were collected. t-1 ,I t ,I t+1 , each frame of the image has three channels R, G, B. The channels of these three frames are stitched together in sequence to form a new tensor, which is specifically expressed as:

[0065] X c =Concat(I t-1 ,I t ,I t+1 )∈R H×W×9 .

[0066] For example, the three frames of images are all standard color images, as follows:

[0067] I t-1 ∈R 256×256×3 ;

[0068] I t ∈R 256×256×3 ;

[0069] I t+1 ∈R 256×256×3 ;

[0070] The channel order after splicing is:

[0071] X c [:,:,0:3]=I t-1 ; The previous frame I t-1 Fill to X c Channels 0-2;

[0072] X c [:,:,3:6]=I t ; Set the current frame I t Fill to X cChannels 3-5;

[0073] X c [:,:,6:9]=I t+1 ; The next frame I t+1 Fill to X c Channels 6-8.

[0074] Step 1-2: Use the lightweight optical flow model, namely the PWCNet model, to calculate the optical flow map F between the current frames t , that is, the motion vector between frames, detects the trajectory of the reflective point P according to the change of the high brightness area t ;

[0075] F t =LightFlow(I t-1 ,I t );

[0076] P t =DetectReflectedPoints(I t );

[0077] S t =Trajectory(P t-1 ,P t ,P t+1 );

[0078] Among them, F t is the optical flow map, which represents the pixel motion vector map from the t-1th frame to the tth frame, and contains two channels: horizontal and vertical;

[0079] P t is the coordinate set of the reflection points detected in the t-th frame; S t is the spatial trajectory path formed by the reflection point in the three frames of images;

[0080] In this embodiment, LightFlow is an optical flow estimation function used to estimate the motion vector of each pixel between two adjacent frames. Specifically, based on the PWCNet model, the two frames are sampled in a pyramidal downsampling manner, and feature matching is performed step by step. A cost volume is constructed in the feature space, and a CNN is used to extract matching features. The 2D displacement vector of each pixel is predicted, and finally a two-dimensional tensor is output:

[0081] F t ∈R H×W×2 ;

[0082] Among them, there are 2 channels, one is horizontal displacement and the other is vertical displacement.

[0083] DetectReflectedPoints is a reflection point detection function used to detect the reflection points from the current frame image I t The coordinate set of the highlighted reflective area is extracted from the image, which specifically includes converting the image into a grayscale image, using the threshold method to detect pixels with brightness higher than a certain threshold, using the brightness stable area detection method (such as local peak detection + area growing method) to eliminate noise, and combining color information (for example, only retaining RGB bright and white points) to avoid false detection of teeth or mucous membranes.

[0084] P t The specific output is expressed as:

[0085] P t ={(x1,y1),(x2,y2),...,(x n ,y n )};

[0086] Among them, (x n ,y n ) represents a pixel position detected as a reflective point.

[0087] The specific steps of DetectReflectedPoints are as follows:

[0088] Step A1: Get image I t ∈R H×W×3 ;

[0089] Step A2: Image I t Convert to grayscale image:

[0090] G(x,y)=0.299×I (t，R) (x,y)+0.587×I (t，G) (x,y)+0.114×I (t，B) (x,y)

[0091] Among them, R, G, B represent color channels, (x, y) is the pixel;

[0092] Step A3: Set the brightness threshold T L , retain the highlighted area:

[0093] G(x,y)>TL Primary election highlights;

[0094] Step A4: Combine the local peak value with the regional average brightness to exclude random noise and small areas;

[0095] Step A5: retaining pixels whose RGB components are both high and close (in this embodiment, pixels that are close to white);

[0096] Step A6: Output the two-dimensional coordinate set of the selected pixels:

[0097] P t ={(x1,y1),(x2,y2),...,(x n ,y n )}.

[0098] Trajectory is a trajectory generation function, which is based on the set of reflective points in three frames. The motion path of the generated reflective points is specifically generated by using an inter-frame matching algorithm (such as a method based on position distance or optical flow back projection) in P t-1 ,P t ,P t+1 Find the correspondence between the reflective points in the three frames; establish the trajectory segments or point set sequences of the points according to the changes in the positions of the reflective points, and finally generate structured representations such as trajectory lines, speed direction, and motion trends.

[0099] In this embodiment, Trajectory includes first matching the reflective points in three frames based on Euclidean distance or neighborhood relationship, constructing a continuous path of points, and then eliminating points with sudden jumps and incoherent directions to ensure the trajectory is continuous and smooth, and finally forming a trajectory point set S t , which is usually the position of the stable reflection point tracked continuously for three frames in the current frame.

[0100] According to S t Constructing reflective guide map G t , specifically converting the reflection point position into an image mask, which is specifically expressed as:

[0101] G t =S t →TrajectoryMap∈R H×W ;

[0102] TrajectoryMap is the image mask, the reflection guide map G t In this embodiment, the TrajectoryMap projects the trajectory lines into an image mask, that is, sets the pixel values ​​at all trajectory paths as highlight marks.

[0103] In this embodiment, TrajectoryMap specifically includes first converting S t The position of each point in is projected into a two-dimensional map with the same resolution as the original image to form a sparse point map:

[0104]

[0105] The sparse map is then fuzzy diffused to form a continuous bias map that reflects the influence of the surrounding area of ​​the trajectory point, and finally a dynamic guided bias map of size H×W is generated, which is used as the subsequent attention weighting term.

[0106] Step 1-3: Splice the temporal image X c , optical flow graph F t , reflective guide map G t Combined into the input feature tensor X∈R H ×W×12 :

[0107] X=Concat(X c ,F t ,G t )∈R H×W×12 ;

[0108] Among them, Concat(·) means splicing. Optical flow map F t It represents the pixel motion vector graph from the t-1th frame to the tth frame, which contains two channels, representing the horizontal and vertical displacement of each pixel respectively. t [:,:,0] is the horizontal optical flow channel, F t [:,:,1] is the vertical optical flow channel, which is a tensor F t ∈R H×W×2 .

[0109] When constructing the input feature tensor X through multi-channel splicing, F t The two channels of the image are followed by the image stitching tensor X in their original order. c After splicing, it can be used as the 9th and 10th channels.

[0110] X is the multi-channel image data of the final input model, including 12 channels, that is, 9 channels of 3 images + 2 channels of optical flow map + 1 channel of reflective guidance map.

[0111] Step 2: Establish a model module in the image processing unit and construct a Fog-Lite model to perform temporal modeling and guided image defogging on the input feature tensor; the Fog-Lite model integrates temporal modeling, optical flow information, and reflective point guided attention mechanism to accurately segment and enhance the fog area in the image, and finally generates and outputs the fog area mask M f and clear image I * The Fog-Lite model is specifically shown in the following formula:

[0112] M f ,I * =F Fog-Lite (X);

[0113] F Fog-Lite (X) = D°A°E°T(X);

[0114] Among them, T(X) represents the temporal feature embedding module, which is used to model the input feature tensor X between frames. It mainly converts the image into a token sequence suitable for Transformer through image patch division and vectorized mapping method; E represents the encoding module, which is a multi-layer attention structure based on the Transformer architecture and is used to perform context modeling and feature extraction on the token sequence; A represents the guided attention mechanism, which specifically includes a dynamic position bias method constructed using the trajectory of reflective points to guide the multi-head attention mechanism to focus on abnormal lighting areas and fog areas; D represents the decoding module, which consists of a fog mask decoder and an image reconstruction module, and generates fog mask maps M respectively. f and clear image I * ;° represents the composite operation of functions, specifically applying multiple functions in sequence;

[0115] When executing step 2, the specific steps include:

[0116] Step 2-1: Divide the input feature tensor X into multiple small blocks, namely patches;

[0117] Then map each small patch into a token vector of fixed dimension and construct a token sequence:

[0118] T = Embed(X);

[0119] Where T is the Token sequence, which serves as the input of the Transformer encoder; Embed(X) represents the image embedding function, including patch partitioning and linear mapping algorithms;

[0120] In this embodiment, Embed(X) divides the input image feature tensor X into patches and converts each patch into a token sequence through linear mapping, and provides the result to the Transformer encoder for processing.

[0121] Embed(X) specifically includes first dividing X into multiple image patches of fixed size P×P (for example, each patch is 16×16); then flattening each patch and expressing each patch as a token vector of dimension D through a learnable linear mapping Wembed, finally forming a token sequence T∈R {N×D} , where N is the number of patches and D is the embedding dimension.

[0122] For example, the input tensor X∈R {H×W×3} The image is divided into non-overlapping blocks of a fixed size P×P, each of which is called a Patch, where 3 represents the three channels of the RGB color image.

[0123] If the original image size is H×W and the size of each patch is P×P, a total of N patches will be generated:

[0124] N=(H×W)÷P 2 ;

[0125] Each patch is flattened into a vector (size P×P×3), and then passed through a learnable linear projection W e mbed∈R {(P×P×3)×D} , and is finally mapped into a Token vector of length D.

[0126] Input the token sequence T into the Transformer encoding module E to obtain the context-enhanced token representation:

[0127] T′=E(T);

[0128] Where E(T) represents the Transformer encoding module, which includes multiple layers of multi-head attention and feedforward networks; T′ is the token sequence after context enhancement;

[0129] Step 2-2: Follow the reflective guide G t Construct dynamic position bias matrix B pt , guiding the Transformer encoding module to focus on the local fog area:

[0130]

[0131] Among them, Q, K, and V are respectively the Query matrix, the Key matrix, and the Value matrix, all of which are generated by T′ mapping; * indicates transposition; d k Indicates the dimension of the Key vector, used for normalization;

[0132] Attention(·) outputs the guided feature map, denoted as TA;

[0133] Step 2-3: Decode the attention feature map into the fog area mask map M through the MLP decoder f , combined with the original image to generate the final clear image I * ;

[0134] D={D mask (TA),D image (TA)};

[0135] M f =D mask (TA);

[0136] I * =D image(TA,I t );

[0137] Among them, D mask is a mask decoder used to generate fog area mask; D image It is an image restoration module used to restore the features to the original image I t Fusion generates clear images;

[0138] D mask and D image Constitutes the decoding module D.

[0139] In this embodiment, the mask decoder is used to generate a fog area mask map from the attention encoding result TA. Specifically, a semantic segmentation decoding structure including multi-layer convolution and upsampling can be adopted to output a pixel-level probability map with a dimension of H×W.

[0140] The image restoration module is used to combine TA with the original image I t Texture information of the dehazed image I * , which can be constructed based on feature fusion and step-by-step restoration structure (such as deconvolution or sub-pixel upsampling), and has the ability to restore clear images from feature space.

[0141] Step 3: Create an output module in the image processing unit, which calls the fog mask image M output by the Fog-Lite model. f and clear image I * , and highlight the boundary of the fog area, while visually analyzing the movement trajectory of the reflective points and the fog area to form a dynamic recognition auxiliary interface and display it on the display screen.

[0142] When executing step 3, the specific steps include:

[0143] Step 3-1: Generate the fog mask M f and clear image I * Perform fusion display, generate visualization results, and display them on the display screen;

[0144] Step 3-2: Transform the reflection point trajectory P in the continuous frames t Dynamically superimposed on the output image, it assists in judging the dynamic behavior of fog interference.

[0145] The image acquisition module is an optical imaging probe of an intraoral camera, anesthesia video laryngoscope, or fiberoptic bronchoscope; the image processing unit is an industrial computer. In this embodiment, the intraoral camera, anesthesia video laryngoscope probe, or fiberoptic bronchoscope probe uses a high-definition imaging component that supports continuous frame acquisition with a resolution of ≥720p. The industrial computer is an industrial control host equipped with a GPU (such as an RTXA2000 professional graphics card) and a high-performance CPU.

[0146] The real-time defogging method for oral images based on dynamic optical flow perception described in the present invention solves the technical problem of accurately identifying foggy areas in oral images by combining optical flow information with guidance of reflective point trajectories during dynamic imaging. The present invention adopts lightweight optical flow estimation and Transformer structure, has high inference efficiency, and can be applied to real-time image processing scenarios in actual oral diagnosis and treatment. Through three-frame splicing and optical flow estimation, it realizes effective modeling of dynamic changes in continuous images, improves defogging accuracy, and uses reflective point trajectories to construct dynamic position bias, guides the attention mechanism to focus on abnormal lighting and fogged areas, effectively reduces false detections, and the final output image is significantly superior to existing methods in terms of contrast, texture details and visible edges, enhancing the visual recognition and recognition assistance effect of the image.

Claims

1. A real-time oral image dehazing method based on dynamic optical flow perception, characterized by: The steps include: Step 1: The image acquisition module continuously acquires three frames of oral images, encapsulates these three frames and sends them to the image processing unit; the image processing unit uses a lightweight optical flow network to estimate the motion vector between frames, and extracts the trajectory of the reflective points by detecting the brightness stable area; at the same time, the three frames of images, the optical flow map and the reflective guide map are spliced ​​to obtain the input feature tensor Step 2: Create a model module in the image processing unit and build The model performs temporal modeling and guided image dehazing on the input feature tensor; The model integrates time series modeling, optical flow information, and reflective point guided attention mechanism to accurately segment and enhance the fog area in the image, and finally generates and outputs the fog area mask map. and clear images The model is specifically shown in the following formula: in, Represents the temporal feature embedding module, which is used to embed the input feature tensor Perform inter-frame modeling through images Division and vectorization mapping method to convert the image into of sequence; Represents the encoding module, which is based on The multi-layer attention structure of the architecture is used to Context modeling and feature extraction of sequences; It represents the guided attention mechanism, specifically including a dynamic position bias method constructed using the trajectory of reflective points to guide the multi-head attention mechanism to focus on abnormal lighting areas and foggy areas; Decoding module, which consists of fog mask decoder and image reconstruction module, generates fog mask image and clear images Represents a composite operation of functions, specifically applying multiple functions in sequence; Step 3: Create an output module in the image processing unit and call the output module Fog area mask output by the model and clear images The boundary of the fog area is highlighted, and the movement trajectory of the reflective points and the fog area are visually analyzed to form a dynamic recognition auxiliary interface, which is displayed on the display screen.

2. The method for real-time dehazing of oral images based on dynamic optical flow perception according to claim 1, characterized in that: The image acquisition module is an intraoral camera, an anesthesia video laryngoscope or an optical imaging probe of a fiber bronchoscope; the image processing unit is an industrial computer.

3. The method for real-time dehazing of oral images based on dynamic optical flow perception according to claim 1, characterized in that: When executing step 1, the specific steps include: Step 1-1: The image acquisition module continuously acquires three frames of oral images, which are represented as in Indicates the frame number, is the image frame; Normalize the three frames of image to generate a unified preset input size is the image height, is the image width; Stitching to form the time channel image: in, Time stitching images, represents the set of real numbers; Step 1-2: Use a lightweight optical flow model, i.e. Model, calculate the optical flow map between the current frames That is, the motion vector between frames, based on the changes in the high-brightness area, detects the trajectory of the reflective point in, is the optical flow map, which represents the Frame to The pixel motion vector map of the frame, which contains two channels: horizontal and vertical; For the The coordinate set of the reflective points detected in the frame; is the spatial trajectory path formed by the reflection point in the three frames of images; according to Building a reflective guide map Steps 1-3: Temporal stitching of images Optical flow map Reflective guide map Combined into input feature tensor in, Representation splicing; optical flow map Indicates that from Frame to The pixel motion vector graph of the frame contains two channels, which represent the horizontal and vertical displacement of each pixel respectively. is the horizontal optical flow channel, is the vertical optical flow channel, which is a tensor as a whole 4. The method for real-time dehazing of oral images based on dynamic optical flow perception according to claim 3, characterized in that: When executing step 2, the specific steps include: Step 2-1: Input feature tensor Divide into multiple small blocks, i.e. Then each small piece Mapping to fixed dimensions Vector, build sequence: in, for sequence, as The input of the encoder; represents the image embedding function; Will sequence Input to Encoding module After obtaining context enhancement express: in, express Encoding module; Context-enhanced sequence; Step 2-2: Follow the reflective guide Constructing a dynamic position bias matrix guide The encoding module focuses on the local fog area: in, They are matrix, matrix, Matrix, all composed of Mapping generation; * indicates transposition; express The dimension of the vector, used for normalization; Output the guided feature map, denoted as Step 2-3: Pass The decoder decodes the attention feature map into a fog mask map Combined with the original image to generate the final clear image in, It is a mask decoder used to generate fog area mask; It is an image restoration module used to restore features to the original image. Fusion generates clear images; and Constitutes the decoding module 5. The method for real-time dehazing of oral images based on dynamic optical flow perception according to claim 1, characterized in that: When executing step 3, the specific steps include: Step 3-1: Generate the fog mask and clear images Perform fusion display, generate visualization results, and display them on the display screen; Step 3-2: Track the reflection points in consecutive frames Dynamically superimposed on the output image, it assists in judging the dynamic behavior of fog interference.

Images