Computer vision-based dysphagia screening system

Abstract

The application provides a computer vision-based dysphagia screening system, and relates to the field of image recognition technology. The system comprises an image acquisition module, which acquires image data of a swallowing event to obtain a time-series image sequence; a motion field establishment module, which extracts a moving point trajectory sequence from the time-series image sequence and establishes a global motion transformation field; a motion separation module, which corrects the global motion transformation field by registration and separates the motion of a stabilized image sequence to obtain a local deformation component; a region segmentation module, which segments a key region of the local deformation component to obtain swallowing function parameters; and a risk identification module, which identifies the risk of dysphagia of the swallowing function parameters and generates a dysphagia screening report. The application can solve the technical problem of poor accuracy of computer vision-based dysphagia screening in the prior art and achieve the technical effect of improving the accuracy of computer vision-based dysphagia screening.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a computer vision-based dysphagia screening system. Background Technology

[0002] Existing computer vision-based methods for screening dysphagia primarily focus on analyzing gold-standard images such as video fluorescence swallowing angiography, aiming to improve the objectivity and efficiency of assessment through automated image processing techniques. However, existing technologies face a series of technical limitations in practice. First, existing methods are often inadequate in handling motion artifacts. Motion artifacts mainly originate from unavoidable involuntary head movements, sudden coughing, or significant traction of the neck skin accompanying swallowing movements, as well as streaking noise caused by metallic implants in imaging equipment. Existing technologies typically employ simple global image registration or basic filtering algorithms to attempt to suppress these artifacts, but these methods often struggle to distinguish artifact motion from genuine, subtle swallowing physiological movements. For example, a slight head movement might be misinterpreted by the algorithm as the initial elevation of the Adam's apple, leading to incorrect staging of the swallowing event; and general denoising algorithms for metallic artifacts, while eliminating streaks, easily erase important anatomical boundary details, such as the contour of the vallecula, causing bias in subsequent quantitative analysis of residues. Second, existing solutions to the motion blur problem have significant limitations. Motion blur is caused by the extremely rapid swallowing motion, coupled with the limited frame rate of routine clinical imaging, resulting in motiony anatomical structures appearing as blurry images and edges in a single frame. Existing methods often rely on edge enhancement of single-frame static images or traditional inter-frame differencing. The former struggles to recover true edges when blurring is severe, while the latter distorts inter-frame variation information due to blurring, making it impossible to accurately track the bolus trajectory or calculate its speed. Furthermore, current technical frameworks typically treat dynamic artifacts and motion blur as independent problems, failing to fundamentally model their temporal coupling. For example, a violent cough is often followed by a compensatory rapid swallow; this sequential correlation is not effectively utilized to improve the analysis's fault tolerance.

[0003] In summary, existing technologies suffer from technical problems such as the inability to robustly separate and compensate for dynamic artifacts introduced by non-swallowing activities, and the lack of effective modeling and recovery capabilities for blurring caused by rapid physiological movements of swallowing. These issues lead to significant errors or even distortions in key biomechanical features extracted from dynamic image sequences, further affecting the accuracy and reliability of screening results. Summary of the Invention

[0004] The purpose of this application is to provide a computer vision-based swallowing disorder screening system to address the technical problems in the prior art, such as the inability to robustly separate and compensate for dynamic artifacts introduced by non-swallowing activities, and the lack of effective modeling and recovery capabilities for blurring caused by rapid physiological movements of swallowing. These problems result in large errors or even distortions in key biomechanical features extracted from dynamic image sequences, further affecting the accuracy and reliability of screening results.

[0005] In view of the above problems, this application provides a computer vision-based swallowing disorder screening system, comprising: an image acquisition module for acquiring and preprocessing image data of swallowing events to obtain a time-series image sequence; a motion field establishment module for extracting activity point trajectory sequences from the time-series image sequence and establishing a global motion transformation field; a motion separation module for registering and correcting the global motion transformation field to obtain a stable image sequence, and performing motion separation on the stable image sequence based on a velocity flow field to obtain local deformation components; a region segmentation module for segmenting key regions of the local deformation components to obtain swallowing function parameters; and a risk identification module for identifying the impairment risk of the swallowing function parameters and generating a swallowing disorder screening report.

[0006] Preferably, the computer vision-based dysphagia screening system further includes: an image sequence acquisition unit for acquiring dynamic image sequences of swallowing events; and a time-series image sequence output unit for unifying the resolution and frame rate of the dynamic image sequences, performing color space conversion, and outputting the time-series image sequences.

[0007] Preferably, the computer vision-based dysphagia screening system further comprises: a non-swallowing activity point extraction unit, used to extract non-swallowing activity points of the face and neck in the head region of the time-series image sequence; a swallowing activity point extraction unit, used to extract swallowing activity points of the face and neck excluding the non-swallowing activity points in the time-series image sequence; and an activity point trajectory sequence acquisition unit, used to acquire the activity point trajectory sequence in the time-series image sequence based on the swallowing activity points.

[0008] Preferably, the computer vision-based swallowing disorder screening system is further configured with: a reference frame acquisition unit, configured to use the resting frame before swallowing initiation in the activity point trajectory sequence as a reference frame and acquire the reference coordinate space of the reference frame; and a global motion transformation field establishment unit, configured to extract the activity point image coordinates of the t-th frame of the activity point trajectory sequence and map them to the reference coordinate space to compensate for the motion of the swallowing activity point and establish the global motion transformation field.

[0009] Preferably, the computer vision-based dysphagia screening system is further configured with: a pixel value acquisition unit, used to invert the global motion transformation field to obtain the pixel value of the t-th frame; and a stabilized image sequence generation unit, used to perform spatial transformation and resampling on the global motion transformation field based on the pixel value of the t-th frame to generate a stabilized image sequence that compensates for global motion.

[0010] Preferably, the computer vision-based dysphagia screening system is further configured as follows: a local deformation component obtaining unit, used to analyze the motion vector field of the stabilized image sequence, and to divide the flow patterns of overall motion and local deformation using spatial consistency, thereby obtaining global motion components and local deformation components.

[0011] Preferably, the computer vision-based dysphagia screening system further comprises: an input tensor construction unit, used to slide and crop three-dimensional image blocks of the local deformation components at a preset time step to construct an input tensor, wherein the input tensor includes tensor dimensions, including spatial height, spatial width, number of channels, and temporal depth; a three-dimensional mask sequence output unit, used to perform three-dimensional convolution segmentation on the input tensor to output a three-dimensional mask sequence; and a swallowing function parameter acquisition unit, used to acquire the swallowing function parameters based on the three-dimensional mask sequence, wherein the swallowing function parameters include kinematic parameters and morphological and temporal parameters.

[0012] Preferably, the computer vision-based dysphagia screening system is further configured to: a laryngeal motion parameter acquisition channel, used to analyze the displacement-time curve of the laryngeal protuberance landmark in the vertical direction, and differentiate it to obtain the velocity-time curve, extract the maximum displacement, maximum velocity, and rise time to obtain the laryngeal protuberance motion trajectory, calculate the phase difference between the hyoid bone and the laryngeal protuberance motion trajectory, obtain the hyoid-larynx complex motion coordination, and acquire laryngeal motion parameters; a bolus motion parameter acquisition channel, used to calculate the bolus motion trajectory of the bolus centroid in the pharyngeal cavity, analyze the bolus motion velocity and the minimum distance between the bolus and the airway inlet, and acquire bolus motion parameters; a pharyngeal cavity opening parameter acquisition channel, used to analyze the change curve of the anteroposterior diameter of the pharyngeal cavity at the level of the upper esophageal sphincter over time, and acquire pharyngeal cavity opening parameters; and a kinematic parameter acquisition channel, used to obtain the kinematic parameters by taking the laryngeal motion parameters, the bolus motion parameters, and the pharyngeal cavity opening parameters.

[0013] Preferably, the computer vision-based dysphagia screening system is further configured to: obtain a pharyngeal residual volume channel, which analyzes the frame after swallowing and calculates the pixel area of ​​the residue in the epiglottic vallecula and piriform sinus regions to obtain the pharyngeal residual volume; obtain a swallowing timing channel, which divides the duration of the oral phase, pharyngeal phase, and esophageal phase based on the bolus position to obtain the swallowing timing; and obtain morphological and temporal parameters, which use the pharyngeal residual volume and the swallowing timing as the morphological and temporal parameters.

[0014] Preferably, the computer vision-based dysphagia screening system further includes: a dysphagia deviation vector output unit, used to compare the dysphagia function parameters with normal dysphagia parameters in a normal value database, and output a dysphagia deviation vector; a risk level label obtaining unit, used to determine whether the dysphagia event has a risk of impairment based on the dysphagia deviation vector, and obtain a risk level label; and a report generation unit, used to generate the dysphagia screening report through the risk level label.

[0015] The technical solution provided in this application has at least the following technical effects or advantages: by achieving the technical goal of a computer vision screening that can automatically, accurately and robustly analyze dynamic images and output quantitative swallowing function assessment, it achieves the technical effects of improving the automation level and objectivity of the screening process and enhancing the resistance to motion interference in complex real-world scenarios.

[0016] The above description is merely an overview of the technical solution of this application. To enable a clearer understanding of the technical means of this application and to facilitate its implementation according to the description, and to make the above and other objects, features, and advantages of this application more apparent, specific embodiments of this application are described below. It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent through the following description. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the structure of the computer vision-based swallowing disorder screening system of this application.

[0019] Figure 2 This is a schematic diagram of the image acquisition module in the computer vision-based dysphagia screening system of this application.

[0020] Figure labeling: Image acquisition module 1, sports field establishment module 2, motion separation module 3, region segmentation module 4, risk identification module 5. Detailed Implementation

[0021] This application provides a computer vision-based swallowing disorder screening system, addressing the technical problems in existing technologies. These problems stem from the failure to robustly separate and compensate for dynamic artifacts introduced by non-swallowing activities, and the lack of effective modeling and recovery capabilities for blurring caused by rapid physiological swallowing movements. These issues lead to significant errors or even distortions in key biomechanical features extracted from dynamic image sequences, further affecting the accuracy and reliability of screening results. The system aims to achieve the technical goal of providing a computer vision-based screening system that can automatically, accurately, and robustly analyze dynamic images and output quantitative swallowing function assessments. This will improve the automation and objectivity of the screening process and enhance its resistance to motion interference in complex real-world scenarios.

[0022] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. It should be understood that this application is not limited to the exemplary embodiments described herein. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application. It should also be noted that, for ease of description, only the parts related to this application are shown in the accompanying drawings, not all of them.

[0023] Please see Figure 1 and Figure 2 This application provides a computer vision-based swallowing disorder screening system, specifically including:

[0024] Image acquisition module 1 is used to acquire image data of swallowing events and perform preprocessing to obtain a time-series image sequence.

[0025] Furthermore, this application also includes: an image sequence acquisition unit for acquiring dynamic image sequences of swallowing events; and a time-series image sequence output unit for unifying the resolution and frame rate of the dynamic image sequence, performing color space conversion, and outputting the time-series image sequence.

[0026] Specifically, the entire process of a subject performing a swallowing action is recorded using an image capture device, thereby obtaining a dynamic image sequence of the swallowing event. The swallowing event refers to the complete physiological process from the preparation of the food bolus through the mouth, its transport through the pharynx, to its entry into the esophagus. The dynamic image sequence refers to a data set consisting of multiple frames of images captured sequentially in chronological order by the image capture device.

[0027] Subsequently, standardized preprocessing operations are performed on the dynamic image sequence. These include adjusting the resolution and frame rate of all images in the sequence to the same preset specifications, where resolution refers to the number of pixels per frame and frame rate refers to the number of image frames per second in the sequence; and converting the image's color space to a preset target color space, such as from red-green-blue color space to grayscale color space. Color space conversion refers to the process of mapping data representing image colors from one representation system to another based on specific mathematical relationships. The final output is a time-series image sequence with uniform specifications, facilitating stable processing by subsequent algorithms. This means that after preprocessing, each frame is strictly aligned in the time dimension and has a consistent representation.

[0028] The motion field establishment module 2 is used to extract the trajectory sequence of activity points based on the time-series image sequence and establish a global motion transformation field.

[0029] Furthermore, this application also includes: a non-swallowing activity point extraction unit, used to extract non-swallowing activity points of the face and neck in the head region of the time-series image sequence; a swallowing activity point extraction unit, used to extract swallowing activity points of the face and neck in the time-series image sequence excluding the non-swallowing activity points; and an activity point trajectory sequence acquisition unit, used to acquire the activity point trajectory sequence in the time-series image sequence based on the swallowing activity points.

[0030] Furthermore, this application also includes: a reference frame acquisition unit, used to take the resting frame before swallowing initiation in the activity point trajectory sequence as a reference frame and acquire the reference coordinate space of the reference frame; and a global motion transformation field establishment unit, used to extract the activity point image coordinates of the t-th frame of the activity point trajectory sequence and map them to the reference coordinate space to compensate for the motion of the swallowing activity point and establish the global motion transformation field.

[0031] Specifically, non-swallowing activity points are identified and located from each frame of the time-series image sequence. These non-swallowing activity points remain relatively spatially stationary or undergo only rigid displacement with the head during the physiological swallowing event. Their locations include, for example, the root of the nose, the outer canthi of both eyes, and the bilateral tragus points. Extracting these non-swallowing activity points establishes a stable spatial reference system for subsequent separation of overall head movements.

[0032] Within the same temporal image sequence, a separate set of key points, distinct from non-swallowing activity points, is identified and located as swallowing activity points. These swallowing activity points undergo specific spatial movements during swallowing, and their locations include the Adam's apple (the anterior prominence of the thyroid cartilage) and the anterior point of the hyoid body. Swallowing activity points are distinguished from non-swallowing activity points based on their morphological and positional changes within the image sequence, both spatially and functionally.

[0033] Subsequently, using each swallowing activity point as the tracking target, visual tracking methods such as cross-frame matching or optical flow calculation are used to determine the two-dimensional or three-dimensional coordinates of each point in each frame of the temporal image sequence, thereby obtaining a coordinate data that changes continuously over time, which serves as the trajectory sequence of the activity point and represents the complete kinematic information of the swallowing-related anatomical structures in the time dimension.

[0034] A specific image frame is selected from the activity point trajectory sequence as a baseline. This selected frame is defined as the reference frame, which is the image frame in which the subject is in a static state before the swallowing action begins. Then, a reference coordinate space is obtained based on the reference frame. The reference coordinate space is a two-dimensional coordinate system defined with the image plane of the reference frame as the reference. Its origin, coordinate axis directions, and scale units constitute the unified standard for all subsequent spatial measurements.

[0035] Next, for any t-th frame in the activity point trajectory sequence, where t represents a specific index in the time series, the image coordinates of all swallowing activity points in the t-th frame are extracted. Image coordinates refer to the specific row and column positions of the swallowing activity points in the digital image pixel matrix. A spatial transformation algorithm is used to map the image coordinates from the local coordinate system of the t-th frame to the reference coordinate space. This aims to compensate for the motion of the swallowing activity points. Motion compensation refers to using geometric correction to offset the global displacement caused by the overall head movement, thereby separating the local physiological motion. Based on the mapping relationship, a global motion transformation field is established. The global motion transformation field is a mathematical representation, expressed as a transformation matrix or vector field, describing the overall spatial transformation relationship that all pixels must undergo from any t-th frame to the reference frame.

[0036] The motion separation module 3 is used to register and correct the global motion transformation field to obtain a stable image sequence, and to perform motion separation on the stable image sequence based on the velocity flow field to obtain a local deformation component.

[0037] Furthermore, this application also includes: a pixel value acquisition unit, used to invert the global motion transformation field to obtain the pixel value of the t-th frame; and a stabilized image sequence generation unit, used to perform spatial transformation and resampling on the global motion transformation field based on the pixel value of the t-th frame to generate a stabilized image sequence that compensates for global motion.

[0038] Furthermore, this application also includes: a local deformation component obtaining unit, used to analyze the motion vector field of the stabilized image sequence, and use spatial consistency to divide the flow patterns of overall motion and local deformation to obtain global motion components and local deformation components.

[0039] Specifically, the global motion transformation field undergoes a mathematical inverse operation. This inverse operation is performed to obtain the correspondence between the reference coordinate space and the original coordinate space of frame t. Based on this inverse transformation, the specific location of the original image of frame t corresponding to a point in the reference coordinate space is determined, thereby obtaining the pixel value of frame t. The pixel value of frame t refers to the intensity or color value at a specific coordinate within the digital image matrix of frame t in the time-series image sequence.

[0040] Furthermore, using inverse transform relationships and pixel value information, specific geometric processing is performed on each frame of the image, including spatial transformation and resampling. Spatial transformation refers to the geometric operation of recalculating and locating the coordinates of all pixels in the image according to the inverse transform model. Resampling is necessary because the new coordinate positions after transformation may not be located at integer grid points of the original image pixels; therefore, an interpolation algorithm is used to calculate the pixel intensity value at the new coordinate positions. Interpolation algorithms include bilinear interpolation and cubic convolution interpolation. By performing transformation and resampling operations based on the pixel values ​​of its own frame t for each frame in the temporal image sequence, a stable image sequence is finally output. Compensation for global motion refers to eliminating the overall image drift or rotation caused by rigid head movement. The stable image sequence is a continuous set of image data in which non-swallowing-related global motion has been effectively suppressed, making swallowing-related local physiological movements visually prominent.

[0041] Motion analysis is performed on the stabilized image sequence to calculate the motion vector field. The motion vector field refers to the field composed of the two-dimensional displacement vectors of each pixel between adjacent frames, estimated from the stabilized image sequence using optical flow or block matching algorithms, and represents the apparent motion information of all visible points in the image.

[0042] The spatial consistency principle is used to classify motion vector fields into patterns. Spatial consistency refers to the uniformity and continuity of motion patterns in the spatial distribution of an image. Specifically, overall motion refers to the flow patterns with highly consistent motion directions exhibited in a large area of ​​the image, corresponding to residual rigid displacements that are not fully compensated. Local deformation, on the other hand, refers to the non-rigid flow patterns exhibited in specific local areas of the image that are significantly different from the surrounding overall motion patterns, such as expansion, contraction, or shear deformation. These two types of flow patterns can be separated through cluster analysis or robust statistical methods.

[0043] Subsequently, after the division was completed, global motion components and local deformation components were obtained. Global motion components refer to the model parameters or vector field subsets extracted from the motion vector field that represent the global translation or rotation of the image. Local deformation components refer to the vector field part that remains after removing global motion components from the motion vector field and mainly reflects the deformation and motion of the target organ or tissue itself, such as the motion corresponding to the elevation of the larynx and the closure of the pharynx.

[0044] Region segmentation module 4 is used to segment the local deformation components into key regions to obtain swallowing function parameters.

[0045] Furthermore, this application also includes: an input tensor construction unit, used to slide and crop three-dimensional image blocks on the local deformation components at a preset time step to construct an input tensor, the input tensor including tensor size, the tensor size including spatial height, spatial width, number of channels and temporal depth; a three-dimensional mask sequence output unit, used to perform three-dimensional convolution segmentation on the input tensor to output a three-dimensional mask sequence; and a swallowing function parameter acquisition unit, used to acquire the swallowing function parameters based on the three-dimensional mask sequence, the swallowing function parameters including kinematic parameters and morphological and temporal parameters.

[0046] Furthermore, this application also includes: a laryngeal motion parameter acquisition channel, used to analyze the displacement-time curve of the laryngeal protuberance landmark in the vertical direction, and differentiate it to obtain the velocity-time curve, extract the maximum displacement, maximum velocity, and rise time to obtain the laryngeal protuberance motion trajectory, calculate the phase difference between the hyoid bone and the laryngeal protuberance motion trajectory, obtain the hyoid-larynx complex motion coordination, and acquire laryngeal motion parameters; a bolus motion parameter acquisition channel, used to calculate the bolus motion trajectory of the bolus centroid in the pharyngeal cavity, analyze the bolus motion velocity and the minimum distance between the bolus and the airway inlet, and acquire bolus motion parameters; a pharyngeal cavity opening parameter acquisition channel, used to analyze the change curve of the anteroposterior diameter of the pharyngeal cavity at the level of the upper esophageal sphincter over time, and acquire pharyngeal cavity opening parameters; and a kinematic parameter acquisition channel, used to use the laryngeal motion parameters, the bolus motion parameters, and the pharyngeal cavity opening parameters as the kinematic parameters.

[0047] Furthermore, this application also includes: a pharyngeal residual volume acquisition channel, used to analyze the frame after swallowing is completed, calculate the pixel area of ​​the residue in the epiglottic vallecula and piriform sinus regions, and obtain the pharyngeal residual volume; a swallowing timing acquisition channel, used to divide the duration of the oral phase, pharyngeal phase, and esophageal phase based on the bolus position, and obtain the swallowing timing; and a morphological and temporal parameter acquisition channel, used to obtain the morphological and temporal parameters with the pharyngeal residual volume and the swallowing timing.

[0048] Specifically, a 3D image patch is slidably extracted from the local deformation component at a preset time step. The preset time step refers to the fixed interval of frames between the starting frames of adjacent image patches. Sliding extraction refers to continuously moving a fixed-length window in the time dimension to cover the entire sequence. A 3D image patch is a three-dimensional data unit composed of both spatial and temporal dimensions, containing pixel information of the same spatial region from multiple consecutive frames. An input tensor is then constructed, which is a standard data structure format accepted by deep learning networks. The input tensor includes tensor dimensions, including spatial height, spatial width, number of channels, and temporal depth. Spatial height and spatial width together describe the pixel range of the image patch in a two-dimensional plane; the number of channels represents the color or feature map hierarchy of the image, with a channel value of 1 for grayscale images; and the temporal depth refers to the number of consecutive frames contained in the image patch, encoding the temporal information of the motion.

[0049] Next, the constructed input tensor undergoes 3D convolutional segmentation. 3D convolutional segmentation is a deep learning operation that utilizes 3D convolutional kernels to simultaneously extract and analyze features across three dimensions: spatial height, spatial width, and temporal depth. The 3D convolutional segmentation network employs an encoder-decoder structure. The encoder is responsible for progressive downsampling to extract abstract features, while the decoder is responsible for progressive upsampling to restore spatial details and achieve pixel-level classification. The final output is a 3D mask sequence, which is a binary or multi-class labeled sequence corresponding to the spatiotemporal dimensions of the original input, classifying each pixel in each frame as a specific anatomical structure or background, such as identifying regions like food boluses or airways.

[0050] Furthermore, based on a three-dimensional mask sequence, swallowing function parameters are quantitatively obtained. These parameters are a set of quantitative indicators derived from imaging data to objectively assess swallowing safety and effectiveness. Specifically, by analyzing the displacement-time curve of the laryngeal landmark in the vertical direction, which is a function graph describing the spatial position of the laryngeal prominence over time, the rate of change graph, i.e., the velocity-time curve, can be obtained by mathematically differentiating this curve. Extreme points and characteristic intervals can be extracted from the curve, thus obtaining specific measurements such as maximum displacement, maximum velocity, and rise time, which together constitute a quantitative description of the laryngeal prominence's movement trajectory.

[0051] Furthermore, by calculating the phase difference between the hyoid bone movement trajectory and the laryngeal protuberance movement trajectory, the phase difference is a physical quantity that measures the degree of temporal offset between two periodic movements. This leads to the hyoid-larynx complex movement coordination, a comprehensive index reflecting the coordination of swallowing neuromuscular control, which is used to classify laryngeal movement parameters.

[0052] Meanwhile, by calculating the change in the spatial position of the bolus centroid within the pharynx, the trajectory of the bolus movement can be obtained. The bolus centroid is the point representing the geometric center of the bolus. Analyzing the trajectory can yield the bolus movement speed and the minimum distance between the bolus and the airway inlet. The airway inlet refers to the narrowest part where the trachea connects to the pharynx. These multiple indicators are collectively referred to as bolus movement parameters.

[0053] Furthermore, the changes in the anteroposterior diameter of the pharynx at the level of the upper esophageal sphincter over time were analyzed. The level of the upper esophageal sphincter refers to a specific anatomical plane at the junction of the pharynx and esophagus, and the anteroposterior diameter is the distance between the anterior and posterior walls of the pharynx at this plane. From these changes, the pharyngeal opening parameter can be obtained, which reflects the instantaneous patency of the pharyngeal passage during swallowing. Finally, the laryngeal motion parameters, bolus motion parameters, and pharyngeal opening parameters were collectively summarized into kinematic parameters.

[0054] Simultaneously, morphological and temporal parameters were analyzed. By analyzing specific image frames after the swallowing action was completed, the pixel area of ​​the residue in the vallecula and pyriform sinus regions was calculated. The vallecula and pyriform sinus are physiological depressions in the pharynx that easily trap food boluses. The pixel area refers to the total number of pixels identified as residue, which, after calibration, can reflect the actual amount of residue, thus obtaining the amount of residue in the pharyngeal cavity.

[0055] Furthermore, based on the movement and position of the bolus within the pharynx, the continuous swallowing event is divided into three typical phases: the oral phase, the pharyngeal phase, and the esophageal phase. The oral phase is the preparation and propulsion of the bolus within the oral cavity; the pharyngeal phase is the passage of the bolus through the pharynx and the triggering of protective reflexes; and the esophageal phase is the entry of the bolus into the esophagus and its transport to the stomach. The duration of each phase is measured to obtain the swallowing sequence. The amount of pharyngeal residue and the swallowing sequence are defined as the aforementioned morphological and temporal parameters.

[0056] Risk identification module 5 is used to identify the risk of impairment in the swallowing function parameters and generate a swallowing disorder screening report.

[0057] Furthermore, this application also includes: a swallowing deviation vector output unit, used to compare the swallowing function parameters with normal swallowing parameters in the normal value database, and output a swallowing deviation vector; a risk level label obtaining unit, used to determine whether there is a risk of obstruction in the swallowing event based on the swallowing deviation vector, and obtain a risk level label; and a report generation unit, used to generate a swallowing disorder screening report through the risk level label.

[0058] Specifically, swallowing normal parameters are retrieved from a pre-established normal value database. The normal value database refers to a set of reference standards constructed by collecting and statistically analyzing data from a large number of subjects whose swallowing function has been clinically confirmed to be normal. Swallowing normal parameters are the statistical average values ​​and normal fluctuation ranges of various swallowing function parameters stored in the database.

[0059] The swallowing function parameters of the current subject—namely, the quantified kinematic, morphological, and temporal parameters—are compared one by one with the corresponding normal parameters in the database, and differences are calculated to output a swallowing deviation vector. The swallowing deviation vector is a mathematical vector where each dimension represents the standardized offset of a specific swallowing function parameter relative to its normal reference value, expressed as a multiple of standard deviation or a percentage difference, quantitatively characterizing the degree of dispersion between the subject's swallowing pattern and the normal pattern.

[0060] Next, the swallowing deviation vector is used as input features and analyzed through a pre-defined classification decision model. This model can be a lightweight classifier trained using machine learning algorithms or a rule engine built on clinical knowledge, used to comprehensively evaluate the deviation patterns of all parameters. The calculation determines whether a swallowing event poses a risk of impairment; this risk refers to the probability of an event impairing swallowing safety or effectiveness. Finally, a clear risk level label is output. This label is a discrete classification result, expressed as a qualitative level such as low risk, medium risk, or high risk, or a corresponding numerical score, used to summarize the overall risk level of the screened event.

[0061] Next, the automated report generation step is executed, using risk level labels as the basis to drive the report generation engine. The risk level labels are correlated and integrated with specific swallowing function parameter values, calculated deviation vectors, and pre-defined report templates to automatically generate a structured swallowing disorder screening report. The swallowing disorder screening report is a comprehensive document integrating key data, visualizations, and textual conclusions. Its content includes at least a list of abnormal parameters and risk level conclusions, serving as the final output of the screening method.

[0062] In summary, the computer vision-based dysphagia screening system provided in this application has the following technical effects: by achieving the technical goal of a computer vision screening system that can automatically, accurately and robustly analyze dynamic images and output quantitative dysphagia function assessments, it improves the automation level and objectivity of the screening process and enhances the resistance to motion interference in complex real-world scenarios.

[0063] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0064] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of this application and its equivalents, this application also intends to include such modifications and variations.

Claims

1. A computer vision-based swallowing disorder screening system, characterized in that, include: The image acquisition module is used to acquire and preprocess image data of swallowing events to obtain a time-series image sequence. The motion field establishment module is used to extract the trajectory sequence of active points based on the time-series image sequence and establish a global motion transformation field; The motion separation module is used to register and correct the global motion transformation field to obtain a stable image sequence, and to perform motion separation on the stable image sequence based on the velocity flow field to obtain a local deformation component; The region segmentation module is used to segment the local deformation components into key regions to obtain swallowing function parameters; The risk identification module is used to identify the risk of impairment in the swallowing function parameters and generate a swallowing disorder screening report; The sports field creation module includes: The non-swallowing activity point extraction unit is used to extract non-swallowing activity points of the face and neck in the head region of the time-series image sequence; A swallowing activity point extraction unit is used to extract swallowing activity points from the face and neck in the time-series image sequence, excluding non-swallowing activity points. The activity point trajectory sequence acquisition unit is used to acquire the activity point trajectory sequence in the time-series image sequence based on the swallowing activity points.

2. The computer vision-based swallowing disorder screening system as described in claim 1, characterized in that, The image acquisition module includes: The image sequence acquisition unit is used to acquire dynamic image sequences of swallowing events; The time-series image sequence output unit is used to unify the resolution and frame rate of the dynamic image sequence, perform color space conversion, and output the time-series image sequence.

3. The computer vision-based swallowing disorder screening system as described in claim 1, characterized in that, The sports field creation module also includes: The reference frame acquisition unit is used to acquire the reference coordinate space of the reference frame by taking the resting frame before swallowing initiation in the activity point trajectory sequence as the reference frame. The global motion transformation field establishment unit is used to extract the image coordinates of the active point in the t-th frame of the active point trajectory sequence, map them to the reference coordinate space, compensate for the motion of the swallowing active point, and establish the global motion transformation field.

4. The computer vision-based swallowing disorder screening system as described in claim 3, characterized in that, The motion separation module includes: The pixel value is obtained by calculating the inverse of the global motion transformation field to obtain the pixel value of the t-th frame. A stable image sequence generation unit is used to perform spatial transformation and resampling on the global motion transformation field based on the pixel value of the t-th frame to generate a stable image sequence that compensates for global motion.

5. The computer vision-based swallowing disorder screening system as described in claim 1, characterized in that, The motion separation module further includes: The local deformation component is obtained by analyzing the motion vector field of the stabilized image sequence. It uses spatial consistency to divide the flow patterns of overall motion and local deformation, and obtains the global motion component and the local deformation component.

6. The computer vision-based swallowing disorder screening system as described in claim 1, characterized in that, The region segmentation module includes: An input tensor construction unit is used to slide and crop three-dimensional image blocks on the local deformation components at a preset time step to construct an input tensor. The input tensor includes tensor dimensions, which include spatial height, spatial width, number of channels, and temporal depth. A 3D mask sequence output unit is used to perform 3D convolution segmentation on the input tensor and output a 3D mask sequence. The swallowing function parameter acquisition unit is used to acquire the swallowing function parameters based on the three-dimensional mask sequence. The swallowing function parameters include kinematic parameters and morphological and temporal parameters.

7. The computer vision-based swallowing disorder screening system as described in claim 6, characterized in that, The swallowing function parameter acquisition unit includes: The laryngeal motion parameter acquisition channel is used to analyze the displacement-time curve of the laryngeal protuberance landmark in the vertical direction, and to obtain the velocity-time curve by differentiation. The maximum displacement, maximum velocity, and rise time are extracted to obtain the laryngeal protuberance motion trajectory. The phase difference between the hyoid bone and the laryngeal protuberance motion trajectory is calculated to obtain the hyoid-larynx complex motion coordination and acquire laryngeal motion parameters. The bolus motion parameters are obtained through a channel to calculate the trajectory of the bolus's center of mass within the pharynx, analyze the bolus motion velocity and the minimum distance between the bolus and the airway inlet, and obtain the bolus motion parameters. The pharyngeal opening parameter was obtained through a channel, which was used to analyze the change curve of the anteroposterior diameter of the pharynx at the level of the upper esophageal sphincter over time, and thus obtained the pharyngeal opening parameter. A kinematic parameter acquisition channel is used to obtain the laryngeal motion parameters, the bolus motion parameters, and the pharyngeal opening parameters as the kinematic parameters.

8. The computer vision-based swallowing disorder screening system as described in claim 6, characterized in that, The swallowing function parameter acquisition unit further includes: The pharyngeal residual volume is obtained through a channel used to analyze the frame after swallowing is completed, and the pixel area of ​​the residue in the vallecula and pyriform sinus regions is calculated to obtain the pharyngeal residual volume; The swallowing sequence is obtained through a channel, which is used to divide the duration of the oral phase, pharyngeal phase, and esophageal phase based on the position of the bolus, thus obtaining the swallowing sequence. A morphological and temporal parameter acquisition channel is used to obtain the morphological and temporal parameters based on the residual amount in the pharynx and the swallowing sequence.

9. The computer vision-based swallowing disorder screening system as described in claim 1, characterized in that, The risk identification module includes: The swallowing deviation vector output unit is used to compare the swallowing function parameters with the normal swallowing parameters in the normal value database and output the swallowing deviation vector. The risk level labeling unit is used to determine whether there is a risk of obstruction to the swallowing event based on the swallowing deviation vector, and to obtain a risk level label. The report generation unit is used to generate the swallowing disorder screening report based on the risk level label.

Images