Laryngoscope imaging analysis method and system based on deep learning

Abstract

The invention discloses a laryngoscope imaging analysis method and system based on deep learning. The method comprises the steps of data acquisition, data preprocessing, lesion static detection, vocal cord dynamic analysis and laryngoscope imaging analysis. The invention relates to the technical field of laryngoscope analysis, in particular to a laryngoscope imaging analysis method and system based on deep learning, according to the scheme, two mutually supporting functional designs of lesion static detection and vocal cord dynamic analysis are adopted, detection of an intuitive lesion area is carried out through a static image, and assistance and support are carried out through vocal cord dynamic analysis; detecting by adopting an improved convolutional neural network method combined with feature optimization; the vocal cord dynamic analysis is carried out by adopting a residual error improved bidirectional long-short-term memory neural network method combined with bidirectional improved time sequence convolution, a powerful auxiliary reference is provided for static image lesion detection, meanwhile, the method can also be used as an independent detection function, and the overall connectivity and reusability of the method are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of laryngoscope analysis, and in particular to a laryngoscope imaging analysis method and system based on deep learning. Background Art

[0002] The deep learning-based laryngoscope imaging analysis method is a system that uses advanced artificial intelligence technology to automatically process and analyze laryngoscope images. This method uses a deep learning model to perform static lesion detection on the vocal cords and surrounding tissues, and can effectively identify and classify various pathological changes. At the same time, the system also has the function of vocal cord dynamic analysis, which monitors the movement of the vocal cords in real time and evaluates its functional status. This intelligent analysis not only improves the accuracy and efficiency of diagnosis, but also provides important decision-making support for clinicians, promoting the digital transformation of otolaryngology.

[0003] However, in the existing laryngoscopic imaging analysis methods, there is a technical problem that the existing intelligent laryngoscopic imaging analysis technology is mostly used to analyze and annotate static images, and is directly used in diagnosis or research. However, when laryngoscopic imaging analysis simply uses static images, it is easy to ignore lesions or potential risks that are difficult to find through intuitive observation, which leads to insufficient analysis depth of laryngoscopic imaging analysis and cannot meet objective technical needs; in the existing static lesion detection methods of laryngoscopic imaging analysis, there is a technical problem that the analysis depth of static images in the existing technology depends on the feature mining depth, and the existing intelligent methods still have room for improvement in the deep feature mining of static laryngoscopic images; in the existing vocal cord dynamic analysis methods of laryngoscopic imaging analysis, there is a technical problem that it is difficult for traditional methods to effectively combine vocal cord dynamic analysis and laryngoscopic image lesion detection. Summary of the invention

[0004] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides a laryngoscope imaging analysis method and system based on deep learning. In the existing laryngoscope imaging analysis method, there is a problem that the existing laryngoscope imaging analysis intelligent technology is mostly aimed at analyzing and annotating static images, and directly put into diagnosis or research for use. However, when laryngoscope imaging analysis simply uses static images, it is easy to ignore lesions or potential risks that are difficult to find through intuitive observation, which leads to insufficient analysis depth of laryngoscope imaging analysis and cannot meet the technical needs of objective technology. The present invention creatively adopts two mutually supporting functional designs of static lesion detection and dynamic analysis of vocal cords. The static images are used to detect the intuitive lesion area, and the dynamic analysis of vocal cords is used to assist and support it. At the same time, the two functions are modularly differentiated in design, which can be used as a separate function or in combination, thereby improving the overall automation of the method and the comprehensive usability of the system. In the existing static lesion detection method of laryngoscope imaging analysis, there is a problem that the analysis depth of static images in the existing technology depends on the mining depth of features, while the existing intelligent method There is still room for improvement in the technical problem of deep feature mining of static laryngoscopy images. This solution creatively adopts an improved convolutional neural network method combined with feature optimization for detection. By combining artificial features, binary features and pre-trained extracted features, and constructing an independently trained convolutional network to further deepen the feature extraction process, it finally combines the attention mechanism and the pre-trained detection model for efficient detection, thereby improving the efficiency and accuracy of static lesion detection. In view of the technical problem that traditional methods are difficult to effectively combine vocal cord dynamic analysis and laryngoscopy image lesion detection in the existing vocal cord dynamic analysis methods of laryngoscopy imaging analysis, this solution creatively adopts a residual improved bidirectional long short-term memory neural network method combined with bidirectional improved temporal convolution to perform vocal cord dynamic analysis. Through the integration and combination of neural networks constructed by two bidirectional information capture mechanisms, the subtle and macro features in the dynamic features of the vocal cords are deeply extracted, thereby effectively reflecting the abnormalities of vocal cord activity, providing a powerful auxiliary reference for static image lesion detection. At the same time, it can also be used as a separate detection function, improving the overall connectivity and reusability of the method.

[0005] The technical solution adopted by the present invention is as follows: The present invention provides a laryngoscope imaging analysis method based on deep learning, the method comprising the following steps:

[0006] Step S1: data collection;

[0007] Step S2: data preprocessing;

[0008] Step S3: static lesion detection;

[0009] Step S4: vocal cord dynamic analysis;

[0010] Step S5: laryngoscope imaging analysis.

[0011] Furthermore, in step S1, the data collection is used to collect the original data set required for laryngoscopic imaging analysis, specifically, by collecting and obtaining the original data set for laryngoscopic imaging analysis from the medical system and clinical database through multi-type data collection and data annotation;

[0012] The laryngoscope imaging analysis original data set specifically includes a static laryngoscope image set, a vocal cord dynamic video sequence set and a patient basic information set.

[0013] Further, in step S2, the data preprocessing is used to enhance and optimize the original data, specifically to perform multimodal data optimization processing on the original data set of laryngoscope imaging analysis to obtain an optimized data set of laryngoscope imaging analysis, which specifically includes the following steps:

[0014] Step S21: data cleaning processing, specifically, performing duplicate data removal, missing data processing and imaging quality detection processing operations on the laryngoscope imaging analysis original data set in sequence to obtain a cleaned processing data set;

[0015] Step S22: data enhancement processing, specifically performing an image enhancement operation on the static laryngoscope image set and performing a video enhancement operation on the vocal cord dynamic video sequence set to obtain an enhanced data set;

[0016] Step S23: data normalization processing, specifically performing image normalization and video frame alignment operations on the enhanced data set to obtain a normalized data set;

[0017] Step S24: preliminary feature extraction, specifically using a pre-trained standard convolutional neural network to perform feature extraction on the image data in the normalized data set to obtain image feature data, and using a time series analysis method to perform feature extraction on the video data in the normalized data set to obtain motion feature data, and perform feature integration to obtain a laryngoscope imaging feature data set;

[0018] Step S25: data set division, specifically dividing the laryngoscope imaging feature data set and the normalized data set into a training set, a validation set and a test set for model training and debugging;

[0019] Step S26: pre-processing data, specifically, pre-processing the data through the data cleaning process, the data enhancement process, the data normalization process, the preliminary feature extraction and the data set division to obtain a laryngoscope imaging analysis optimization data set, and storing the laryngoscope imaging analysis optimization data set for data reading, writing and updating.

[0020] Further, in step S3, the static lesion detection is used to preliminarily identify and classify the lesion information of the laryngoscope static image, specifically, based on the static image data set in the laryngoscope imaging analysis optimization data set, using an improved convolutional neural network method combined with feature optimization to perform static lesion detection to obtain static lesion recognition reference data, specifically including the following steps:

[0021] Step S31: pre-trained feature detection and extraction, specifically using a pre-trained Yolov8 model to perform feature extraction on a static image data set in the laryngoscope imaging analysis optimization data set to obtain pre-trained feature data;

[0022] Step S32: extracting local binary pattern features, specifically using local binary patterns to extract texture features in the static image data set in the laryngoscope imaging analysis optimization data set to obtain binary texture feature data;

[0023] Step S33: feature integration, specifically, using the Hadamard product fusion method to perform feature fusion on the pre-trained feature data, the binary texture feature data and the image feature data in the laryngoscope imaging feature data set to obtain integrated image feature data;

[0024] Step S34: constructing a convolutional neural network, specifically constructing a standard convolutional neural network including a convolutional layer, a pooling layer and a fully connected layer, receiving the integrated image feature data as data input, performing deep feature extraction, and connecting an attention module after the output layer of the standard convolutional neural network to perform feature representation enhancement, and connecting the output of the attention module to a pre-trained Yolov8 target detection model to perform lesion static detection to obtain lesion static detection information;

[0025] Step S35: loss improvement optimization, specifically constructing a weighted integrated loss function as the overall loss function of the model to optimize model training. The calculation formula of the weighted integrated loss function is:

[0026] ;

[0027] Where, L total is the weighted ensemble loss function, is the bounding box loss weight, L bbox is the bounding box loss function, which is calculated using the smooth L1 norm. is the class loss weight, L class is the category loss function, which is calculated using the cross entropy loss function. is the local binary loss weight, L LBP is a local binary loss function, which is calculated using mean square error. is the attention loss weight, Latt is the attention loss function, which is calculated using KL divergence;

[0028] Step S36: Static lesion detection model training, specifically, training the static lesion detection model through the pre-training feature detection extraction, the local binary pattern feature extraction, the feature integration and the loss improvement optimization to obtain the static lesion detection model Model VD ;

[0029] Step S37: Static lesion detection, specifically using the lesion static detection model Model VD , based on the static image data set in the laryngoscope imaging analysis optimization data set, static lesion detection is performed to obtain static lesion recognition reference data.

[0030] Further, in step S4, the vocal cord dynamic analysis is used to evaluate the motor function of the vocal cords and provide auxiliary analysis information, specifically, based on the laryngoscope imaging analysis optimization data set, a residual improved bidirectional long short-term memory neural network method combined with a bidirectional improved temporal convolution is used to perform vocal cord dynamic analysis to obtain vocal cord dynamic evaluation reference data, specifically including the following steps:

[0031] Step S41: construct a bidirectional improved temporal convolutional subnet, specifically, introduce a bidirectional information capture mechanism on the basis of the standard temporal convolutional network, and construct a bidirectional improved temporal convolutional subnet, and process the bidirectional feature information in sequence through one-dimensional residual convolution and red connection layer to obtain the deep temporal vocal cord dynamic features, and the calculation formula is:

[0032] ;

[0033] In the formula, Z BTCN is the deep temporal vocal cord dynamic feature, BatchNormal(·) is the pooling connection layer function, f(·) is the classifier activation function, specifically the S-type activation function, W is the temporal convolution weight, 1DConv(·) is the one-dimensional residual convolution function, y t is the bidirectional improved temporal convolution output, b is the temporal convolution bias term, is the forward temporal convolution function, is the backward temporal convolution function, is the matrix addition operator, X t It is the raw data input of the bidirectional improved temporal convolutional subnetwork, which is used to represent the motion feature data and vocal cord dynamic video sequence set in the laryngoscope imaging analysis optimization dataset;

[0034] Step S42: constructing a bidirectional long short-term memory subnet, specifically constructing a standard bidirectional long short-term memory neural network for optimizing the feature depth processing of time series data, and introducing a multi-head self-attention mechanism to process the deep temporal vocal cord dynamic features, and performing one-dimensional convolution layer processing on the output of the standard bidirectional long short-term memory neural network and the output of the multi-head self-attention mechanism through feature splicing to obtain deep integrated dynamic features, constructing the bidirectional long short-term memory subnet, and classifying the deep integrated dynamic features by constructing a softmax classifier to obtain the output data of the bidirectional long short-term memory subnet;

[0035] Step S43: Vocal cord dynamic analysis model training, specifically, performing vocal cord dynamic analysis model training through the bidirectional improved temporal convolution subnet and the bidirectional long short-term memory subnet to obtain the vocal cord dynamic analysis model Model VA ;

[0036] Step S44: vocal cord dynamic analysis, specifically using the vocal cord dynamic analysis model Model VA , vocal cord dynamic analysis is performed based on the laryngoscope imaging analysis optimization data set to obtain vocal cord dynamic evaluation reference data.

[0037] Furthermore, in step S5, the laryngoscopic imaging analysis is used to integrate the static and dynamic analysis results to perform an integrated analysis of laryngoscopic imaging, specifically by combining the static lesion identification reference data and the vocal cord dynamic assessment reference data to perform a dynamic and static collaborative analysis of laryngoscopic imaging to obtain laryngoscopic imaging analysis reference data.

[0038] The present invention provides a laryngoscope imaging and analysis system based on deep learning, comprising a data acquisition module, a data preprocessing module, a lesion static detection module, a vocal cord dynamic analysis module and a laryngoscope imaging and analysis module;

[0039] The data acquisition module is used for data acquisition, and obtains a raw data set for laryngoscope imaging analysis through data acquisition, and sends the raw data set for laryngoscope imaging analysis to a data preprocessing module;

[0040] The data preprocessing module is used for data preprocessing, and obtains a laryngoscope imaging analysis optimization data set through data preprocessing, and sends the laryngoscope imaging analysis optimization data set to the lesion static detection module and the vocal cord dynamic analysis module;

[0041] The lesion static detection module is used for lesion static detection, obtains static lesion identification reference data through lesion static detection, and sends the static lesion identification reference data to the laryngoscope imaging analysis module;

[0042] The vocal cord dynamic analysis module is used for vocal cord dynamic analysis, obtains vocal cord dynamic evaluation reference data through vocal cord dynamic analysis, and sends the vocal cord dynamic evaluation reference data to the laryngoscope imaging analysis module;

[0043] The laryngoscope imaging analysis module is used for laryngoscope imaging analysis, and obtains laryngoscope imaging analysis reference data through laryngoscope imaging analysis.

[0044] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0045] (1) In the existing laryngoscope imaging analysis methods, the existing intelligent laryngoscope imaging analysis technology is mainly used for analyzing and annotating static images, and is directly used in diagnosis or research. However, when laryngoscope imaging analysis uses static images alone, it is easy to ignore lesions or potential risks that are difficult to find through intuitive observation, which leads to insufficient analysis depth of laryngoscope imaging analysis and cannot meet the objective technical needs. This solution creatively adopts two mutually supporting functional designs: static lesion detection and dynamic vocal cord analysis. The static images are used to detect the lesion area intuitively, and the dynamic vocal cord analysis is used to assist and support it. At the same time, the two functions are modularly differentiated and can be used as separate functions or in combination, which improves the overall automation of the method and the comprehensive usability of the system.

[0046] (2) In the existing static lesion detection methods of laryngoscope imaging analysis, there is a technical problem that the analysis depth of the existing technology for static images depends on the depth of feature mining, and the existing intelligent methods still have room for improvement in the deep feature mining of static laryngoscope images. This solution creatively adopts an improved convolutional neural network method combined with feature optimization for detection. By combining artificial features, binary features and pre-trained extracted features, and constructing an independently trained convolutional network to further deepen the feature extraction process, it finally combines the attention mechanism and the pre-trained detection model for efficient detection, thereby improving the efficiency and accuracy of static lesion detection.

[0047] (3) In view of the technical problem that traditional methods cannot effectively combine vocal cord dynamic analysis and laryngoscope image lesion detection in the existing vocal cord dynamic analysis methods of laryngoscopic imaging analysis, this scheme creatively adopts the residual improved bidirectional long short-term memory neural network method combined with bidirectional improved temporal convolution to perform vocal cord dynamic analysis. By integrating and combining the neural networks constructed by two bidirectional information capture mechanisms, the subtle and macro features of the vocal cord dynamic characteristics are deeply extracted, thereby effectively reflecting the abnormalities of vocal cord activity, providing a powerful auxiliary reference for static image lesion detection. At the same time, it can also be used as a separate detection function, which improves the overall connectivity and reusability of the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of a process flow of a laryngoscope imaging analysis method based on deep learning provided by the present invention;

[0049] Figure 2 A schematic diagram of a laryngoscope imaging analysis system based on deep learning provided by the present invention;

[0050] Figure 3 This is a schematic diagram of the process of data preprocessing in step S2;

[0051] Figure 4 This is a schematic diagram of the process of static lesion detection in step S3;

[0052] Figure 5 Schematic diagram of the process of vocal cord dynamic analysis in step S4.

[0053] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0054] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0055] In the description of the present invention, it should be understood that terms such as “upper”, “lower”, “front”, “back”, “left”, “right”, “top”, “bottom”, “inside” and “outside” indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0056] Example 1, see Figure 1 The present invention provides a laryngoscope imaging analysis method based on deep learning, which comprises the following steps:

[0057] Step S1: data collection;

[0058] Step S2: data preprocessing;

[0059] Step S3: static lesion detection;

[0060] Step S4: vocal cord dynamic analysis;

[0061] Step S5: laryngoscope imaging analysis.

[0062] By performing the above operations, it is found that in the existing laryngoscope imaging analysis methods, the existing laryngoscope imaging analysis intelligent technology is mostly used to analyze and annotate static images, and is directly used in diagnosis or research. However, when laryngoscope imaging analysis simply uses static images, it is easy to ignore lesions or potential risks that are difficult to find through intuitive observation, which leads to insufficient analysis depth of laryngoscope imaging analysis and cannot meet the objective technical needs. This solution creatively adopts two mutually supporting functional designs of static lesion detection and dynamic vocal cord analysis. Static images are used to detect intuitive lesion areas, and vocal cord dynamic analysis is used to assist and support them. At the same time, the two functions adopt a modular differentiated design, which can be used as separate functions or in combination, thereby improving the overall automation of the method and the comprehensive usability of the system.

[0063] Example 2, see Figure 1 and Figure 2 In step S1, the data collection is used to collect the original data set required for laryngoscopic imaging analysis, specifically, by collecting and obtaining the original data set for laryngoscopic imaging analysis from the medical system and clinical database through multi-type data collection and data annotation;

[0064] The laryngoscope imaging analysis original data set specifically includes a static laryngoscope image set, a vocal cord dynamic video sequence set and a patient basic information set;

[0065] The static laryngoscope image set specifically includes normal vocal cord laryngoscope images and abnormal vocal cord laryngoscope images;

[0066] The normal vocal cord laryngoscope image is used as control group data;

[0067] The abnormal vocal cord laryngoscopic images specifically include malignant tumor images, benign lesion images, inflammatory lesion images and functional abnormality images;

[0068] The malignant tumor image specifically refers to a laryngeal cancer image; the benign lesion image specifically includes a polyp and a cyst image; the inflammatory lesion image specifically refers to a laryngitis image; the functional abnormality image specifically refers to a vocal cord paralysis image;

[0069] The vocal cord dynamic video sequence set specifically includes normal phonation video records and abnormal phonation records of the patient, and synchronously records the phonation audio data;

[0070] The normal occurrence video record is used as control group data;

[0071] The abnormal vocalization record of the patient specifically refers to a vocal cord movement video of a patient with diseased vocal cords in a vocalization state; the video duration range of the vocal cord dynamic video sequence set specifically ranges from 5 seconds to 10 seconds;

[0072] The data annotation specifically includes static image annotation and dynamic audio and video annotation;

[0073] The static image annotation specifically refers to manually annotating the lesion type, lesion location and severity of each static image;

[0074] The dynamic audio and video annotation specifically refers to manually annotating the important frames of each video and each audio, and the frequency characteristics and amplitude characteristics of the vocal cord movement;

[0075] The basic patient information set, specifically including the patient's age, gender and medical history information, is used to assist the model in identifying potential differential characteristics of patient groups.

[0076] Example 3, see Figure 1 , Figure 2 and Figure 3 This embodiment is based on the above embodiment. In step S2, the data preprocessing is used to enhance and optimize the original data, specifically to perform multimodal data optimization processing on the original data set of laryngoscope imaging analysis to obtain an optimized data set of laryngoscope imaging analysis, which specifically includes the following steps:

[0077] Step S21: data cleaning processing, specifically, performing duplicate data removal, missing data processing and imaging quality detection processing operations on the laryngoscope imaging analysis original data set in sequence to obtain a cleaned processing data set;

[0078] The deduplication of data refers to manually checking and deleting duplicate images and audio and video data;

[0079] The processing of missing data specifically refers to deleting missing basic patient information;

[0080] The imaging quality detection process specifically refers to removing blurry, poor quality or damaged images and audio and video;

[0081] Step S22: data enhancement processing, specifically performing an image enhancement operation on the static laryngoscope image set and performing a video enhancement operation on the vocal cord dynamic video sequence set to obtain an enhanced data set;

[0082] The image enhancement operation specifically includes random rotation, scaling, flipping operations, brightness and contrast adjustment operations, and random noise addition operations;

[0083] The video enhancement operation specifically includes key frame extraction, random video translation and cropping, and random blur addition operations;

[0084] Step S23: data normalization processing, specifically performing image normalization and video frame alignment operations on the enhanced data set to obtain a normalized data set;

[0085] Step S24: preliminary feature extraction, specifically using a pre-trained standard convolutional neural network to perform feature extraction on the image data in the normalized data set to obtain image feature data, and using a time series analysis method to perform feature extraction on the video data in the normalized data set to obtain motion feature data, and perform feature integration to obtain a laryngoscope imaging feature data set;

[0086] Step S25: data set division, specifically dividing the laryngoscope imaging feature data set and the normalized data set into a training set, a validation set and a test set for model training and debugging;

[0087] Step S26: pre-processing data, specifically, pre-processing the data through the data cleaning process, the data enhancement process, the data normalization process, the preliminary feature extraction and the data set division to obtain a laryngoscope imaging analysis optimization data set, and storing the laryngoscope imaging analysis optimization data set for data reading, writing and updating.

[0088] Example 4, see Figure 1 , Figure 2 and Figure 4 This embodiment is based on the above embodiment. In step S3, the static lesion detection is used to preliminarily identify and classify the lesion information of the laryngoscope static image. Specifically, based on the static image data set in the laryngoscope imaging analysis optimization data set, an improved convolutional neural network method combined with feature optimization is used to perform static lesion detection to obtain static lesion recognition reference data. Specifically, the following steps are included:

[0089] Step S31: pre-trained feature detection and extraction, specifically using a pre-trained Yolov8 model to perform feature extraction on a static image data set in the laryngoscope imaging analysis optimization data set to obtain pre-trained feature data;

[0090] Step S32: extracting local binary pattern features, specifically using local binary patterns to extract texture features in the static image data set in the laryngoscope imaging analysis optimization data set to obtain binary texture feature data, and the calculation formula is:

[0091] ;

[0092] Where LBP(·) is the local binarization function, x is the horizontal pixel index of the image in the static image dataset, y is the vertical pixel index of the image in the static image dataset, P is the total number of neighborhood pixels of the image in the static image dataset, p is the neighborhood pixel index, and s(·) is the indicator function used to indicate the Returns 1 if yes, otherwise returns 0. is the domain pixel value of the image in the static image dataset, is the central pixel value of the image in the static image dataset;

[0093] Step S33: feature integration, specifically, using the Hadamard product fusion method to perform feature fusion on the pre-trained feature data, the binary texture feature data and the image feature data in the laryngoscope imaging feature data set to obtain integrated image feature data;

[0094] Step S34: constructing a convolutional neural network, specifically constructing a standard convolutional neural network including a convolutional layer, a pooling layer and a fully connected layer, receiving the integrated image feature data as data input, performing deep feature extraction, and connecting an attention module after the output layer of the standard convolutional neural network to perform feature representation enhancement, and connecting the output of the attention module to a pre-trained Yolov8 target detection model to perform lesion static detection to obtain lesion static detection information;

[0095] Step S35: loss improvement optimization, specifically constructing a weighted integrated loss function as the overall loss function of the model to optimize model training. The calculation formula of the weighted integrated loss function is:

[0096] ;

[0097] Where, L total is the weighted ensemble loss function, is the bounding box loss weight, L bbox is the bounding box loss function, which is calculated using the smooth L1 norm. is the class loss weight, L class is the category loss function, which is calculated using the cross entropy loss function. is the local binary loss weight, L LBP is a local binary loss function, which is calculated using mean square error. is the attention loss weight, L att is the attention loss function, which is calculated using KL divergence;

[0098] Step S36: Static lesion detection model training, specifically, training the static lesion detection model through the pre-training feature detection extraction, the local binary pattern feature extraction, the feature integration and the loss improvement optimization to obtain the static lesion detection model Model VD ;

[0099] Step S37: Static lesion detection, specifically using the lesion static detection model Model VD, based on the static image data set in the laryngoscope imaging analysis optimization data set, static lesion detection is performed to obtain static lesion recognition reference data.

[0100] By performing the above operations, in view of the technical problem that in the existing static lesion detection method of laryngoscope imaging analysis, the analysis depth of the existing technology for static images depends on the feature mining depth, and the existing intelligent methods still have room for improvement in the deep feature mining of static laryngoscope images, this solution creatively adopts an improved convolutional neural network method combined with feature optimization for detection, combines artificial features, binary features and pre-trained extracted features, and constructs an independently trained convolutional network to further deepen the feature extraction process, and finally combines the attention mechanism and the pre-trained detection model for efficient detection, thereby improving the efficiency and accuracy of static lesion detection.

[0101] Example 5, see Figure 1 , Figure 2 and Figure 5 This embodiment is based on the above embodiment. In step S4, the vocal cord dynamic analysis is used to evaluate the motor function of the vocal cord and provide auxiliary analysis information. Specifically, based on the laryngoscope imaging analysis optimization data set, a residual improved bidirectional long short-term memory neural network method combined with a bidirectional improved temporal convolution is used to perform vocal cord dynamic analysis to obtain vocal cord dynamic evaluation reference data. Specifically, the following steps are included:

[0102] Step S41: construct a bidirectional improved temporal convolutional subnet, specifically, introduce a bidirectional information capture mechanism on the basis of the standard temporal convolutional network, and construct a bidirectional improved temporal convolutional subnet, and process the bidirectional feature information in sequence through one-dimensional residual convolution and red connection layer to obtain the deep temporal vocal cord dynamic features, and the calculation formula is:

[0103] ;

[0104] In the formula, Z BTCN is the deep temporal vocal cord dynamic feature, BatchNormal(·) is the pooling connection layer function, f(·) is the classifier activation function, specifically the S-type activation function, W is the temporal convolution weight, 1DConv(·) is the one-dimensional residual convolution function, y t is the bidirectional improved temporal convolution output, b is the temporal convolution bias term, is the forward temporal convolution function, is the backward temporal convolution function, is the matrix addition operator, X t It is the raw data input of the bidirectional improved temporal convolutional subnetwork, which is used to represent the motion feature data and vocal cord dynamic video sequence set in the laryngoscope imaging analysis optimization dataset;

[0105] Step S42: constructing a bidirectional long short-term memory subnet, specifically constructing a standard bidirectional long short-term memory neural network for optimizing the feature depth processing of time series data, and introducing a multi-head self-attention mechanism to process the deep temporal vocal cord dynamic features, and performing one-dimensional convolution layer processing on the output of the standard bidirectional long short-term memory neural network and the output of the multi-head self-attention mechanism through feature splicing to obtain deep integrated dynamic features, constructing the bidirectional long short-term memory subnet, and classifying the deep integrated dynamic features by constructing a softmax classifier to obtain the output data of the bidirectional long short-term memory subnet;

[0106] Step S43: Vocal cord dynamic analysis model training, specifically, performing vocal cord dynamic analysis model training through the bidirectional improved temporal convolution subnet and the bidirectional long short-term memory subnet to obtain the vocal cord dynamic analysis model Model VA ;

[0107] Step S44: vocal cord dynamic analysis, specifically using the vocal cord dynamic analysis model Model VA , vocal cord dynamic analysis is performed based on the laryngoscope imaging analysis optimization data set to obtain vocal cord dynamic evaluation reference data.

[0108] By performing the above operations, in view of the technical problem that traditional methods in the existing laryngoscope imaging analysis of vocal cord dynamic analysis methods are difficult to effectively combine vocal cord dynamic analysis and laryngoscope image lesion detection, this scheme creatively adopts a residual improved bidirectional long short-term memory neural network method combined with a bidirectional improved temporal convolution to perform vocal cord dynamic analysis. Through the integration and combination of neural networks constructed by two bidirectional information capture mechanisms, the subtle and macro features of the vocal cord dynamic characteristics are deeply extracted, thereby effectively reflecting the abnormalities of vocal cord activity, providing a powerful auxiliary reference for static image lesion detection. At the same time, it can also be used as a separate detection function, which improves the overall connectivity and reusability of the method.

[0109] Embodiment 7, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S5, the laryngoscope imaging analysis is used to integrate the static and dynamic analysis results to perform an integrated analysis of the laryngoscope imaging. Specifically, the laryngoscope imaging dynamic and static collaborative analysis is performed by combining the static lesion identification reference data and the vocal cord dynamic evaluation reference data to obtain the laryngoscope imaging analysis reference data.

[0110] Table 1 Schematic table of contents of laryngoscope imaging analysis reference data

[0111]

[0112] Table 1 is a schematic table of the contents of the laryngoscope imaging analysis reference data. As shown in the table, the static lesion recognition reference data specifically includes lesion location, lesion category, lesion area, lesion confidence and binary texture features;

[0113] The lesion position is specifically represented by a four-tuple bounding box coordinate (X, Y, W, H), where X is a horizontal coordinate value, Y is a vertical coordinate value, W is a bounding box area width, and H is a bounding box area height;

[0114] The vocal cord dynamic evaluation reference data specifically includes symmetry index, vibration frequency, vibration amplitude and vocal cord closure state;

[0115] The laryngoscope imaging analysis comprehensive reference data specifically includes comprehensive lesion detection results, comprehensive lesion trends, diagnosis auxiliary information, diagnosis auxiliary confidence and abnormality type identification information.

[0116] Embodiment 8, see Figure 1 and Figure 2 , this embodiment is based on the above embodiment, and a laryngoscope imaging analysis system based on deep learning provided by the present invention includes a data acquisition module, a data preprocessing module, a lesion static detection module, a vocal cord dynamic analysis module and a laryngoscope imaging analysis module;

[0117] The data acquisition module is used for data acquisition, and obtains a raw data set for laryngoscope imaging analysis through data acquisition, and sends the raw data set for laryngoscope imaging analysis to a data preprocessing module;

[0118] The data preprocessing module is used for data preprocessing, and obtains a laryngoscope imaging analysis optimization data set through data preprocessing, and sends the laryngoscope imaging analysis optimization data set to the lesion static detection module and the vocal cord dynamic analysis module;

[0119] The lesion static detection module is used for lesion static detection, obtains static lesion identification reference data through lesion static detection, and sends the static lesion identification reference data to the laryngoscope imaging analysis module;

[0120] The vocal cord dynamic analysis module is used for vocal cord dynamic analysis, obtains vocal cord dynamic evaluation reference data through vocal cord dynamic analysis, and sends the vocal cord dynamic evaluation reference data to the laryngoscope imaging analysis module;

[0121] The laryngoscope imaging analysis module is used for laryngoscope imaging analysis, and obtains laryngoscope imaging analysis reference data through laryngoscope imaging analysis.

[0122] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0123] While the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that many changes, modifications, substitutions and variations can be made to the embodiments without departing from the principles and spirit of the invention.

[0124] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A laryngoscope imaging analysis method based on deep learning, characterized in that: The method comprises the following steps: Step S1: data acquisition, acquiring the original data set for laryngoscope imaging analysis; Step S2: data preprocessing to obtain a laryngoscope imaging analysis optimized data set; Step S3: static lesion detection, according to the static image data set in the laryngoscope imaging analysis optimization data set, adopt the improved convolutional neural network method combined with feature optimization to perform static lesion detection, and obtain static lesion recognition reference data, which specifically includes the following steps: step S31: pre-training feature detection extraction; step S32: local binary pattern feature extraction; step S33: feature integration; step S34: constructing a convolutional neural network; step S35: loss improvement optimization; step S36: lesion static detection model training; step S37: lesion static detection; Step S4: vocal cord dynamic analysis, based on the laryngoscope imaging analysis optimization data set, using the residual improved bidirectional long short-term memory neural network method combined with bidirectional improved temporal convolution to perform vocal cord dynamic analysis to obtain vocal cord dynamic evaluation reference data; Step S5: laryngoscope imaging analysis, performing dynamic and static collaborative analysis of laryngoscope imaging to obtain laryngoscope imaging analysis reference data.

2. The laryngoscope imaging analysis method based on deep learning according to claim 1, characterized in that: In step S1, the laryngoscope imaging analysis raw data set specifically includes a static laryngoscope image set, a vocal cord dynamic video sequence set and a patient basic information set.

3. The laryngoscope imaging analysis method based on deep learning according to claim 2, characterized in that: In step S2, the data preprocessing is used to enhance and optimize the original data, specifically to perform multimodal data optimization processing on the original data set of laryngoscope imaging analysis to obtain an optimized data set of laryngoscope imaging analysis, which specifically includes the following steps: Step S21: data cleaning processing, specifically, performing duplicate data removal, missing data processing and imaging quality detection processing operations on the laryngoscope imaging analysis original data set in sequence to obtain a cleaned processing data set; Step S22: data enhancement processing, specifically performing an image enhancement operation on the static laryngoscope image set and performing a video enhancement operation on the vocal cord dynamic video sequence set to obtain an enhanced data set; Step S23: data normalization processing, specifically performing image normalization and video frame alignment operations on the enhanced data set to obtain a normalized data set; Step S24: preliminary feature extraction, specifically using a pre-trained standard convolutional neural network to perform feature extraction on the image data in the normalized data set to obtain image feature data, and using a time series analysis method to perform feature extraction on the video data in the normalized data set to obtain motion feature data, and perform feature integration to obtain a laryngoscope imaging feature data set; Step S25: data set division, specifically dividing the laryngoscope imaging feature data set and the normalized data set into a training set, a validation set and a test set for model training and debugging; Step S26: pre-processing data, specifically, pre-processing the data through the data cleaning process, the data enhancement process, the data normalization process, the preliminary feature extraction and the data set division to obtain a laryngoscope imaging analysis optimization data set, and storing the laryngoscope imaging analysis optimization data set for data reading, writing and updating.

4. The laryngoscope imaging analysis method based on deep learning according to claim 3, characterized in that: In step S3, the static lesion detection is used to preliminarily identify and classify the lesion information of the laryngoscope static image, specifically, based on the static image data set in the laryngoscope imaging analysis optimization data set, an improved convolutional neural network method combined with feature optimization is used to perform static lesion detection to obtain static lesion recognition reference data, specifically including the following steps: Step S31: pre-trained feature detection and extraction, specifically using a pre-trained Yolov8 model to perform feature extraction on a static image data set in the laryngoscope imaging analysis optimization data set to obtain pre-trained feature data; Step S32: extracting local binary pattern features, specifically using local binary patterns to extract texture features in the static image data set in the laryngoscope imaging analysis optimization data set to obtain binary texture feature data; Step S33: feature integration, specifically, using the Hadamard product fusion method to perform feature fusion on the pre-trained feature data, the binary texture feature data and the image feature data in the laryngoscope imaging feature data set to obtain integrated image feature data; Step S34: constructing a convolutional neural network, specifically constructing a standard convolutional neural network including a convolutional layer, a pooling layer and a fully connected layer, receiving the integrated image feature data as data input, performing deep feature extraction, and connecting an attention module after the output layer of the standard convolutional neural network to perform feature representation enhancement, and connecting the output of the attention module to a pre-trained Yolov8 target detection model to perform lesion static detection to obtain lesion static detection information; Step S35: loss improvement optimization, specifically constructing a weighted integrated loss function as the overall loss function of the model to optimize model training. The calculation formula of the weighted integrated loss function is: ； Where, L total is the weighted ensemble loss function, is the bounding box loss weight, L bbox is the bounding box loss function, which is calculated using the smooth L1 norm. is the class loss weight, L class is the category loss function, which is calculated using the cross entropy loss function. is the local binary loss weight, L LBP It is a local binary loss function, which is calculated using mean square error. is the attention loss weight, L att is the attention loss function, which is calculated using KL divergence; Step S36: Static lesion detection model training, specifically, training the static lesion detection model through the pre-training feature detection extraction, the local binary pattern feature extraction, the feature integration and the loss improvement optimization to obtain the static lesion detection model Model VD ; Step S37: Static lesion detection, specifically using the lesion static detection model Model VD , based on the static image data set in the laryngoscope imaging analysis optimization data set, static lesion detection is performed to obtain static lesion recognition reference data.

5. The method for laryngoscope imaging analysis based on deep learning according to claim 4, characterized in that: In step S4, the vocal cord dynamic analysis is used to evaluate the motor function of the vocal cords and provide auxiliary analysis information. Specifically, based on the laryngoscope imaging analysis optimization data set, a residual improved bidirectional long short-term memory neural network method combined with a bidirectional improved temporal convolution is used to perform vocal cord dynamic analysis to obtain vocal cord dynamic evaluation reference data, which specifically includes the following steps: Step S41: constructing a bidirectional improved temporal convolutional subnet, specifically, introducing a bidirectional information capture mechanism on the basis of a standard temporal convolutional network, constructing a bidirectional improved temporal convolutional subnet, and sequentially processing bidirectional feature information through one-dimensional residual convolution and red connection layers to obtain deep temporal vocal cord dynamic features; Step S42: constructing a bidirectional long short-term memory subnet, specifically constructing a standard bidirectional long short-term memory neural network for optimizing the feature depth processing of time series data, and introducing a multi-head self-attention mechanism to process the deep temporal vocal cord dynamic features, and performing one-dimensional convolution layer processing on the output of the standard bidirectional long short-term memory neural network and the output of the multi-head self-attention mechanism through feature splicing to obtain deep integrated dynamic features, constructing the bidirectional long short-term memory subnet, and classifying the deep integrated dynamic features by constructing a softmax classifier to obtain the output data of the bidirectional long short-term memory subnet; Step S43: Vocal cord dynamic analysis model training, specifically, performing vocal cord dynamic analysis model training through the bidirectional improved temporal convolution subnet and the bidirectional long short-term memory subnet to obtain the vocal cord dynamic analysis model Model VA ; Step S44: vocal cord dynamic analysis, specifically using the vocal cord dynamic analysis model Model VA , vocal cord dynamic analysis is performed based on the laryngoscope imaging analysis optimization data set to obtain vocal cord dynamic evaluation reference data.

6. The method for laryngoscope imaging analysis based on deep learning according to claim 5, characterized in that: In step S41, the calculation formula for constructing the bidirectional improved temporal convolutional subnetwork is: ； In the formula, Z BTCN is the deep temporal vocal cord dynamic feature, BatchNormal(·) is the pooling connection layer function, f(·) is the classifier activation function, specifically the S-type activation function, W is the temporal convolution weight, 1DConv(·) is the one-dimensional residual convolution function, y t is the bidirectional improved temporal convolution output, b is the temporal convolution bias term, is the forward temporal convolution function, is the backward temporal convolution function, is the matrix addition operator, X t It is the raw data input of the bidirectional improved temporal convolutional subnetwork, which is used to represent the motion feature data and vocal cord dynamic video sequence set in the laryngoscope imaging analysis optimization dataset.

7. The method for laryngoscope imaging analysis based on deep learning according to claim 6, characterized in that: In step S5, the laryngoscopic imaging analysis is used to integrate the static and dynamic analysis results to perform an integrated analysis of laryngoscopic imaging, specifically by combining the static lesion identification reference data and the vocal cord dynamic assessment reference data to perform a dynamic and static collaborative analysis of laryngoscopic imaging to obtain laryngoscopic imaging analysis reference data.

8. A laryngoscope imaging analysis system based on deep learning, used to implement a laryngoscope imaging analysis method based on deep learning as described in any one of claims 1 to 7, characterized in that: It includes a data acquisition module, a data preprocessing module, a lesion static detection module, a vocal cord dynamic analysis module and a laryngoscope imaging analysis module.

9. The deep learning-based laryngoscope imaging analysis system according to claim 8, characterized in that: The data acquisition module is used for data acquisition, and obtains a raw data set for laryngoscope imaging analysis through data acquisition, and sends the raw data set for laryngoscope imaging analysis to a data preprocessing module; The data preprocessing module is used for data preprocessing, and obtains a laryngoscope imaging analysis optimization data set through data preprocessing, and sends the laryngoscope imaging analysis optimization data set to the lesion static detection module and the vocal cord dynamic analysis module; The lesion static detection module is used for lesion static detection, obtains static lesion identification reference data through lesion static detection, and sends the static lesion identification reference data to the laryngoscope imaging analysis module; The vocal cord dynamic analysis module is used for vocal cord dynamic analysis, obtains vocal cord dynamic evaluation reference data through vocal cord dynamic analysis, and sends the vocal cord dynamic evaluation reference data to the laryngoscope imaging analysis module; The laryngoscope imaging analysis module is used for laryngoscope imaging analysis, and obtains laryngoscope imaging analysis reference data through laryngoscope imaging analysis.

Images