Epistaxis identification method based on multi-scale feature fusion and computer device

Abstract

The invention relates to the field of epistaxis identification, in particular to an epistaxis identification method based on multi-scale feature fusion and a computer device. According to the scheme, the method comprises the steps of extracting local features of an epistaxis image by adopting a full convolutional network, extracting global features by adopting Transform, capturing global context information, performing different scale transformation on a local feature map and a global feature map, generating feature maps of different scales, performing multi-scale feature fusion, performing context fusion, and calculating an attention weight. And finally, outputting final features, then carrying out cross entropy loss calculation and model training, and finally identifying epistaxis through the trained model. Aiming at the problems that a data set is unbalanced and a small bleeding area is easy to ignore, the attention weight is dynamically adjusted in the training process, and the method focuses on difficult negative samples and small sample areas, so that the accuracy is improved. And finally, the accuracy of epistaxis image recognition is improved. The method is suitable for epistaxis identification.

Description

Technical Field

[0001] The present invention relates to the field of nosebleed recognition, and in particular to a nosebleed recognition method and a computer device based on multi-scale feature fusion. Background Art

[0002] With the continuous advancement of medical imaging technology, the application of computer vision in clinical practice has also developed significantly, especially in the analysis of conditions such as epistaxis. Automated image analysis can help medical professionals accurately identify the source of bleeding, assess the severity of the condition, and plan treatment strategies. However, despite the potential benefits, the current challenges in epistaxis image processing still involve multiple technical and clinical issues. One of the key issues is the variability of the presentation of epistaxis cases, especially in terms of severity and potential causes. Secondary epistaxis may be related to trauma, medications (especially anticoagulants), or other health conditions (such as hypertension or blood disorders). This variability makes it difficult for automated systems to consistently and accurately process images for diagnosis and treatment. Coupled with the variations in image quality and lighting conditions during nasal endoscopy, the training datasets used for epistaxis detection and segmentation are often limited in size and diversity. Current epistaxis image analysis has the following problems: Epistaxis image recognition accuracy and limited dataset diversity. Current epistaxis image analysis methods have difficulty in accurate segmentation, especially for small or irregular bleeding areas. Furthermore, differences in image quality, lighting conditions, and limited datasets lead to poor generalization, reducing the accuracy of epistaxis recognition. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a nosebleed recognition method and a computer device based on multi-scale feature fusion, thereby improving the accuracy of nosebleed recognition.

[0004] The present invention adopts the following technical solutions to achieve the above-mentioned purpose. In a first aspect, the present invention provides a method for identifying nosebleeds based on multi-scale feature fusion, comprising: S1. Use a fully convolutional network to extract local features of nosebleed images in the following way: , I represents the input image, represents a fully convolutional network, Represents a local feature map; S2, using Transformer to extract global features and capture global context information; Input the local feature map into Transformer to obtain the global feature. , Represents the global feature map; S3, perform multi-scale feature fusion; The local feature map and the global feature map are transformed into different scales to generate feature maps of different scales; For the local feature map, generate a local multi-scale feature pyramid: , S represents different scales, represents the downsampling operation, Represents a local multi-scale feature map; For the global feature map, generate a global multi-scale feature pyramid: , Represents a global multi-scale feature map; Fuse local and global feature maps of different scales; Splicing and fusion: ; Additive Fusion: , Represents the fused feature map; S4, perform context fusion and output the final features; The contextual attention mechanism is used to assign different weights to local features and global features at each scale; Attention weight calculation: , represents feature fusion, FC represents the fully connected layer, Represents the weight at each scale; Finally, the final features are output: , represents the final feature map; S5, cross entropy loss calculation; , represents the cross entropy loss, represents the model's predicted probability for the correct category, is a balancing factor used to deal with class imbalance. is a tuning parameter, called the focus parameter, which is used to adjust the model's focus on difficult samples; S6, CNN model training; Input the final feature map and calculate the predicted probability of each pixel in the final feature map; Calculate the cross entropy loss according to the cross entropy loss calculation formula, then increase the loss for the set sample and adjust and , update the CNN model weights; Finally, the trained CNN model is used to identify nosebleeds.

[0005] Furthermore, before using the full convolutional network to extract the local features of the nosebleed image, the method also includes: enhancing the nosebleed image and expanding the data set, specifically including: Perform geometric rotation on the nosebleed image as follows: ; Translate the nosebleed image as follows: ,in is the translation distance; Adjust the contrast and brightness of the nosebleed image as follows: ,in Control contrast, Control brightness, is the rotation angle, Respectively represent the coordinates of the original nosebleed image, Respectively represent the enhanced nosebleed image coordinates, represents the enhanced image of nose bleeding; Expand the nosebleed image dataset by generating adversarial networks, including: Generate fake images from random noise z through the generator of the GAN ; At the same time, the real image x and the fake image Input the discriminator of the generated adversarial network and output the judgment result and , represents the authenticity score of the real image x, Represents a fake image Authenticity rating; Optimizing the loss function of the generative adversarial network, the goal of the generator is to minimize the following loss: , represents the loss function of the generator; The goal of the discriminator is to maximize the following loss: , represents the loss function of the discriminator; Finally, the nosebleed image is expanded through the trained generative adversarial network.

[0006] Furthermore, before using the full convolutional network to extract the local features of the nosebleed image, the method further includes: performing illumination normalization processing on the nosebleed image, specifically including: Perform lighting analysis on the input nosebleed image: Calculate the brightness distribution of the nosebleed image and convert the nosebleed image from RGB color space to LAB color space. The conversion formula is as follows: , L is the brightness channel, A and B are the color channels respectively; Calculate the brightness histogram: Perform histogram analysis on the brightness channel L to obtain the illumination distribution in the nosebleed image in the following way: , where H(L) is the histogram of the brightness channel, Represents the frequency of brightness values ​​in the image; Detecting areas of uneven lighting: Calculate the maximum, minimum, and mean brightness of the nosebleed image as follows: ; in is the maximum brightness value in the image, is the minimum brightness value, is the average brightness of the image; Processing of overly bright and dark areas: For overly bright areas, the brightness is reduced so that the difference between it and the average brightness is within the set range, as follows: ,in It is an adjustment parameter that controls the amplitude of brightness adjustment. Indicates the adjusted brightness value; For dark areas, the brightness is increased so that the difference between the area and the average brightness is within the set range, as follows: ,in is another adjustment parameter used to enhance the brightness of dark areas; Global illumination normalization: The adjusted brightness channel is normalized as follows: ; in and are the upper and lower limits of the brightness value; Reconstruct the image: The processed brightness channel The A and B channels are recombined with the unchanged ones to form a new LAB image as follows: ; Then convert the LAB image back to RGB color space to get the normalized image as follows: , Represents a normalized image.

[0007] Furthermore, in step S1, when the fully convolutional network uses an adaptive convolution kernel to extract local features of the nosebleed image, the method is as follows: , Indicates the convolution kernel size at the current feature scale; ; and Represent the minimum and maximum sizes of the convolution kernel, respectively. is the change in the gradient of the current feature map, It is a tuning parameter used to control the sensitivity of the convolution kernel adjustment. It is the maximum value of the gradient change and is used for normalization.

[0008] Furthermore, the input image includes an endoscopic image, a CT image, and an MRI image. Before extracting the local features, the method further includes: Align endoscopic images, CT images, and MRI images through rigid and non-rigid registration; Rigid registration is used to solve rotation and translation errors to ensure that the CT image or MRI image is aligned with the endoscopic image. The rigid transformation matrix is ​​as follows: ,in, are the transformed coordinates, is the rotation matrix, is the translation vector, are the input image coordinates; Non-rigid registration is used to handle situations other than rigid registration. The non-rigid registration formula is as follows: ,in, is the weight, is the deformation kernel function, It is a reference point.

[0009] In a second aspect, the present invention provides a computer device, including a memory, wherein the memory stores program instructions, and when the program instructions are run, the above-mentioned nosebleed recognition method based on multi-scale feature fusion is executed.

[0010] The beneficial effects of the present invention are: The present invention adopts a fully convolutional network to extract local feature information of nosebleed images and adopts a Transformer to extract global feature information, thereby improving the accuracy of feature information.

[0011] To address the problems of data set imbalance and small bleeding areas being easily ignored, the present invention uses the Focal Loss function to dynamically adjust the attention weight during the training process, focusing on difficult negative samples and small sample areas, thereby improving the accuracy.

[0012] To address the problem of insufficient diversity in data sets, the present invention expands the size of data sets through image enhancement techniques (such as rotation, translation, random cropping, etc.), and uses generative adversarial networks (GANs) to synthesize new images, thereby generating more nasal bleeding images under different conditions and enhancing the generalization ability of the model.

[0013] Before the image analysis begins, the present invention applies an adaptive illumination normalization algorithm to address the problem of uneven illumination. The algorithm adjusts the illumination differences in the image to make the overly dark or overly bright areas uniform, thus avoiding interference of illumination changes with the detection of bleeding sources. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a flow chart of a nosebleed recognition method based on multi-scale feature fusion provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0015] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0016] The present invention provides a nosebleed recognition method based on multi-scale feature fusion, such as Figure 1 As shown, including: S1, using fully convolutional network to extract local features of nosebleed images; The fully convolutional network is mainly used to extract local details in the image. The bleeding source in the nasal cavity may be small and irregular in shape, so the fully convolutional network can extract details from the image layer by layer and capture local features such as small bleeding sources.

[0017] The specific method is as follows: , I represents the input image, represents a fully convolutional network, Represents a local feature map. In the present invention, Represents a feature map of larger size, where each pixel represents the feature of the local area.

[0018] S2, using Transformer to extract global features; Global feature extraction uses Transformer, which is good at capturing global context information and can understand the structure and semantics of the entire image. Transformer uses the self-attention mechanism to analyze the correlation between different areas in the image, which is particularly suitable for the overall understanding of complex anatomical structures.

[0019] The specific method is as follows: Input the local feature map into Transformer to obtain the global feature. , Representing the global feature map, the self-attention mechanism of Transformer can capture the correlation of distant regions in the image and improve the global understanding of complex structures.

[0020] S3, perform multi-scale feature fusion; The core of multi-scale feature fusion is to combine local features and global features to ensure that the model can process both detail information and overall structure at the same time.

[0021] Feature pyramid construction: Local features extracted from FCN And the global features generated by Transformer After different scale transformations, feature maps of different scales are generated to capture different spatial information.

[0022] For the local feature map, generate a local multi-scale feature pyramid: , S represents different scales, represents the downsampling operation; For the global feature map, generate a global multi-scale feature pyramid: ; Through the above methods, global and local features at different levels can be obtained.

[0023] Feature Fusion: The local and global features of different scales are fused. The present invention mainly uses splicing fusion and supplemented by additive fusion to obtain richer and more significant features.

[0024] Splicing and fusion: ; After splicing, further feature extraction is performed through convolution operation to obtain richer fusion features.

[0025] Additive Fusion: , Represents the fused features; Features of different scales are fused by adding pixel by pixel to enhance feature expression.

[0026] S4, perform context fusion and output the final features; The feature fusion effect is further enhanced by the contextual attention mechanism, which assigns different weights to local features and global features at each scale, ensuring that the features of key areas (bleeding sources) are amplified.

[0027] Attention weight calculation: , represents feature fusion, FC represents the fully connected layer, Represents the weight at each scale; Finally, the final features are output: , represents the final feature map; S5, cross entropy loss calculation; Focal Loss is an improved version of cross entropy loss, which aims to increase the model's attention to difficult samples by reducing the contribution of simple samples to the loss. The formula is as follows: , represents the cross entropy loss, represents the model's predicted probability for the correct category, is a balancing factor used to deal with class imbalance. is a tuning parameter, called the focusing parameter, which is used to adjust the model's attention to difficult samples.

[0028] Class imbalance processing: By introducing the balancing factor ,Focal Loss gives more weight to the small sample (small bleeding source) categories, ensuring that they are not ignored during training.

[0029] Difficult Sample Enhancement: By adjusting the parameters , when the model predicts a low probability for a sample (i.e., a sample that is difficult to predict, often a small bleeding source), The loss value will be enlarged, forcing the model to pay more attention to these difficult-to-detect areas. On the contrary, for those samples that are easy to detect, the loss value is reduced to reduce over-training on simple samples.

[0030] Flexibility to accommodate different bleed areas: As training progresses, the model will gradually increase the weights of those difficult-to-detect bleeding areas (such as bleeding spots with low contrast or irregular morphology), and ultimately enhance the model's sensitivity to these areas.

[0031] S6. CNN model training: Input the final feature map and calculate the predicted probability of each pixel in the final feature map; Calculate the cross entropy loss according to the cross entropy loss calculation formula, then increase the loss for the set sample and adjust and , update the CNN model weights; Finally, the trained CNN model is used to identify nosebleeds.

[0032] The present invention enhances nosebleed images by performing geometric transformation, color adjustment and other operations on existing data to generate more diverse training data. This helps to expand the scale of the data set and improve the generalization ability of the model. The specific steps are as follows: Perform geometric rotation on the nosebleed image as follows: ; Translate the nosebleed image as follows: ,in is the translation distance, is the rotation angle, Respectively represent the coordinates of the original nosebleed image, Respectively represent the enhanced nosebleed image coordinates, represents the enhanced image of nose bleeding; Adjust the contrast and brightness of the nosebleed image as follows: ,in Control contrast, Control brightness; Generative adversarial networks are a type of generative model that generates high-quality synthetic images through adversarial training of two networks (generator and discriminator).

[0033] Generator: Responsible for generating nasal bleeding images similar to real images from random noise.

[0034] Discriminator: Responsible for distinguishing generated images from real images.

[0035] The goal of the generative adversarial network is to make the images generated by the generator more and more realistic, and eventually be able to deceive the discriminator.

[0036] Expand the nosebleed image dataset by generating adversarial networks, including: Generate fake images from random noise z through the generator of the GAN ; At the same time, the real image x and the fake image Input the discriminator of the generated adversarial network and output the judgment result and , represents the authenticity score of the real image x, Represents a fake image Authenticity rating; Optimizing the loss function of the generative adversarial network, the goal of the generator is to minimize the following loss: , represents the loss function of the generator; The goal of the discriminator is to maximize the following loss: , represents the loss function of the discriminator; Finally, the nosebleed image is expanded through the trained generative adversarial network.

[0037] The present invention applies an adaptive illumination normalization algorithm to solve the problem of uneven illumination. By adjusting the illumination difference in the image, the overly dark or overly bright areas are made uniform, thereby avoiding interference of illumination changes on bleeding source detection.

[0038] The illumination normalization algorithm is used to deal with the problem of uneven illumination in images, especially in nasal endoscope images. Due to the complex light conditions, there are often too bright or too dark areas, which affect the subsequent detection of bleeding sources. The purpose of illumination normalization is to adjust the illumination differences in the image to make the overall illumination of the image uniform, thereby improving image quality and enhancing detection accuracy.

[0039] Perform illumination normalization on the nosebleed image, including: Perform lighting analysis on the input nosebleed image: Calculate the brightness distribution of the nosebleed image and convert the nosebleed image from RGB color space to LAB color space. The conversion formula is as follows: , L is the brightness channel, A and B are the color channels respectively; Calculate the brightness histogram: Perform histogram analysis on the brightness channel L to obtain the illumination distribution in the image. The histogram shows the frequency of different brightness values ​​in the image, which helps to find overly bright and dark areas in the image.

[0040] The specific method is as follows: , where H(L) is the histogram of the brightness channel, Represents the frequency of brightness values ​​in the image; Detecting areas of uneven lighting: The histogram can be used to identify which parts of the image are too bright and which parts are too dark. In order to further determine the degree of uneven lighting, the maximum, minimum and average brightness of the image are calculated.

[0041] Calculate the maximum, minimum, and mean brightness of the nosebleed image as follows: ; in is the maximum brightness value in the image, is the minimum brightness value, is the average brightness of the image; Processing of overly bright and dark areas: For overly bright areas, the brightness is reduced so that the difference between it and the average brightness is within the set range, that is, it is close to the average brightness, as follows: ,in It is an adjustment parameter that controls the amplitude of brightness adjustment; For dark areas, the brightness is increased so that the difference between the area and the average brightness is within the set range, that is, it is close to the average brightness, as follows: ,in is another adjustment parameter used to enhance the brightness of dark areas; Global illumination normalization: Normalize the adjusted brightness channel to ensure that the brightness value is within the specified range (set to 0 to 255). This can eliminate the impact of lighting differences and make the brightness distribution of the image more uniform.

[0042] The adjusted brightness channel is normalized as follows: ; in and are the upper and lower limits of the brightness value; Reconstruct the image: The processed brightness channel The A and B channels are recombined with the unchanged ones to form a new LAB image as follows: ; Then convert the LAB image back to RGB color space to get the normalized image as follows: , Represents a normalized image.

[0043] The present invention combines FCN and Transformer to design an adaptive convolution kernel, with the goal of improving the model's ability to extract features from different regions of the complex structure of the nasal cavity. For narrow areas and local details, the adaptive convolution kernel will shrink to capture fine features; for spacious areas, the adaptive convolution kernel will expand to obtain more contextual information. With this model, local details and global semantics can be taken into account, improving the accuracy of locating and detecting bleeding sources.

[0044] Adaptive convolution kernel adjustment The core idea of ​​adaptive convolution kernel is to automatically adjust the size of convolution kernel according to different regions. By analyzing the local features of the image (such as gradient changes), it can be determined whether the size of the convolution kernel needs to be adjusted. The formula for adaptive convolution kernel size is as follows: in: is the convolution kernel size at the current feature scale; and Represent the minimum and maximum sizes of the convolution kernel (set to arrive ); is the change in the gradient of the current feature map; It is a tuning parameter, set to 0.5, which is used to control the sensitivity of convolution kernel adjustment; It is the maximum value of the gradient change and is used for normalization.

[0045] The present invention also provides a method for locating the source of nasal bleeding based on multimodal information fusion.

[0046] Multimodal information fusion technology aims to combine information from different imaging modes (such as endoscopic images, CT, MRI, etc.) to make up for the shortcomings of a single modality in capturing anatomical structure and detail information. By combining the local details of endoscopic images with the global anatomical information of CT and MRI, the accuracy of identifying the source of internal nasal bleeding can be improved. Multimodal information fusion technology can help doctors understand the nasal structure more comprehensively, especially in complex surgical environments.

[0047] Specifically include: 1. Data preprocessing Before multimodal information fusion, images of different modalities (endoscopic images, CT, MRI) need to be aligned to ensure that they are analyzed in the same spatial coordinate system. This is called image registration. The registration process usually requires spatial alignment of the global anatomical information of CT or MRI with the local information of the endoscopic image.

[0048] Rigid registration: used to solve rotation and translation errors to ensure that the CT / MRI image is aligned with the endoscopic image. The rotation angle range is set as The rigid transformation matrix is ​​as follows: ,in, are the transformed coordinates, is the rotation matrix, The translation vector is set to 3px, are the input image coordinates.

[0049] Non-rigid registration: used to handle more complex deformation situations, the present invention uses non-rigid deformation algorithms such as B-spline or thin plate spline (TPS) to solve the elastic differences in anatomical structures. The non-rigid registration formula is as follows: in, is the weight, is the deformation kernel function, It is a reference point.

[0050] 2. Multimodal feature extraction Useful features are extracted from different modalities (endoscopic images, CT, MRI). The features of endoscopic images are mainly detailed features that capture local bleeding areas; the features of CT and MRI are mainly global anatomical structure information.

[0051] Endoscopic images: Use convolutional neural networks to extract local features and generate feature vectors , the convolution kernel size is set to The formula is as follows: in, It is an endoscopic image, and CNN extracts features.

[0052] CT / MRI images: Extract anatomical features through a Transformer-based global feature extraction network to generate feature vectors , the number of attention heads , ensuring that context information is captured effectively. The formula is as follows: in, is a CT or MRI image, Extract global features.

[0053] 3. Multimodal Feature Alignment Useful features are extracted from different modalities (endoscopic images, CT, MRI). The features of endoscopic images are mainly detailed features that capture local bleeding areas; the features of CT and MRI are mainly global anatomical structure information.

[0054] Endoscopic images: Use convolutional neural networks to extract local features and generate feature vectors , the convolution kernel size is set to The formula is as follows: in, It is an endoscopic image, and CNN extracts features.

[0055] CT / MRI images: Extract anatomical features through a Transformer-based global feature extraction network to generate feature vectors , the number of attention heads , ensuring that context information is captured effectively. The formula is as follows: in, is a CT or MRI image, Extract global features.

[0056] Since the features of endoscopic images and CT / MRI images come from different modalities and spatial resolutions, they need to be aligned before fusion. The goal of feature alignment is to ensure that features from different modalities can be compared and fused in the same scale and space.

[0057] Feature alignment: Align the local features of the endoscopic image with the global features of the CT / MRI. Align the feature maps through scale space transformation and convolution operations, with a downsampling step of 2 to ensure that feature maps of different resolutions can be aligned. The formula is as follows: in, is an alignment operation, including feature downsampling or upsampling.

[0058] Fusion strategy: The present invention adopts additive fusion and splicing fusion.

[0059] Additive fusion: Endoscopic features are element-wise added with CT / MRI features to combine local and global features.

[0060] Splicing fusion: Splice the features together. The formula is as follows: It is then further processed by a convolutional layer, where the convolution kernel size is set to The formula is as follows: 4. Modality Weighting and Attention Mechanism After feature fusion, in order to ensure that key modalities (such as anatomical information of CT / MRI or local detail information of endoscope) receive higher weights, the attention mechanism can be used to automatically assign feature weights of different modalities.

[0061] Modality weighting mechanism: Assign weights to features of different modalities through the attention mechanism to emphasize key feature areas.

[0062] in, and are the weights of CT / MRI and endoscopy features respectively. FC is the fully connected layer. Softmax is used to normalize the weights to ensure that the weight of each modality is between 0 and 1.

[0063] Self-attention mechanism: Through the self-attention mechanism, the model can automatically focus on key anatomical structures or local details in a specific modality. The number of heads in the self-attention mechanism is set to 8 to ensure that the model can pay attention to the details and global information of different modalities when fusing features.

[0064] Among them, Q, K, and V are the query matrix, key matrix, and value matrix, respectively. is the dimension of the key matrix.

[0065] 5. Bleeding source location and final feature analysis The fused features It contains local detail features from the endoscope and global anatomical structure information from CT / MRI. Through further convolution layer processing, it can be used to locate the bleeding source inside the nasal cavity. Through classification or regression network, the specific location of the bleeding source is determined. The formula is as follows: Among them, Classifier is a network used for classification or positioning, which can output the location coordinates of the bleeding source. The classifier used is a simple fully connected layer, and the output dimension of the classification network is the coordinates related to the location of the bleeding source (for example, coordinate).

[0066] The invention also provides a nasal bleeding positioning method based on time series analysis.

[0067] When processing continuous frame images, time series analysis algorithms can effectively detect dynamic changes, especially in dynamic scenes in the field of medical images (such as locating the source of nasal bleeding). By modeling continuous image frames in time series, blood flow changes can be detected, and dynamic information such as the location of the bleeding source and the direction of blood flow can be inferred. Specifically, in continuous image frames (such as dynamic videos taken by an endoscope), the present invention hopes to identify dynamic changes in the image through a time series analysis algorithm, thereby determining the blood flow trajectory and the location of the bleeding source.

[0068] Specifically include: 1. Image preprocessing and feature extraction First, extract the continuous frames of the sequence from the video frame. Assume that the continuous frames are After preprocessing, each frame of image is subjected to feature extraction.

[0069] Preprocessing includes: Denoising: Use Gaussian filtering or bilateral filtering to remove noise from each frame.

[0070] Image segmentation: Use FCN or other segmentation networks to extract possible bleeding areas.

[0071] in, is the preprocessed image.

[0072] Feature Extraction: To extract key image features from each frame, the present invention uses a fully convolutional network (FCN) to extract local features, where the convolution kernel size is set to . Eigenvector Represents the feature representation of the image at a certain point in time. The formula is as follows: in, It is The feature vector of the frame.

[0073] 2. Dynamic Change Detection By modeling the image features of continuous frames in time series, the dynamic changes of blood flow can be detected. The present invention adopts three commonly used time series models, namely, autoregressive model, moving average model and autoregressive moving average model. If the time dependency is complex, the present invention will adopt the long short-term memory network, which is also a common deep learning time series.

[0074] Ⅰ. Autoregressive model: In the autoregressive model, the image features at the current moment It can be predicted by the features of the previous moments: in: is the coefficient of the autoregressive model, indicating the past the impact of moments on the present moment; is the autoregressive order, usually adjusted according to the data and set to 2; is the noise term, representing the error.

[0075] Ⅱ. Moving average model: The moving average model considers that the image features at the current moment are affected by the past error terms: in: is the moving average coefficient.

[0076] is the past error term, which represents the impact of previous forecast errors on the current forecast.

[0077] is the moving average order, set to 2.

[0078] III. Autoregressive Moving Average Model: Combining the autoregressive model and the moving average model, the features at the current moment are affected by the features and errors of the previous moments: IV. LSTM (Long Short-Term Memory Network) model: When the temporal dependencies are complex, the LSTM model can be used, whose structure is suitable for both long-term and short-term dependencies.

[0079] in, It is the hidden state of LSTM, representing the memory of all past moments at the current moment, and is set to 128 in the present invention.

[0080] 3. Detection of dynamic changes Through time series modeling, the difference between the features of the current frame and the previous frame can be calculated, thereby detecting the dynamic changes in blood flow.

[0081] in, is the feature change value between the current frame and the previous frame. If the feature change value exceeds a certain threshold, it is considered that the current frame has undergone significant dynamic changes (i.e., nasal blood flow). The change detection formula is as follows: in, Indicates the amplitude of feature change. The threshold needs to be set according to actual data. In the present invention, it is usually set to 0.1 or 0.2.

[0082] 4. Bleeding source location and trajectory prediction By analyzing the sequence of changes over a period of time, the direction and speed of bleeding can be further determined. arrive Obvious changes are detected between the two, and the direction of blood flow can be inferred by estimating the trajectory of the changes.

[0083] The direction formula is as follows: in, and They are Frame and Direction eigenvalue changes.

[0084] The speed formula is as follows: in, is the time frame interval, indicating the speed of blood flow.

[0085] 5. Localization of bleeding source According to the detected blood flow direction and speed, the position of the bleeding source can be reversely inferred, assuming that the position of the bleeding source is the starting point of the dynamic change in consecutive frames.

[0086] The above is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not deviate from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. A nosebleed recognition method based on multi-scale feature fusion, characterized in that: include: S1. Use a fully convolutional network to extract local features of nosebleed images in the following way: , I represents the input image, represents a fully convolutional network, Represents a local feature map; S2, using Transformer to extract global features and capture global context information; Input the local feature map into Transformer to obtain the global feature. , Represents the global feature map; S3, perform multi-scale feature fusion; The local feature map and the global feature map are transformed into different scales to generate feature maps of different scales; For the local feature map, generate a local multi-scale feature pyramid: , S represents different scales, represents the downsampling operation, Represents a local multi-scale feature map; For the global feature map, generate a global multi-scale feature pyramid: , Represents a global multi-scale feature map; Fuse local and global feature maps of different scales; Splicing and fusion: ； Additive Fusion: , Represents the fused feature map; S4, perform context fusion and output the final feature map; Different weights are assigned to local feature maps and global feature maps at each scale through the contextual attention mechanism; Attention weight calculation: , represents feature fusion, FC represents the fully connected layer, Represents the weight at each scale; Finally, the final feature map is output: , represents the final feature map; S5, cross entropy loss calculation; , represents the cross entropy loss, represents the model's predicted probability for the correct category, is a balancing factor used to deal with class imbalance. is a tuning parameter, called the focus parameter, which is used to adjust the model's focus on difficult samples; S6, CNN model training; Input the final feature map and calculate the predicted probability of each pixel in the final feature map; Calculate the cross entropy loss according to the cross entropy loss calculation formula, then increase the loss for the set sample and adjust and , update the CNN model weights; Finally, the trained CNN model is used to identify nosebleeds.

2. The nosebleed recognition method based on multi-scale feature fusion according to claim 1 is characterized in that: Before using the full convolutional network to extract the local features of the nosebleed image, the method also includes: enhancing the nosebleed image and expanding the data set, specifically including: Perform geometric rotation on the nosebleed image as follows: ； Translate the nosebleed image as follows: ,in is the translation distance; Adjust the contrast and brightness of the nosebleed image as follows: ,in Control contrast, Control brightness, is the rotation angle, Respectively represent the coordinates of the original nosebleed image, Respectively represent the enhanced nosebleed image coordinates, represents the enhanced image of nose bleeding; Expand the nosebleed image dataset by generating adversarial networks, including: Generate fake images from random noise z through the generator of the GAN ; At the same time, the real image x and the fake image Input the discriminator of the generated adversarial network and output the judgment result and , represents the authenticity score of the real image x, Represents a fake image Authenticity rating; Optimizing the loss function of the generative adversarial network, the goal of the generator is to minimize the following loss: , represents the loss function of the generator; The goal of the discriminator is to maximize the following loss: , represents the loss function of the discriminator; Finally, the nosebleed image is expanded through the trained generative adversarial network.

3. The nosebleed recognition method based on multi-scale feature fusion according to claim 1 is characterized in that: Before using the full convolutional network to extract the local features of the nosebleed image, the method further includes: performing illumination normalization processing on the nosebleed image, specifically including: Perform lighting analysis on the input nosebleed image: Calculate the brightness distribution of the nosebleed image and convert the nosebleed image from RGB color space to LAB color space. The conversion formula is as follows: , L is the brightness channel, A and B are the color channels respectively; Calculate the brightness histogram: Perform histogram analysis on the brightness channel L to obtain the illumination distribution in the nosebleed image in the following way: , where H(L) is the histogram of the brightness channel, Represents the frequency of brightness values ​​in the image; Detecting areas of uneven lighting: Calculate the maximum, minimum, and mean brightness of the nosebleed image as follows: ； in is the maximum brightness value in the image, is the minimum brightness value, is the average brightness of the image; Processing of overly bright and dark areas: For overly bright areas, the brightness is reduced so that the difference between it and the average brightness is within the set range, as follows: ,in It is an adjustment parameter that controls the amplitude of brightness adjustment. Indicates the adjusted brightness value; For dark areas, the brightness is increased so that the difference between the area and the average brightness is within the set range, as follows: ,in is another adjustment parameter used to enhance the brightness of dark areas; Global illumination normalization: The adjusted brightness channel is normalized as follows: ； in and are the upper and lower limits of the brightness value; Reconstruct the image: The processed brightness channel The A and B channels are recombined with each other to form a new LAB image as follows: ； Then convert the LAB image back to RGB color space to get the normalized image as follows: , Represents a normalized image.

4. The nosebleed recognition method based on multi-scale feature fusion according to claim 1, characterized in that: In step S1, when the fully convolutional network uses an adaptive convolution kernel to extract local features of the nosebleed image, the method is as follows: , Indicates the convolution kernel size at the current feature scale; ； and Respectively represent the minimum and maximum sizes of the convolution kernel, is the change in the current feature map gradient, It is a tuning parameter used to control the sensitivity of the convolution kernel adjustment. is the maximum value of the gradient change and is used for normalization.

5. The nosebleed recognition method based on multi-scale feature fusion according to claim 1, characterized in that: The input image includes an endoscopic image, a CT image, and an MRI image. Before extracting the local features, the method further includes: Align endoscopic images, CT images, and MRI images through rigid and non-rigid registration; Rigid registration is used to solve rotation and translation errors to ensure that the CT image or MRI image is aligned with the endoscopic image. The rigid transformation matrix is ​​as follows: ,in, are the transformed coordinates, is the rotation matrix, is the translation vector, are the input image coordinates; Non-rigid registration is used to handle situations other than rigid registration. The non-rigid registration formula is as follows: ,in, is the weight, is the deformation kernel function, It is a reference point.

6. A computer device comprising a memory, wherein the memory stores program instructions, characterized in that: When the program instructions are executed, the nosebleed recognition method based on multi-scale feature fusion as described in any one of claims 1 to 5 is executed.

Images