Endoscopic video quality evaluation method, system, device and medium based on multi-distortion attribute reasoning

Through the endoscopic video quality evaluation method based on multi-distortion attribute inference, the space-time feature extraction and feature interaction modules are used to solve the complexity and inaccuracy of endoscopic video quality evaluation, improving the accuracy of the evaluation and reducing the Missed diagnosis and misdiagnosis.

CN118485637BActive Publication Date: 2024-11-22SHENZHEN UNIV
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410625761.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-20
Publication Date
2024-11-22
Estimated Expiration
2044-05-20

AI Technical Summary

Technical Problem

The quality evaluation of endoscopic videos is complex and inaccurate. Various factors affect the quality of the video, resulting in misdiagnosis or misdiagnosis, and affecting the treatment effect.

Method used

The endoscopic video quality evaluation method based on multi-distortion attribute inference is adopted. Through the quality evaluation algorithm network, including spatiotemporal feature extraction unit and feature interaction and reasoning unit, the overall quality score of the video and the quality score of each distortion attribute are obtained, and the multi-attributes are used. The feature interaction module and the scoring behavior-guided reasoning module are evaluated.

Benefits of technology

It improves the accuracy of endoscopic video quality evaluation, can reveal the characteristics and rules of video distortion more comprehensively, and reduces the occurrence of missed diagnosis and misdiagnosis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118485637B_ABST
    Figure CN118485637B_ABST
Patent Text Reader

Abstract

The present invention discloses an endoscopic video quality evaluation method, system, device and medium based on multi-distortion attribute reasoning, and relates to the technical field of image / video quality evaluation. The method comprises: inputting the endoscopic video to be tested into a quality evaluation algorithm network for video quality evaluation, and obtaining the quality scores of each distortion attribute and the overall quality score of the video; the quality evaluation algorithm network comprises a spatiotemporal feature extraction unit and a feature interaction and reasoning unit connected by a mapping operation; the spatiotemporal feature extraction unit comprises a video frame perception network, a video segment perception network and a feature fusion module; the video frame perception network and the video segment perception network are both connected to the feature fusion module; the feature interaction and reasoning unit comprises a multi-attribute feature interaction module and a scoring behavior guided reasoning module connected in sequence; the output end of the feature fusion module is connected to the input end of the multi-attribute feature interaction module. The present invention can improve the accuracy of endoscopic video quality evaluation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image / video quality evaluation, and in particular to an endoscope video quality evaluation method, system, device and medium based on multi-distortion attribute reasoning. Background Art

[0002] As of 2020, colorectal cancer has become one of the most deadly cancers worldwide, with the third highest incidence and the second highest mortality rate in the world. This cancer not only threatens the lives of millions of people, but also imposes a heavy burden on society and the medical system. The treatment process of colorectal cancer is complicated and lengthy, and the treatment cost is high, which puts a huge mental and financial pressure on patients and their families. Medical research shows that colorectal diseases such as colorectal inflammation, vascular lesions and polyps are often early signs of colorectal cancer. Therefore, early identification and screening of these symptoms are crucial and can significantly improve the success rate of treatment. In clinical work, endoscopic examination helps doctors to intuitively observe the morphological characteristics of lesions and is considered to be the most effective method to identify these early lesions. However, meticulous endoscopic examinations are highly dependent on experienced physicians and are time-consuming, which greatly affects the efficiency of disease screening. During an endoscopic examination, the doctor inserts the endoscope into the patient's body and uses a handle to accurately navigate to a specific area, such as the stomach, colon, etc. The camera on the endoscope captures real-time video of the organs in the body and transmits it to the display screen for the doctor to observe. Doctors observe the video to look for any abnormal lesions or lesions. However, due to a variety of factors, endoscopic videos often face the problem of low quality. These factors include abnormal brightness caused by the claustrophobic environment of the gastrointestinal tract, frame drops during transmission, and blurring and distortion caused by shooting jitter. According to relevant surveys, about 71.5% of clinical endoscopic data is low-quality. These low-quality videos are prone to missed or misdiagnoses, which in turn affects the treatment effect and prognosis of patients.

[0003] To solve the above problems, the research on endoscopic video quality evaluation methods is particularly urgent and important. However, there are many factors that affect video quality, so in clinical research, we cannot limit ourselves to studying a specific quality distortion. Given the diversity and complexity of these distortion factors, evaluating the quality of endoscopic videos has become a practical and challenging issue. Summary of the invention

[0004] The purpose of the present invention is to provide an endoscopic video quality evaluation method, system, device and medium based on multi-distortion attribute reasoning, which can improve the accuracy of endoscopic video quality assessment.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] An endoscope video quality assessment method based on multi-distortion attribute reasoning, comprising:

[0007] Obtain the endoscope video to be tested;

[0008] The endoscope video to be tested is input into a quality evaluation algorithm network for video quality evaluation to obtain quality scores of various distortion attributes and the overall quality score of the video; the quality evaluation algorithm network includes a spatiotemporal feature extraction unit and a feature interaction and reasoning unit connected by a mapping operation; the spatiotemporal feature extraction unit includes a video frame perception network, a video segment perception network and a feature fusion module; the video frame perception network and the video segment perception network are both connected to the feature fusion module; the feature interaction and reasoning unit includes a multi-attribute feature interaction module and a scoring behavior guided reasoning module connected in sequence; the output end of the feature fusion module is connected to the input end of the multi-attribute feature interaction module.

[0009] Optionally, the endoscope video to be tested is input into a quality evaluation algorithm network for video quality evaluation to obtain quality scores of various distortion attributes and an overall quality score of the video, specifically including:

[0010] First, the endoscope video to be tested is input into the quality evaluation algorithm network to perform continuous sampling and sparse sampling operations, wherein the continuous sampling follows the sampling strategy of FASTER-VQA, and the samples obtained after sampling are then respectively sent to the video segment perception network and the video frame perception network to extract temporal features from the video segment and spatial features from the video frame;

[0011] Then, the two extracted features are fused using the feature fusion module to obtain the shared features;

[0012] Secondly, five nonlinear mapping operations are used to map the shared features to obtain each feature;

[0013] Then, the five features are input into the multi-attribute feature interaction module for adaptive feature interaction to obtain five attribute-specific features. The corresponding connected prediction heads are used to map the attribute-specific features into quality scores of the distorted attributes.

[0014] Finally, the scoring behavior is used to guide the reasoning module to further aggregate and reason the specific features of each distortion attribute to obtain the overall video quality score.

[0015] Optionally, the multi-attribute feature interaction module includes 5 parallel cross-attention blocks; and the scoring behavior guided reasoning module includes two layers of graph convolution layers.

[0016] Optionally, the training process of the quality assessment algorithm network is:

[0017] Acquire training data from a VQA database; the training data includes training videos and corresponding labels; the labels include quality scores of various distortion attributes and an overall quality score of the video;

[0018] Construct an endoscope video quality assessment network based on multi-distortion attribute reasoning;

[0019] The training data is input into the endoscopic video quality assessment network based on multi-distortion attribute reasoning, and training is performed with the goal of minimizing the loss between the output of the endoscopic video quality assessment network based on multi-distortion attribute reasoning and the label, and the trained network is determined as the quality assessment algorithm network.

[0020] Optionally, the loss function used during training is:

[0021]

[0022] Among them, G o is the overall quality label MOS, G n Then it is the quality label MOS of the nth distortion attribute; is the mean square error loss function commonly used in the field of quality assessment; Y o is the overall quality score of the video; Y n is the quality score of the nth distortion attribute.

[0023] The present invention also provides an endoscope video quality evaluation system based on multi-distortion attribute reasoning, comprising:

[0024] A video acquisition module, used to acquire the endoscope video to be tested;

[0025] A video evaluation module is used to input the endoscope video to be tested into a quality evaluation algorithm network for video quality evaluation to obtain the quality scores of each distortion attribute and the overall quality score of the video; the quality evaluation algorithm network includes a spatiotemporal feature extraction unit and a feature interaction and reasoning unit connected by a mapping operation; the spatiotemporal feature extraction unit includes a video frame perception network, a video segment perception network and a feature fusion module; the video frame perception network and the video segment perception network are both connected to the feature fusion module; the feature interaction and reasoning unit includes a multi-attribute feature interaction module and a scoring behavior guided reasoning module connected in sequence; the output end of the feature fusion module is connected to the input end of the multi-attribute feature interaction module.

[0026] The present invention also provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the above-mentioned endoscopic video quality assessment method based on multi-distortion attribute reasoning.

[0027] The present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the endoscopic video quality assessment method based on multi-distortion attribute reasoning as described above.

[0028] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0029] The present invention discloses an endoscope video quality evaluation method, system, device and medium based on multi-distortion attribute reasoning, the method comprising obtaining an endoscope video to be tested; inputting the endoscope video to be tested into a quality evaluation algorithm network for video quality evaluation, and obtaining quality scores of each distortion attribute and the overall quality score of the video; the quality evaluation algorithm network comprises a spatiotemporal feature extraction unit and a feature interaction and reasoning unit connected by a mapping operation; the spatiotemporal feature extraction unit comprises a video frame perception network, a video segment perception network and a feature fusion module; the video frame perception network and the video segment perception network are both connected to the feature fusion module; the feature interaction and reasoning unit comprises a multi-attribute feature interaction module and a scoring behavior guided reasoning module connected in sequence; the output end of the feature fusion module is connected to the input end of the multi-attribute feature interaction module. The present invention can improve the accuracy of endoscope video quality evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0031] Figure 1 This is a framework diagram of an endoscope video quality assessment algorithm based on multi-distortion attribute reasoning in this embodiment;

[0032] Figure 2 Schematic diagram of the structure of the multi-attribute feature interaction module in this embodiment;

[0033] Figure 3 Schematic diagram of the scoring behavior guiding reasoning module structure in this embodiment;

[0034] Figure 4 is a schematic diagram of the correlation matrix between the attributes in this embodiment;

[0035] Figure 5 Schematic diagram of the process of the endoscope video quality evaluation method of the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] The purpose of the present invention is to provide an endoscopic video quality evaluation method, system, device and medium based on multi-distortion attribute reasoning, which can improve the accuracy of endoscopic video quality assessment.

[0038] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0039] like Figure 5 As shown, the present invention provides an endoscope video quality evaluation method based on multi-distortion attribute reasoning, comprising:

[0040] Step 100: Obtain the endoscope video to be tested;

[0041] Step 200: Input the endoscope video to be tested into a quality evaluation algorithm network for video quality evaluation to obtain quality scores of various distortion attributes and an overall quality score of the video; the quality evaluation algorithm network includes a spatiotemporal feature extraction unit and a feature interaction and reasoning unit connected by a mapping operation; the spatiotemporal feature extraction unit includes a video frame perception network, a video segment perception network and a feature fusion module; the video frame perception network and the video segment perception network are both connected to the feature fusion module; the feature interaction and reasoning unit includes a multi-attribute feature interaction module and a scoring behavior guided reasoning module connected in sequence; the output end of the feature fusion module is connected to the input end of the multi-attribute feature interaction module.

[0042] Based on the above technical solution, the following embodiments are provided.

[0043] This embodiment proposes a novel endoscopic video quality assessment algorithm (DARNet) based on multi-distortion attribute reasoning. This method can comprehensively evaluate multiple quality distortion attributes, not only predict the overall quality score of the video, but also infer the specific quality score of each distortion attribute. The algorithm consists of two main parts: a spatiotemporal feature extraction unit and a feature interaction and reasoning unit. The spatiotemporal feature extraction unit is responsible for extracting the quality features of the video in time and space, while the feature interaction and reasoning unit is used to analyze the interaction between different distortion attributes, and predict the quality score of each distortion attribute and the overall quality score of the video.

[0044] Overall algorithm framework:

[0045] like Figure 1 As shown in the figure, the network of the endoscopic video quality assessment algorithm based on multi-distortion attribute inference takes the endoscopic video as input and performs continuous sampling and sparse sampling operations on the video. It is particularly important to point out that the continuous sampling here follows the sampling strategy of FASTER-VQA. The samples obtained after sampling are then sent to the video segment perception network and the video frame perception network respectively. It aims to extract temporal features from video segments and spatial features from video frames. Subsequently, the features extracted by these two networks are combined through a feature fusion module to generate a shared feature P α ; Secondly, five nonlinear mapping operations with similar structures are used to map the shared features into five features Then, the five features are input into the multi-attribute feature interaction module to perform adaptive feature interaction to mine the relevant features between each attribute and obtain five attribute-specific features. Use the prediction head to map attribute-specific features into the quality score Y of the distorted attribute i (i∈{1,2,…,5); Finally, the scoring behavior guided reasoning module is used to further aggregate the five attribute-specific features and infer the overall quality score Y of the video o .

[0046] Spatiotemporal feature extraction unit:

[0047] The spatiotemporal feature extraction unit is a dual-branch network, which includes a video frame perception network (upper branch) and a video segment perception network (lower branch). The video frame perception network is a ResNet50 pre-trained on the IQA database. The parameters of this network remain unchanged during the training process (i.e., frozen). It is responsible for feature extraction of sparsely sampled video frames to obtain spatial features. H and W are the height and width of the video frame respectively. It is worth noting that in this embodiment, the spatial features of each frame are spliced ​​to obtain The video segment perception network is a VideoSwinTransformerTiny pre-trained on the Kinetics-400 dataset. Unlike the previous branch, this network focuses on the processing of continuously sampled video segments and aims to extract temporal features from them. T=16 is half the number of frames in the video clip. Since the channel dimension of spatial features is large, directly fusing such high-dimensional features with temporal features may cause the network to process the two features unbalancedly. To solve this problem, this embodiment uses two linear layers to reduce the dimension of features, and a ReLU activation function is embedded between the two linear layers to effectively reduce the dimension of spatial features. Then, spatial features After dimensionality reduction, it becomes Next, and The input is sent to the fusion module for feature fusion, which consists of two linear layers and a GELU activation function, which is located between the two linear layers. Through this module, spatial features and temporal features are effectively fused to generate quality shared features N = T × H × W. Finally, in order to capture the specificity of each distortion attribute, five nonlinear mapping operations with similar structures are used to transform Converted into 5 features As shown below, where θ i Represents the nonlinear operation of the i-th attribute parameter:

[0048]

[0049] Feature interaction and reasoning unit:

[0050] The quality of endoscopic videos is affected by many distortion attributes, and the degree to which these attributes affect the overall quality of the video varies significantly. These distortion attributes often do not exist in isolation, but interact and coexist. Therefore, in-depth exploration of the interactions between these attributes is not only crucial for accurately evaluating the quality score of each distortion attribute in endoscopic videos, but also can more comprehensively reveal and understand the characteristics and laws of endoscopic video distortion. In view of this, this embodiment proposes a feature interaction and reasoning unit, whose main purpose is to explore the interactive relationship between different attributes, so as to improve the accuracy of quality prediction. Figure 1 As shown, the feature interaction and reasoning unit includes a multi-attribute feature interaction module and a scoring behavior guided reasoning module.

[0051] Multi-attribute feature interaction module:

[0052] Inspired by the excellent ability of the cross-attention mechanism in processing feature interactions from different sources, this paper proposes a multi-attribute feature interaction module based on the cross-attention mechanism. The design goal of this module is to deeply explore the relevant features between different attributes in an adaptive way. Figure 2 As shown, this module will As input, output the unique characteristics of each attribute Specifically, this module contains 5 parallel Cross-Attention (CA) blocks. Take the nth CA block as an example, Through linear mapping get Through linear mapping and Get the Key respectively and exist and Before performing linear mapping, it needs to be added to the position code. Then, the output of the attention module It can be expressed as:

[0053]

[0054] Among them, σ(·) is the SoftMax function, the feature channel d = C, and T represents the matrix transposition operation. For a CA block, by sequentially inputting other features Generate different Key and Value, and get 4 output features. Then, this embodiment compares each output feature with Perform residual connection and input the obtained features into LayerNormalization (LN) and Multi-Layer Perception (MLP), as shown in the following formula:

[0055]

[0056] in, yes The feature obtained by adding the position code. Concatenate in the channel dimension to get the unique features of the nth distortion attribute Finally, this embodiment will Input regression head, which consists of two linear layers and a GELU activation function embedded between the two layers, generates the quality score Y of the nth distortion attribute n .

[0057] Scoring behavior guides reasoning module:

[0058] By analyzing the scoring behavior, this embodiment found significant correlation differences between different distortion attributes, which significantly affected the overall quality assessment. Inspired by the excellent modeling ability of graph convolutional neural networks in processing non-Euclidean data, this embodiment designed a scoring behavior guided reasoning module. This module considers the relationship between the attributes revealed by the data labels, so as to achieve accurate inference of the overall quality score. Figure 3 As shown in Figure 1, the scoring behavior guided reasoning module contains two layers of graph convolutional layers (GCN Layer). Specifically, this module transforms the feature and the adjacency matrix As input, update the features through GCNLayer

[0059]

[0060] Among them, h(·) is LeakyReLU, which is used to increase the nonlinear feature expression ability of the network. is a diagonal matrix, is the learnable mapping parameter matrix of the lth layer. It is worth noting that in order to generate In this embodiment, attribute-specific features Splice and then reshape it Adjacency Matrix It is generated by the following formula:

[0061]

[0062] in, Represents the relationship matrix between the distortion attributes calculated from the data labels by PLCC (Pearson Linear Correlation Coefficient). for It is worth noting that this embodiment is based on Figure 4 Build Specifically, The element value in is set to 1 only when the PLCC value between the corresponding column and row attributes exceeds 0.55. Finally, this embodiment converts the output of the last GCN Layer Input the regression head to map the overall quality score Y of the video o .

[0063] When training the overall network framework, the loss function used is:

[0064]

[0065] Among them, G o is the overall quality label MOS, G n Then it is the quality label MOS of the nth distortion attribute; is the mean square error loss function commonly used in the field of quality assessment; Y o is the overall quality score of the video; Y n is the quality score of the nth distortion attribute.

[0066] Datasets and evaluation metrics:

[0067] This embodiment uses the endoscope quality evaluation dataset MAC-VQA, which contains multiple distortion attribute scores, to evaluate and compare the performance of the proposed method. The dataset comes from the Endoscopy Center of Shenzhen University General Hospital and includes 1,000 endoscope videos with real distortion. Each video contains quality score annotations for five distortion attributes (i.e., spatiotemporal visibility, brightness, specular reflection, stability, and practicality) and an overall quality score annotation. The MAC-VQA dataset is divided into 8:2, and 800 videos are obtained as training datasets and 200 videos are obtained as test datasets. The evaluation indicators used by the network are widely used image / video quality evaluation indicators: SRCC, PLCC, KRCC, and RMSE. It is worth noting that the closer the values ​​of SRCC, PLCC, and KRCC are to 1, the smaller the RMSE value is, indicating that the algorithm has a better prediction effect and can accurately predict videos of various qualities. According to the recommendations of VQEG, before calculating PLCC and RMSE, a four-parameter logistic function is used to map the prediction score s:

[0068]

[0069] Among them, η i (i∈{1,...,4}) are the fitting parameters.

[0070] Experimental setup:

[0071] The network model DARNet in this embodiment is implemented on the PyTorch platform, and all experiments are performed on a workstation equipped with an NVIDIA RTX3090 GPU. In the training phase, the network model uses the Adam optimizer, the initial learning rates of the backbone layer and other layers are 1e-5 and 1e-4 respectively, and the cosine annealing decay strategy is used for decay. The network uses a batch size of 8 and a total of 70 iterations. In the inference phase, for the input video, the sampling strategy mentioned above is used to input the sampled samples into the well-trained model to obtain the predicted quality score.

[0072] Experimental results:

[0073] The network DARNet proposed in this embodiment is compared and analyzed with 9 mainstream video quality evaluation methods: TLVQM, VIDEVAL, VSFA, Simple-VQA, BVQA, FAST-VQA-M, FASTER-VQA, Light-VQ and DOVER. The test results on the MAC-VQA dataset are shown in Table 1. As can be seen from Table 1, the network of this embodiment performs very well in all indicators.

[0074] Table 1 Test results

[0075]

[0076] In addition, the present invention also provides an endoscope video quality evaluation system based on multi-distortion attribute reasoning, comprising:

[0077] A video acquisition module, used to acquire the endoscope video to be tested;

[0078] A video evaluation module is used to input the endoscope video to be tested into a quality evaluation algorithm network for video quality evaluation to obtain the quality scores of each distortion attribute and the overall quality score of the video; the quality evaluation algorithm network includes a spatiotemporal feature extraction unit and a feature interaction and reasoning unit connected by a mapping operation; the spatiotemporal feature extraction unit includes a video frame perception network, a video segment perception network and a feature fusion module; the video frame perception network and the video segment perception network are both connected to the feature fusion module; the feature interaction and reasoning unit includes a multi-attribute feature interaction module and a scoring behavior guided reasoning module connected in sequence; the output end of the feature fusion module is connected to the input end of the multi-attribute feature interaction module.

[0079] The present invention also provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the above-mentioned endoscopic video quality assessment method based on multi-distortion attribute reasoning.

[0080] The present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the endoscopic video quality assessment method based on multi-distortion attribute reasoning as described above.

[0081] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0082] This article uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only used to help understand the core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for evaluating the quality of endoscope videos based on multi-distortion attribute reasoning, characterized in that: include: Obtain the endoscope video to be tested; The endoscope video to be tested is input into a quality evaluation algorithm network for video quality evaluation to obtain quality scores of various distortion attributes and an overall quality score of the video; the quality evaluation algorithm network includes a spatiotemporal feature extraction unit and a feature interaction and reasoning unit connected by a mapping operation; the spatiotemporal feature extraction unit includes a video frame perception network, a video segment perception network and a feature fusion module; the video frame perception network and the video segment perception network are both connected to the feature fusion module; the feature interaction and reasoning unit includes a multi-attribute feature interaction module and a scoring behavior guided reasoning module connected in sequence; the output end of the feature fusion module is connected to the input end of the multi-attribute feature interaction module; The endoscope video to be tested is input into the quality evaluation algorithm network for video quality evaluation to obtain the quality scores of each distortion attribute and the overall quality score of the video, specifically including: First, the endoscope video to be tested is input into the quality evaluation algorithm network to perform continuous sampling and sparse sampling operations, wherein the continuous sampling follows the sampling strategy of FASTER-VQA, and the samples obtained after sampling are then respectively sent to the video segment perception network and the video frame perception network to extract temporal features from the video segment and spatial features from the video frame; Then, the two extracted features are fused using the feature fusion module to obtain the shared features; Secondly, five nonlinear mapping operations are used to map the shared features to obtain each feature; Then, the five features are input into the multi-attribute feature interaction module for adaptive feature interaction to obtain five attribute-specific features. The corresponding connected prediction heads are used to map the attribute-specific features into quality scores of the distorted attributes. Finally, the scoring behavior is used to guide the reasoning module to further aggregate and reason the specific features of each distortion attribute to obtain the overall video quality score; The multi-attribute feature interaction module includes 5 parallel cross-attention blocks; the scoring behavior guided reasoning module includes two layers of graph convolution layers.

2. The method for evaluating endoscope video quality based on multi-distortion attribute reasoning according to claim 1, characterized in that: The training process of the quality assessment algorithm network is: Acquire training data from a VQA database; the training data includes training videos and corresponding labels; the labels include quality scores of various distortion attributes and an overall quality score of the video; Construct an endoscope video quality assessment network based on multi-distortion attribute reasoning; The training data is input into the endoscopic video quality assessment network based on multi-distortion attribute reasoning, and training is performed with the goal of minimizing the loss between the output of the endoscopic video quality assessment network based on multi-distortion attribute reasoning and the label, and the trained network is determined as the quality assessment algorithm network.

3. The method for evaluating endoscope video quality based on multi-distortion attribute reasoning according to claim 2, characterized in that: The loss function used during training is: ; in, is the label MOS of the overall quality, The n The quality label MOS of the distortion attribute; It is the mean square error loss function commonly used in the field of quality assessment; is the overall quality score of the video; For the n The quality score of each distortion attribute.

4. An endoscope video quality assessment system based on multi-distortion attribute reasoning, applying the method as described in any one of claims 1 to 3, characterized in that: include: A video acquisition module, used to acquire the endoscope video to be tested; A video evaluation module is used to input the endoscope video to be tested into a quality evaluation algorithm network for video quality evaluation to obtain the quality scores of each distortion attribute and the overall quality score of the video; the quality evaluation algorithm network includes a spatiotemporal feature extraction unit and a feature interaction and reasoning unit connected by a mapping operation; the spatiotemporal feature extraction unit includes a video frame perception network, a video segment perception network and a feature fusion module; the video frame perception network and the video segment perception network are both connected to the feature fusion module; the feature interaction and reasoning unit includes a multi-attribute feature interaction module and a scoring behavior guided reasoning module connected in sequence; the output end of the feature fusion module is connected to the input end of the multi-attribute feature interaction module.

5. An electronic device, characterized in that: It comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the endoscopic video quality assessment method based on multi-distortion attribute reasoning according to any one of claims 1-3.

6. A computer-readable storage medium, characterized in that: The device stores a computer program, which, when executed by a processor, implements an endoscopic video quality assessment method based on multi-distortion attribute reasoning as described in any one of claims 1 to 3.

Citation Information

Cited By

  • Percutaneous endoscope image feature extraction and analysis method and system

    CN122550394A