Cloud game evaluation method based on space-frequency enhancement and sparse attention fusion and medium

Abstract

The application provides a cloud game evaluation method and medium based on space-frequency enhancement and sparse attention fusion, and belongs to the technical field of image processing. The method comprises the following steps: acquiring a cloud game video stream to be evaluated, and extracting key frames and key region image blocks from the cloud game video stream; inputting the key frames into a pre-trained global feature extraction network to obtain global features, global shallow layer features, global middle layer features and global deep layer features; and inputting the key region image blocks into a pre-trained local feature extraction network to obtain local features, local middle layer features and local deep layer features. The method of the application can precisely capture global scene distortion and local high-sensitive region distortion of a cloud game video by constructing a multi-stage sparse fusion network and a space-frequency multi-level fusion network, while reducing model parameter quantity and calculation cost, meeting real-time inference and terminal lightweight deployment requirements of a cloud game scene.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically relates to a cloud gaming evaluation method and medium based on spatial frequency enhancement and sparse attention fusion. Background Technology

[0002] Cloud gaming, as a real-time interactive online game service model based on cloud computing architecture, is essentially about migrating high-computational tasks such as game rendering to remote servers and transmitting the screen to end users in real time in the form of low-latency video streams. While significantly reducing the hardware threshold of terminal devices, it also introduces unique video quality challenges. Cloud gaming videos are characterized by high dynamism, high content complexity, and special types of distortion. Their real-time interactive characteristics require quality assessment models to not only accurately perceive visual fidelity but also have a lightweight structure to support efficient quality score prediction.

[0003] Most existing video quality assessment methods are designed for general video scenarios and cannot adapt to the special distortion and high dynamic characteristics of cloud gaming videos, resulting in insufficient assessment accuracy. At the same time, existing models mostly adopt heavy network structures with a large number of parameters and high computational complexity, which makes it difficult to meet the needs of real-time inference and lightweight deployment in cloud gaming scenarios and thus cannot achieve efficient application in real cloud gaming scenarios. Summary of the Invention

[0004] To address the problems in the background technology, this invention proposes a cloud gaming evaluation method and medium based on the fusion of spatial frequency enhancement and sparse attention.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, this invention proposes a cloud gaming evaluation method based on the fusion of spatial frequency enhancement and sparse attention, comprising: Acquire the cloud gaming video stream to be evaluated, and extract keyframes and key region image blocks from the cloud gaming video stream; The keyframes are input into a pre-trained global feature extraction network to obtain global features, global shallow features, global mid-level features, and global deep features. The key region image patch is input into a pre-trained local feature extraction network to obtain local features, local mid-level features, and local deep features; The global mid-level features, global deep features, local mid-level features, and local deep features are input into a pre-trained multi-stage sparse fusion network to obtain heterogeneous fusion features. The global shallow features, global mid-level features, and global deep features are input into a pre-trained space-frequency multi-level fusion network to obtain space-frequency enhancement features, gated attention enhancement features, and multi-scale enhancement features. The global features, local features, spatial frequency enhancement features, gated attention enhancement features, multi-scale enhancement features, and heterogeneous fusion features are concatenated to obtain the final fusion feature. The final fused features are mapped to an evaluated quality score.

[0006] Preferably, the step of extracting keyframes from the cloud gaming video stream specifically involves: using an equal-interval sampling strategy to extract several frames from the cloud gaming video stream as keyframes.

[0007] Preferably, extracting key region image blocks from the cloud gaming video stream specifically includes: The keyframes are processed using gradient operators to generate gradient magnitude feature maps; The gradient magnitude feature map is integrally calculated based on a sliding window, and several image blocks with the largest gradient intensity are selected as key region image blocks.

[0008] Preferably, after extracting N frames as keyframes from the cloud gaming video stream using an equal-interval sampling strategy, the method further includes: The keyframes are scaled and padded proportionally.

[0009] Preferably, the global feature extraction network is constructed using ShuffleNetV2 as its backbone network; The local feature extraction network is constructed using ConvNeXtV2 as its backbone network.

[0010] Preferably, the step of inputting the global mid-level features, global deep features, local mid-level features, and local deep features into a pre-trained multi-stage sparse fusion network to obtain heterogeneous fusion features specifically includes: Unify the tensors of global mid-level features, global deep features, local mid-level features, and local deep features, and align the feature dimensions. By using sparse cross attention, multi-stage heterogeneous features are fused from dimension-aligned global mid-level features, global deep features, local mid-level features, and local deep features to obtain heterogeneous fused features.

[0011] Preferably, the step of inputting the global shallow features, global mid-level features, and global deep features into a pre-trained spatial-frequency multi-level fusion network to obtain spatial-frequency enhancement features, gated attention enhancement features, and multi-scale enhancement features specifically includes: The global shallow features are subjected to spatial and frequency domain dual-path fusion processing to obtain spatial-frequency enhanced features; Gated attention processing is applied to the global mid-level features to obtain gated attention enhanced features; The global deep features are subjected to multi-scale attention enhancement processing to obtain multi-scale enhanced features.

[0012] Preferably, mapping the final fused features to the evaluated quality score specifically involves mapping the final fused features to the evaluated quality score using a quality regressor.

[0013] Preferably, the quality regressor is a two-layer multilayer perceptron.

[0014] Secondly, the present invention also proposes a computer-readable storage medium storing computer instructions thereon, which, when executed, can realize the cloud game evaluation method based on spatial frequency enhancement and sparse attention fusion as described in the first aspect.

[0015] The beneficial effects of this invention are: The method of this invention constructs a multi-stage sparse fusion network and a space-frequency multi-level fusion network, and adopts dynamic feature dimension adaptation combined with sparse cross attention to achieve efficient and lightweight fusion of global and local heterogeneous features. At the same time, it implements spatial and frequency domain dual-path enhancement, lightweight gated attention modulation and multi-scale attention enhancement processing for global shallow, medium and deep features respectively. While reducing the number of model parameters and computational overhead and meeting the requirements of real-time inference and lightweight terminal deployment in cloud gaming scenarios, it accurately captures global scene distortion and local high-sensitivity area distortion in cloud gaming videos, effectively improving the evaluation accuracy and real-world application performance of the lightweight no-reference cloud gaming video quality assessment model.

[0016] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures pointed out in the description and the drawings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart of the cloud gaming evaluation method based on spatial frequency enhancement and sparse attention fusion of the present invention is shown; Figure 2 A schematic diagram of the method flow of the present invention is shown; Figure 3 This is a block diagram of the global feature extraction network branches of the present invention; Figure 4 This is a block diagram of the local feature extraction network branches of the present invention; Figure 5 This is a schematic diagram of the multi-stage sparse fusion network of the present invention; Figure 6 This is a schematic diagram of the space-frequency multi-level fusion network of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Reference Figure 1 and Figure 2 As shown, the cloud game evaluation method based on the fusion of spatial frequency enhancement and sparse attention includes the following steps: S10. Obtain the cloud game video stream to be evaluated, and extract key frames and key region image blocks from the cloud game video stream; Because cloud gaming video streams are characterized by large data volumes and a high proportion of redundant information, directly processing all video frames would significantly increase computational overhead, making it difficult to meet the real-time quality assessment requirements in cloud gaming scenarios.

[0021] Keyframes are core frames that represent the content of a cloud gaming video stream. They can convey the overall scene, global image state, and overall distortion performance of the video through a small number of samples. They can also provide complete global information input for the global feature extraction network in subsequent steps, supporting the extraction of global features, global shallow features, global mid-level features, and global deep features.

[0022] Specifically, an equal-interval sampling strategy is used to extract N frames as keyframes from the cloud gaming video stream. At the same time, a proportional scaling strategy is used on the keyframes, scaling the long side of the video frame to 224 pixels and adaptively adjusting the short side by the same proportion. Areas smaller than 224×224 are filled with white pixels to obtain keyframes of uniform size of 224×224.

[0023] Key region image patches are local areas in cloud gaming videos that attract high user attention and are highly sensitive to distortion (such as interactive interfaces, dynamic characters, high-contrast scenes, etc.). They can focus on core local locations that are prone to quality problems such as compression distortion, stuttering, and screen tearing. They can also provide targeted local detail inputs for the local feature extraction network in the subsequent S30 step, supporting the extraction of local features, local mid-level features, and local deep features.

[0024] Specifically, the Sobel operator is first used to process the extracted keyframes, calculate and fuse the gradients in the horizontal and vertical directions to generate the corresponding gradient magnitude feature map. On this basis, a non-overlapping 224×224 sliding window is used to perform integral image calculation on the gradient magnitude feature map, and the three image blocks with the largest gradient intensity in each frame are selected as key region image blocks. The key region image blocks can focus on the core local positions that are prone to quality problems such as compression distortion, stuttering, and screen tearing.

[0025] By combining keyframes with key region image patches, the redundancy of video data is reduced while preserving all global and local key information that affects the evaluation of cloud gaming video quality, laying a data foundation for subsequent feature extraction and fusion that balances efficiency and accuracy.

[0026] S20. Input the keyframes into the pre-trained global feature extraction network to obtain global features, global shallow features, global mid-level features, and global deep features; Because cloud gaming video streams require high real-time inference speeds, directly deploying heavy feature extraction networks on the terminal would cause inference latency, making it difficult to meet the interactive experience requirements. Therefore, the global feature extraction network of this invention is built with the lightweight ShuffleNetV2 as the backbone network, utilizing its grouped convolution and channel shuffling mechanism to extract hierarchical representations of keyframes while controlling the number of parameters.

[0027] Specifically, refer to Figure 3 As shown, the extracted 224×224 keyframes are input into the global feature extraction network. After the initial feature mapping is completed by the initial convolutional layer, the network sequentially passes through ShuffleNet module 1, first downsampling, ShuffleNet module 2, second downsampling, ShuffleNet module 3, third downsampling, and ShuffleNet module 4 to complete global hierarchical feature extraction and output a global semantic feature map.

[0028] Global shallow features: taken from the output of ShuffleNetV2 module 1. At this point, the feature map has a high resolution and contains rich spatial details such as edges and textures.

[0029] Global mid-level features: taken from the output of ShuffleNetV2 module 2. At this point, the feature map has undergone one downsampling, resulting in reduced resolution and increased number of channels, thus balancing local structure and mid-level semantics.

[0030] Global deep features: taken from the output of ShuffleNetV2 module 3. The feature map has undergone two downsampling operations, further reducing the resolution and resulting in stronger semantic discriminative ability.

[0031] Global features: taken from the output of ShuffleNetV2 module 4.

[0032] S30. Input the key region image patch into the pre-trained local feature extraction network to obtain local features, local mid-level features and local deep features; Quality distortion in cloud gaming videos is often concentrated in localized areas of high user attention (such as the edges of dynamic characters, UI text areas, and high-contrast scenes), making it difficult for global features to fully capture these localized sensitive distortions. Therefore, fine-grained feature extraction is required for key image regions. The local feature extraction network of this invention uses ConvNeXtV2 as its backbone network, leveraging its large kernel depthwise separable convolutions and inverse bottleneck structure to enhance the ability to discriminate local textures, edge artifacts, and compression distortion while maintaining high computational efficiency.

[0033] Specifically, refer to Figure 4 As shown, the extracted 224×224 key region image patch is input into the local feature extraction network. After the initial feature mapping is completed by the initial convolutional layer, it passes through ConvNeXtV2 module 1, first downsampling, ConvNeXtV2 module 2, second downsampling, ConvNeXtV2 module 3, third downsampling, and ConvNeXtV2 module 4 in sequence to complete the local hierarchical feature extraction and output the local semantic feature map.

[0034] Local mid-level features: taken from the output of ConvNeXtV2 module 2. At this point, the feature map is sensitive to changes in local structure and can effectively respond to distortions such as compression artifacts and jagged edges.

[0035] Local deep features: taken from the output of ConvNeXtV2 module 3. At this point, the feature map is rich in high-level semantics, which helps to determine whether a local region belongs to a distortable target (such as a character or UI element).

[0036] Local features: taken from the output of ConvNeXtV2 module 4.

[0037] S40. Input the global mid-level features, global deep features, local mid-level features, and local deep features into a pre-trained multi-stage sparse fusion network to obtain heterogeneous fusion features. Because global feature extraction networks and local feature extraction networks have different structures, the mid-level and deep features extracted by the two differ in channel dimension and spatial resolution. Directly concatenating or adding them cannot effectively fuse heterogeneous information. At the same time, cloud gaming video quality evaluation requires lightweight models, and conventional attention mechanisms (such as Transformer) have too high computational complexity.

[0038] The multi-stage sparse fusion network of this invention maps input features of different dimensions to a shared latent space through a dynamic dimension adaptation mechanism, and then uses a sparse cross-attention mechanism to achieve efficient fusion of multi-stage heterogeneous features while reducing computational complexity.

[0039] Specifically, refer to Figure 5 As shown, firstly, dynamic adaptation of feature dimensions is performed, unifying the four sets of feature tensors: global mid-level features, global deep features, local mid-level features, and local deep features, denoted as... In the formula, For batch quantity, For the number of channels, , For the space's height and width. For each input... To perform feature dimension alignment, the expression is:

[0040] In the formula, To achieve adaptive average pooling, the spatial dimension of the four-dimensional feature map is compressed into a vector; To return the dimension of the tensor; To flatten the tensor into two dimensions.

[0041] If the adapted feature dimensions ( If the latent space dimension is a preset value, then dimension alignment is performed using a learnable linear projection, expressed as:

[0042] In the formula, For projection weights, For bias, This refers to the batch quantity.

[0043] After dimensional adaptation, the mid-level features adapted from global branch are sparsely cross-attentiond with the mid-level and deep features adapted from local branch, respectively. The same operation is performed on the deep features adapted from global branch. In the end, there are four pairs of sparse cross-attention combinations.

[0044] For the input features of each group , First, an average process is performed to obtain the average feature. Its expression is:

[0045] Then, perform 8-head cross-attention calculation. For the j-th head ( Its expression is:

[0046] In the formula, That is, the dimension of each attention head; , , Let be the learnable projection matrix of the th order; This is the attention score matrix; Based on sparsity The binary mask matrix retains only the top two elements with the highest attention scores. The connection; This indicates element-wise multiplication; This is the output of the j-th header; It is a normalized exponential function.

[0047] The outputs of the eight heads are concatenated and linearly projected, and the expression is:

[0048] In the formula, To output the projected weights, For bias, For output dimensions.

[0049] The final output of the multi-stage sparse fusion network is:

[0050] In the formula, For learnable scaling matrix, These are heterogeneous fusion features obtained through sparse cross attention.

[0051] S50. Input the global shallow features, global mid-level features and global deep features into the pre-trained space-frequency multi-level fusion network to obtain space-frequency enhancement features, gated attention enhancement features and multi-scale enhancement features; Because cloud gaming video quality is affected by various types of distortion, the sensitivity of features at different levels to distortion varies: shallow features are rich in texture and detail and are sensitive to distortions such as compression artifacts and jagged edges; mid-level features take into account both structure and semantics and respond well to local block artifacts and blur; deep features contain high-level semantics and help to identify overall scene distortion. To customize and enhance the representational capabilities of features at each level, this invention uses a space-frequency multi-level fusion network, designing different enhancement modules for shallow, mid-level, and deep features respectively, to achieve customized enhancement of the network's representational capabilities at different levels.

[0052] Specifically, refer to Figure 6 As shown, the global shallow features, global mid-level features, and global deep features extracted in step S20 are respectively enhanced: For global shallow features, two parallel paths are constructed for enhancement. The first path is the spatial path, which extracts texture and detail features of the image through two convolutional layers, and the output is denoted as... The second path is the frequency domain path. First, a Fast Fourier Transform (FFT) is performed on the input features. In the frequency domain, a learnable high-pass mask is applied to suppress low-frequency background and highlight high-frequency components. Then, nonlinear enhancement is performed using depthwise separable convolution and inverse Fast Fourier Transform (IFFT). The output is denoted as... The outputs of the two paths are processed by learnable scalar weights. and Adaptive fusion is performed, and its expression is:

[0053] In the formula, Use the Sigmoid function to ensure weight normalization; This is a spatial frequency enhancement feature.

[0054] For global mid-level features, a lightweight gated attention module is used. First, the global mid-level features are processed by an efficient channel attention module (ECA). Then, the processed output is combined with the output of shallow feature enhancement. The concatenation is performed to obtain the context vector. Next, a small channel attention gating module is used to generate channel-level gated signals, which are then used to modulate the original features as residuals, expressed as:

[0055] In the formula, This represents a global mid-level feature; For high-efficiency channel attention modules; To splice along the channel dimension; It is a learnable gated weight matrix; For element-wise multiplication; This is a gating attention enhancement feature.

[0056] For global deep features, a multi-scale enhancement strategy is adopted. First, a CBAM (Convolutional Block Attention Module) is introduced, sequentially applying channel attention to identify important feature channels and spatial attention. The output is denoted as... Its expression is:

[0057] In the formula, This provides global deep features. Then, Atrous Spatial Pyramid Pooling (ASPP) is employed, utilizing dilated convolutional kernels of varying sizes to capture multi-scale receptive fields, effectively modeling large-scale block artifacts or color distortion; further, spatial attention is used... Enhance the expressive power of features to obtain output. Its expression is:

[0058] In the formula, Features enhanced by the spatial attention module; For spatial attention operations; For pyramid pooling operations in vacant spaces; This is an operation for the convolutional attention module.

[0059] Finally, and Concatenated along the channel dimension, this becomes the actual output of the deep branch, expressed as:

[0060] In the formula, This is a multi-scale enhancement feature; This is a feature splicing operation performed along the channel dimension.

[0061] S60. The global features, local features, spatial frequency enhancement features, gated attention enhancement features, multi-scale enhancement features, and heterogeneous fusion features are concatenated to obtain the final fusion feature. Specifically, the six features output by the global feature extraction network, the local features output by the local feature extraction network, the spatial-frequency enhancement features output by the spatial-frequency multi-level fusion network, the gated attention enhancement features, the multi-scale enhancement features, and the heterogeneous fusion features output by the multi-stage sparse fusion network are concatenated to form a final fusion feature with complementary information. The expression for concatenating these six features along the channel dimension is as follows:

[0062] In the formula, This is the final fusion feature.

[0063] S70. Map the final fused features to the evaluation quality score.

[0064] To map the fused high-dimensional features into a single quality score, a regressor needs to be designed. Fully connected layers (MLPs) are simple, lightweight, and have strong non-linear fitting capabilities, making them suitable for real-time quality prediction in cloud gaming scenarios.

[0065] The final fusion feature obtained in step S60 The input is fed into a pre-trained quality regressor. The quality regressor is a two-layer multilayer perceptron (MLP), and its expression is:

[0066] In the formula, It consists of two fully connected layers. The first layer maps the input feature dimension to an intermediate hidden dimension (usually set to half of the feature dimension or determined experimentally) and uses a non-linear activation function (such as ReLU). The second layer maps the intermediate hidden dimension to 1 and outputs a scalar value. This scalar value is the quality score output in this step. The higher the score, the better the video quality (or, depending on the label definition during training, usually the higher the better).

[0067] It should be further explained that this invention mainly proposes two feature fusion networks: Multi-Source Sparse Fusion Network (MSSF) and Spatial-Spectral Multi-stage Fusion Network (SSMF-Net). The model training uses the Adam optimizer, with a batch size of 8, an initial learning rate of 0.0001, and an exponential decay strategy. After every two training epochs, the learning rate is multiplied by a decay rate of 0.9, and training lasts for a maximum of 60 epochs.

[0068] The proposed network model was trained and validated on a public dataset. For each dataset, 20 random 8:2 splits were performed independently, and the optimal values ​​of SRCC, PLCC, KRCC, and RMSE were monitored on the validation set of each split. The final result was the arithmetic mean of the results of these 20 independent experiments.

[0069] The method of this invention constructs a multi-stage sparse fusion network and a space-frequency multi-level fusion network, and adopts dynamic feature dimension adaptation combined with sparse cross attention to achieve efficient and lightweight fusion of global and local heterogeneous features. At the same time, it implements spatial and frequency domain dual-path enhancement, lightweight gated attention modulation and multi-scale attention enhancement processing for global shallow, medium and deep features respectively. While reducing the number of model parameters and computational overhead and meeting the requirements of real-time inference and lightweight terminal deployment in cloud gaming scenarios, it accurately captures global scene distortion and local high-sensitivity area distortion in cloud gaming videos, effectively improving the evaluation accuracy and real-world application performance of the lightweight no-reference cloud gaming video quality assessment model.

[0070] Based on the same inventive concept as the above method, this invention also proposes a cloud gaming evaluation system based on spatial frequency enhancement and sparse attention fusion, comprising: The video acquisition module is used to acquire the cloud game video stream to be evaluated and extract key frames and key region image blocks from the cloud game video stream. The global feature extraction module is used to input keyframes into a pre-trained global feature extraction network to obtain global features, global shallow features, global mid-level features, and global deep features. The local feature extraction module is used to input key region image patches into a pre-trained local feature extraction network to obtain local features, local mid-level features, and local deep features. The multi-stage sparse fusion module is used to input global mid-level features, global deep features, local mid-level features, and local deep features into a pre-trained multi-stage sparse fusion network to obtain heterogeneous fused features. The space-frequency multi-level fusion module is used to input global shallow features, global mid-level features and global deep features into the pre-trained space-frequency multi-level fusion network to obtain space-frequency enhanced features, gated attention enhanced features and multi-scale enhanced features; The feature stitching module is used to stitch together global features, local features, spatial frequency enhancement features, gated attention enhancement features, multi-scale enhancement features, and heterogeneous fusion features to obtain the final fused features; The evaluation module maps the final fused features to an evaluation quality score.

[0071] Based on the same inventive concept as the above method, the present invention also proposes a device including a memory and a processor, wherein the memory stores computer instructions that can be executed on the processor, and the processor executes the above-mentioned cloud gaming evaluation method based on spatial frequency enhancement and sparse attention fusion when executing the computer instructions.

[0072] Based on the same inventive concept as the above method, the present invention also proposes a computer-readable storage medium storing computer instructions, which, when executed, can realize the cloud game evaluation method based on spatial frequency enhancement and sparse attention fusion.

[0073] Any references to memory, storage, database, or other media used in the embodiments provided in this invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory.

[0074] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0075] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cloud game evaluation method based on the fusion of spatial frequency enhancement and sparse attention, characterized in that, include: Acquire the cloud gaming video stream to be evaluated, and extract keyframes and key region image blocks from the cloud gaming video stream; The keyframes are input into a pre-trained global feature extraction network to obtain global features, global shallow features, global mid-level features, and global deep features. The key region image patch is input into a pre-trained local feature extraction network to obtain local features, local mid-level features, and local deep features; The global mid-level features, global deep features, local mid-level features, and local deep features are input into a pre-trained multi-stage sparse fusion network to obtain heterogeneous fusion features. The global shallow features, global mid-level features, and global deep features are input into a pre-trained space-frequency multi-level fusion network to obtain space-frequency enhancement features, gated attention enhancement features, and multi-scale enhancement features. The global features, local features, spatial frequency enhancement features, gated attention enhancement features, multi-scale enhancement features, and heterogeneous fusion features are concatenated to obtain the final fusion feature. The final fused features are mapped to an evaluated quality score.

2. The cloud gaming evaluation method based on spatial frequency enhancement and sparse attention fusion according to claim 1, characterized in that, The extraction of keyframes from the cloud gaming video stream specifically involves using an equal-interval sampling strategy to extract several frames from the cloud gaming video stream as keyframes.

3. The cloud gaming evaluation method based on spatial frequency enhancement and sparse attention fusion according to claim 2, characterized in that, Extracting key region image blocks from the cloud gaming video stream specifically includes: The keyframes are processed using gradient operators to generate gradient magnitude feature maps; The gradient magnitude feature map is integrally calculated based on a sliding window, and several image blocks with the largest gradient intensity are selected as key region image blocks.

4. The cloud gaming evaluation method based on spatial frequency enhancement and sparse attention fusion according to claim 2, characterized in that, After extracting N frames as keyframes from the cloud gaming video stream using an equal-interval sampling strategy, the method further includes: The keyframes are scaled and padded proportionally.

5. The cloud gaming evaluation method based on spatial frequency enhancement and sparse attention fusion according to claim 1, characterized in that, The global feature extraction network is constructed with ShuffleNetV2 as the backbone network. The local feature extraction network is constructed using ConvNeXtV2 as its backbone network.

6. The cloud gaming evaluation method based on spatial frequency enhancement and sparse attention fusion according to claim 1, characterized in that, The step of inputting the global mid-level features, global deep features, local mid-level features, and local deep features into a pre-trained multi-stage sparse fusion network to obtain heterogeneous fusion features specifically includes: Unify the tensors of global mid-level features, global deep features, local mid-level features, and local deep features, and align the feature dimensions. By using sparse cross attention, multi-stage heterogeneous features are fused from dimension-aligned global mid-level features, global deep features, local mid-level features, and local deep features to obtain heterogeneous fused features.

7. The cloud gaming evaluation method based on spatial frequency enhancement and sparse attention fusion according to claim 1, characterized in that, The process of inputting the global shallow features, global mid-level features, and global deep features into a pre-trained spatial-frequency multi-level fusion network to obtain spatial-frequency enhancement features, gated attention enhancement features, and multi-scale enhancement features specifically includes: The global shallow features are subjected to spatial and frequency domain dual-path fusion processing to obtain spatial-frequency enhanced features; Gated attention processing is applied to the global mid-level features to obtain gated attention enhanced features; The global deep features are subjected to multi-scale attention enhancement processing to obtain multi-scale enhanced features.

8. The cloud gaming evaluation method based on spatial frequency enhancement and sparse attention fusion according to claim 1, characterized in that, The step of mapping the final fused features to the evaluated quality score specifically involves mapping the final fused features to the evaluated quality score using a quality regressor.

9. The cloud gaming evaluation method based on spatial frequency enhancement and sparse attention fusion according to claim 8, characterized in that, The quality regressor is a two-layer multilayer perceptron.

10. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instructions are executed, the cloud gaming evaluation method based on the fusion of spatial frequency enhancement and sparse attention as described in any one of claims 1 to 9 can be implemented.

Images