Game lag frame detection method and system

Abstract

The invention discloses a jamming frame detection method and system for games, and relates to the technical field of computers, and the method comprises the following steps: fusing multi-modal features by using PyTorch, generating time sequence representation through TCN modeling based on the multi-modal features, mapping into jamming probability through a multilayer perceptron structure based on the time sequence representation, and optimizing the jamming probability by using a Focal Loss loss function. And a lagging detection result is generated. The frame difference threshold is dynamically optimized by adopting DQN reinforcement learning, so that the complexity of a game scene can be self-adapted, and the false alarm rate and the omission ratio are remarkably reduced; multi-modal features are fused through a multi-scale SE attention mechanism and PyTorch, and the feature expression ability and the model robustness are effectively improved; tCN time sequence modeling is used to accurately capture a lagging related time sequence mode, the detection precision under unbalanced data is optimized, and the accuracy and practicability of lagging detection in a high-dynamic game scene are enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer technology, and in particular to a frame freezing detection method and system for games. BACKGROUND

[0002] In recent years, with the rapid development of the electronic game industry, the complexity and real-time requirements of game pictures have significantly increased, and higher requirements have been put forward for frame freezing detection and optimization technology. Traditional frame freezing detection methods mainly rely on frame rate statistics or simple frame difference analysis, such as threshold segmentation based on pixel difference or abnormal detection based on frame time. These methods are effective in early games, but single modal methods are still insufficient in handling high dynamic game scenes, especially in multi-modal data fusion. The existing technology has not fully tapped the synergistic effect of cross-modal features. Although the existing technology has made significant progress in frame freezing detection, there are still some deficiencies. Traditional frame difference methods are sensitive to dynamic scenes, which can easily lead to false positives or missed detections. Moreover, the threshold is usually fixed, making it difficult to adapt to the complexity of different game scenes. Although deep learning-based models have improved feature extraction capabilities, the fusion of multi-modal data often lacks effective normalization mechanisms, which can affect model performance due to the scale difference between different modalities. After detecting frame freezing, existing methods lack fine-grained evaluation of frame freezing severity and dynamic optimization strategies, which cannot adjust rendering parameters in real time to alleviate frame freezing and affect player experience. User feedback is not fully integrated into the model optimization process, making it difficult to continuously improve detection accuracy in the long run. These deficiencies limit the application effect of frame freezing detection technology in high dynamic and complex interactive scenes. The existing frame freezing detection technology has the problems of insufficient adaptability to complex scenes, insufficient fusion of multi-modal features, lack of severity evaluation, and lack of closed-loop optimization mechanism. The present application proposes a frame freezing detection method based on multi-modal data fusion, which combines DQN reinforcement learning to optimize frame difference thresholds, multi-scale SE attention mechanism to enhance feature extraction, TCN time series modeling, Focal Loss to optimize frame freezing probability, and user feedback-driven closed-loop optimization. The present application solves the problems of insufficient accuracy and untimely optimization of existing technology in dynamic scenes, significantly improving the robustness and real-time performance of frame freezing detection. SUMMARY

[0003] In view of the above existing problems, the present application is proposed.

[0004] Therefore, the present application provides a frame freezing detection method and system for games, which solves the problems of insufficient adaptability to complex scenes, insufficient fusion of multi-modal features, lack of severity evaluation, and lack of closed-loop optimization mechanism in existing frame freezing detection technology.

[0005] To solve the above technical problems, the present application provides the following technical solutions: In a first aspect, the present application provides a frame freezing detection method for games, which comprises collecting multi-modal data and calculating frame difference images, dynamically optimizing frame difference thresholds using DQN reinforcement learning, generating a binary mask sequence, obtaining feature maps based on the binary mask sequence through MobileNetV3-Small, optimizing the feature maps using a multi-scale SE attention mechanism, and performing cross-modal feature extraction. Computing global change intensity, fusing multi-modal features using PyTorch, generating temporal representations based on multi-modal features through TCN modeling, mapping the temporal representations to frame freezing probabilities through a multi-layer perception structure, optimizing the frame freezing probabilities using a Focal Loss loss function, and generating frame freezing detection results. Based on the frame freezing detection results, calculating the severity score according to the frame freezing probability and the frame difference intensity, and grading according to the score; According to the severity score and the grading result, dynamically adjusting the rendering parameters, fine-tuning the model parameters through user feedback and the optimized rendering parameters to improve the detection accuracy.

[0006] As a preferred scheme of the frame freezing detection method for games, the method comprises the following steps: collecting multi-modal data and calculating frame difference images, dynamically optimizing frame difference thresholds using DQN reinforcement learning, and generating a binary mask sequence. Collecting cross-modal data, including audio signals and input events, and calculating the pixel difference between the current frame and the previous frame ; Using reinforcement learning (DQN) to dynamically optimize the threshold , respectively to , respectively , and comparing the frame difference image with the corresponding threshold to generate a binary mask and form a binary mask sequence .

[0007] As a preferred scheme of the frame freezing detection method for games, the method comprises the following steps: obtaining feature maps through MobileNetV3-Small, and optimizing the feature maps using a multi-scale SE attention mechanism. , inputting the frame into MobileNetV3-Small to obtain feature maps , applying multi-scale attention to the feature maps through a multi-scale SE attention mechanism to obtain weighted feature maps , and extracting audio features using a fast Fourier transform (FFT) method , use the encoder model (EventEncoder) to extract input event features .

[0008] As a preferred solution of the freeze frame detection method for games described in the present invention, wherein: the calculation of global change intensity, using PyTorch to fuse multimodal features, generating a time series representation based on the multimodal features through TCN modeling, mapping the time series representation to a freeze probability through a multi-layer perceptron structure, using the Focal Loss loss function to optimize the freeze probability, and generating a freeze detection result refers to calculating the global change frame difference intensity , use PyTorch's torch.cat function to splice frame difference features and semantic features to generate a fused feature vector , TCN processes the fusion feature sequence through multi-layer one-dimensional causal convolution , generating a temporal representation , the timing of TCN output is represented by Enter the first fully connected layer in the multi-layer perceptron (MLP) structure, using weights and bias , expand the feature dimension to twice the original one and obtain the intermediate feature representation ,right Apply Dropout for regularization, and send the activated result to the second fully connected layer to obtain the feature representation after restoring the original dimension. , with the initial input Perform residual connection and use LayerNorm for normalization to obtain enhanced representation ,Will Send it to the linear layer, using weights and bias Transform and output the jamming probability through Softmax ; Based on the Focal Loss loss function, the error between the model prediction and the true label is calculated , updated using the SGD optimizer and b, using the optimized and b to recalculate the new jam probability , set the optimal threshold Q, if > threshold Q, then output the jam detection result ,like < threshold Q, then output the jam detection result 0.

[0009] As a preferred scheme of the game frame freezing detection method, wherein: the severity score is calculated according to the frame freezing probability and the frame difference intensity, and the severity score is calculated according to the frame freezing probability and the frame difference intensity According to the score grading, a low severity threshold U and a high severity threshold B are set, if U, it is low severity, if U≤ B, it is medium severity, if B, it is high severity.

[0010] As a preferred scheme of the game frame freezing detection method, wherein: the severity score is calculated according to the frame freezing probability and the frame difference intensity, and the severity score is calculated according to the frame freezing probability and the frame difference intensity And the grading result, trigger targeted rendering optimization strategy, for high severity, use Unity's Screen.SetResolution to reduce resolution from 1080p to 720p, for medium severity, reduce the number of particle effects through Unity's ParticleSystem API, for low severity, use Unity's ShaderLab to switch to low precision shader, the optimization strength is calculated by the score, the adjustment amplitude is equal to the severity score multiplied by the maximum adjustment amplitude, the context parameters are obtained from the game engine in real time, to ensure that the strategy adapts to the current running environment, and the final result is the optimized rendering parameter.

[0011] As a preferred scheme of the game frame freezing detection method, wherein: the severity score is calculated according to the frame freezing probability and the frame difference intensity, and the severity score is calculated according to the frame freezing probability and the frame difference intensity According to the difference between user annotation and TCN model prediction, the CPU is used to calculate the adjustment step to update the frame difference threshold weight, the average error is calculated based on 50 groups of feedback data to adjust, the updated TCN parameters and threshold weight are applied to the frame difference method preprocessing, the newly collected frame sequence is reprocessed to generate a new mask sequence, which is sequentially processed through CNN feature extraction, TCN time series analysis, severity evaluation, dynamic optimization, and user feedback is collected again, the model and threshold are updated in a loop to form a closed loop optimization.

[0012] The feature extraction and fusion module is configured to extract a semantic feature map and combine a multi-scale SE attention mechanism to enhance a frame freezing related feature; The time sequence modeling and frame freezing probability mapping module is configured to input a fusion feature sequence into a TCN to generate a time sequence representation, output a frame freezing probability through a linear layer, and generate a frame freezing detection result; The severity evaluation and grading module is configured to calculate a severity score based on the frame freezing probability and the global frame difference intensity, and in combination with a historical maximum frame difference intensity. The dynamic rendering parameter adjustment module is configured to dynamically adjust game rendering parameters to alleviate frame freezing according to the severity score and the grading result.

[0013] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, any step of the frame freezing detection method for games according to the first aspect of the present application is implemented.

[0014] In a fourth aspect, the present application provides a computer readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, any step of the frame freezing detection method for games according to the first aspect of the present application is implemented.

[0015] The present application has the following beneficial effects: by using DQN reinforcement learning to dynamically optimize the frame difference threshold, the complexity of the game scene can be adapted, and the false alarm and missed detection rates can be significantly reduced; by using the multi-scale SE attention mechanism and PyTorch to fuse multi-modal features, the feature expression ability and model robustness can be effectively improved; by combining FocalLoss and TCN time sequence modeling, the frame freezing related time sequence pattern can be accurately captured and the detection accuracy under unbalanced data can be optimized; based on the severity score and grading mechanism of the frame freezing probability and the frame difference intensity, the basis for dynamically adjusting the rendering parameters is provided, and the frame freezing is alleviated in real time; through the closed-loop optimization driven by user feedback, the model parameters and threshold weights are continuously fine-tuned, the long-term detection accuracy is further improved, the accuracy and practicality of the frame freezing detection in high dynamic game scenes are enhanced, a smoother game experience is provided for the players, and the optimization cost of the game engine is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creating any creative labor.

[0017] Fig. 1 The flowchart of the frame freezing detection method for games in embodiment 1.

[0018] Fig. 2 Structure diagram of the frame freezing detection system for games in Embodiment 1.

[0019] Fig. 3 Dynamic optimization and closed-loop flowchart of the frame freezing detection method for games in Embodiment 1. DETAILED DESCRIPTION

[0020] In order to make the above objectives, characteristics and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0021] In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the present application, therefore the present application is not limited by the specific embodiments disclosed below.

[0022] Secondly, the "one embodiment" or "embodiment" referred to herein means that the specific features, structures or characteristics can be included in at least one implementation of the present application. "In one embodiment" appearing in different places in the specification does not mean the same embodiment, nor is it an embodiment that is separate or alternative to other embodiments.

[0023] Embodiment 1, Reference Figs. 1-3 , for the first embodiment of the present application, the embodiment provides a frame freezing detection method for games, comprising the following steps: S1, collect multi-modal data and calculate frame difference image, use DQN reinforcement learning to dynamically optimize frame difference threshold, and generate binary mask sequence, get feature map based on binary mask sequence through MobileNetV3-Small, use multi-scale SE attention mechanism to optimize feature map, and perform cross-modal feature extraction; Specifically, collect multi-modal data and calculate frame difference image, use DQN reinforcement learning to dynamically optimize frame difference threshold, and generate binary mask sequence, get frame sequence through engine API (such as Unity's RenderTexture) , n is the time window (such as 5 frames), t represents the current time step, use bilinear interpolation (OpenCV's resize function) to reduce the resolution of the original frame to 256x256 pixels, and normalize the pixel value to get , collect cross-modal data, including audio signal (sampling rate 44.1kHz, extract short-time energy and spectral entropy) and input event (record keyboard and mouse operation frequency and timestamp), calculate the pixel difference value between the current frame and the previous frame: , where, is the pixel value of the frame difference image at (x, y), x and y represent the pixel coordinates of the frame, corresponding to the horizontal (column) and vertical (row) positions of the image respectively, and represents the intensity of the screen change; The threshold value is dynamically optimized using reinforcement learning (DQN) to adapt to the complexity of the game scene (such as light and shadow changes, object density), improve the detection accuracy of the frame difference method for the lag area, and reduce false positives and missed detections: , where, and are the mean and standard deviation of the frame difference image, respectively, calculated by the meanStdDev function of OpenCV, is the GPU usage, representing the load of the current device GPU, used to evaluate the impact of hardware performance on lag detection, is the action space, represents the change in the frame difference threshold T adjusted at time step t, is the reward function, which measures the detection performance, defined as , used to optimize the threshold value in reinforcement learning, , , is the weight coefficient, is the accuracy, the proportion of correct lag and non-lag frames predicted by the model to the total number of frames, Recall is the recall rate, the proportion of correct detection of lag frames by the model to the actual lag frames, Latency is the detection delay, the time required by the model from input frame to output lag detection result, is the state input at time step t; For each frame to in the time window (window size n), calculate the frame difference image and compare it with the corresponding threshold to generate a binary mask and form a binary mask sequence , formula: , where, represents the potential lag area at position (x, y), represents the non-lag area.

[0024] ​By calculating the frame difference image across modal data, the picture change intensity is effectively captured, the DQN reinforcement learning is used to dynamically optimize the frame difference threshold, the mean, standard deviation and GPU usage rate are used to construct the reward function, the adaptive game scene complexity is used to significantly reduce the false alarm and missed detection rate; By comparing the frame difference image with the dynamic threshold to generate a binary mask sequence, the potential stuttering area is accurately marked, the robustness and accuracy of stuttering detection are enhanced, the adaptability of traditional methods to dynamic scenes is overcome, a solid foundation is laid for subsequent feature extraction and optimization, and the smoothness of game experience is significantly improved.

[0025] Further, based on the binary mask sequence, a feature map is obtained through MobileNetV3-Small, the feature map is optimized using a multi-scale SE attention mechanism, and a cross-modal feature extraction is performed on the frame processed by the mask to generate an input image focusing on the potential stuttering area, formula: , The frame is input into MobileNetV3-Small (convolutional neural network architecture): , wherein, is a feature map, representing the semantic features of the mask-processed frame, is a pre-training parameter (ImageNet pre-training, fine-tuned on a game frame dataset); The multi-scale SE attention mechanism is applied to the feature map to exert multi-scale attention and enhance features related to stuttering (such as repeated frame textures and static picture areas), formula: )), wherein, and are spatial and channel global average pooling, respectively, s is the spatial dimension, c is the channel dimension, , , , is a fully connected layer weight, obtained through a supervised learning method, is a Sigmoid activation function, wherein x is an input value (output of the fully connected layer), is an output value, ranging from 0 to 1, used to generate attention weights, is a weighted feature map, is an adaptive weight, obtained through an Attention mechanism, (based on frame difference intensity and game scene complexity, dynamically adjusted, ranging from 0 to 1); The Fast Fourier Transform (FFT) method is used to extract audio features (An audio signal with a sampling rate of 44.1kHz is divided into frames by a fixed window (such as 25ms), each frame is windowed (such as Hamming window) and then subjected to fast Fourier transform (FFT) to obtain a spectrum, and the short-time energy (total energy within the frame) and spectral entropy (entropy value of the normalized spectrum) of each frame are calculated as the feature representation of the audio), which is used to analyze the frequency domain features (such as short-time energy, spectral entropy) of the audio signal, and the input event features are extracted using an encoder model (EventEncoder) (The keyboard strokes and mouse movement events of the user at time step t are encoded in time sequence into event sequences, and the frequency (number of times per unit time) and mouse speed (displacement divided by time interval) of each type of event are extracted and input into the EventEncoder model to generate a low-dimensional feature vector representing the current user input behavior pattern through time series modeling (such as RNN or Transformer)), which is used to model the behavior pattern of user operation (such as key frequency, mouse speed).

[0026] The frame processed by the binary mask is input into MobileNetV3-Small to generate a high semantic feature map, effectively capturing the visual features related to lag, combining a multi-scale SE attention mechanism, generating adaptive weights through spatial and channel global average pooling and a fully connected layer, enhancing the features related to lag, improving the detection accuracy, extracting audio features through fast Fourier transform (FFT) and input event features through EventEncoder encoding, realizing cross-modal feature extraction, and significantly improving the robustness and completeness of feature expression through PyTorch fusion of multi-modal features, overcoming the limitations of traditional methods in feature extraction in complex scenes, providing high-quality input for subsequent time series modeling and lag probability mapping, significantly improving the accuracy and applicability of lag detection.

[0027] S2, calculate the global change intensity, fuse the multi-modal features using PyTorch, generate a time series representation based on the multi-modal features through TCN modeling, map the time series representation to a lag probability through a multi-layer perceptron structure, optimize the lag probability using a Focal Loss loss function, and generate a lag detection result; Based on the lag detection result, calculate the severity score according to the lag probability and frame difference intensity, and grade according to the score; Specifically, calculate the global change intensity, fuse the multi-modal features using PyTorch, generate a time series representation based on the multi-modal features through TCN modeling, map the time series representation to a lag probability through a multi-layer perceptron structure, optimize the lag probability using a Focal Loss loss function, and generate a lag detection result , reflecting the overall degree of change in the entire frame of pictures, used to quantify the possibility of lag, formula: , where H and E are frame difference image sizes (e.g., 256x256); Concatenate frame difference features and semantic features using PyTorch's torch.cat function to generate a fused feature vector , improving the robustness and accuracy of stutter detection, formula: , where is to flatten (8x8x128) into an 8192-dimensional vector; TCN (Temporal Convolutional Network) processes the fused feature sequence through multiple layers of one-dimensional causal convolution , generating a temporal representation that captures stutter-related temporal patterns (such as repeated frames, frame skipping), improving detection accuracy, formula: ; where is the TCN parameter, using the Adam optimizer (existing technology, PyTorch's torch.optim.Adam), the loss function is cross-entropy loss, the dataset is 5000 sets of labeled game frame sequences, the learning rate is set to 0.001, and the training is 10 epochs; Input the temporal representation output by TCN into the first fully connected layer in the multi-layer perceptron (MLP) structure, using weights and bias , expand the feature dimension to the original two times, get the intermediate feature representation : , where and belong to the hidden layer MLP, used to extract higher-order features, based on the 5,000 sets of labeled game frame sequence dataset, using the cross-entropy loss function for supervised training, through the backpropagation mechanism in the PyTorch framework and the Adam optimizer (such as torch.nn.Linear) to automatically learn; Apply Dropout to the intermediate feature representation for regularization, send the activated result to the second fully connected layer to get the feature representation after restoring the original dimension , perform residual connection with the initial input , and use LayerNorm for normalization to get the enhanced representation , send to the linear layer using weights and bias Transform and output the probability of freezing through Softmax , formula: , where, and are output layer weights and bias, obtained through supervised learning, using a set of 5,000 labeled game frame sequences (including normal and freezing frames), measuring the prediction error with a cross-entropy loss function (PyTorch's CrossEntropyLoss), and updating through an SGD optimizer (learning rate 0.01, momentum 0.9, PyTorch's torch.optim.SGD) and b; Based on the Focal Loss loss function, used to optimize the model: , where, is the true label (0 = normal, 1 = freezing), is the class weight, obtained through the Class Weighting method, is the focus parameter, obtained through the Grid Search method, tested on the game frame dataset range [1,3], selected through cross-validation to minimize the Focal Loss and maximize the freezing detection accuracy =2, is the error between the model prediction and the true label, measuring the accuracy of freezing classification; Update and b using the SGD optimizer: , where, is the learning rate (0.01), tested on the game frame dataset (5000 groups) through Grid Search, selecting the value with the smallest validation set loss; Recalculate the new freezing probability using the optimized and b , analyze through the ROC curve, calculate the balance point of precision and recall on the game frame dataset, set the best threshold Q, if > threshold Q, output the freezing detection result , if < threshold Q, output the freezing detection result 0 (0 = normal, 1 = freezing).

[0028] The global frame difference is calculated to quantify the degree of picture change, and the torch.cat function of PyTorch is used to splice the frame difference features, semantic features and cross-modal features to generate a fusion feature vector, which significantly improves the robustness and integrity of the features; the TCN processes the fusion feature sequence, captures the time sequence mode related to the stall through a multi-layer one-dimensional causal convolution, and improves the detection accuracy; two-layer MLP structure is used to map the TCN time sequence representation to the stall probability, and the Focal Loss loss function is used for joint training to ensure high-precision stall detection and overcome the limitations of traditional methods in time sequence modeling and unbalanced data processing, providing an efficient and accurate solution for stall detection in complex game scenarios.

[0029] Further, based on the stall detection result, the severity score is calculated according to the stall probability and the frame difference intensity, and the severity score is calculated according to the grading of the score and the combination of the stall probability and the frame difference intensity: , wherein, is the severity score, ranging from [0, 1], is the maximum value of the historical frame difference intensity, obtained by a real-time statistical method, during the game running, the frame difference intensity sequence is calculated using a sliding window (size 100 frames) , the maximum value is calculated by traversing all frame difference intensity values in the window , the max function of NumPy is used to update in real time on the GPU, ensuring that the normalized stall severity score reflects the dynamic range of the current game scene, is the weight coefficient (obtained by Random Search method); According to the score grading, the low severity threshold U and the high severity threshold B are set by the ROC curve analysis method, if <U, it is low severity (such as small inter-frame change, slight stall impact), if U≤ <B, it is medium severity (such as perceptible inter-frame change, moderate stall impact), if ≥B, it is high severity (such as significant inter-frame change, strong stall impact).

[0030] S3, according to the severity score and the grading result, dynamically adjust the rendering parameters, through user feedback and optimized rendering parameters, fine-tune the model parameters to improve the detection accuracy; Specifically, according to the severity score and the grading result, dynamically adjust the rendering parameters based on the severity score And the hierarchical results, trigger targeted rendering optimization strategy, for high severity, using Unity's Screen.SetResolution to reduce the resolution from 1080p to 720p, for medium severity, reduce the number of particle effects through Unity's ParticleSystem API, for low severity, use Unity's ShaderLab to switch to a low-precision shader, the optimization strength is calculated by the score, the adjustment amplitude is equal to the severity score multiplied by the maximum adjustment amplitude (preset to 50%, defined according to device performance such as GPU occupancy), context parameters (such as GPU occupancy) are obtained in real time from the game engine, ensuring that the strategy adapts to the current running environment, and the final result is the optimized rendering parameters (such as resolution, particle number), which relieves the stutter and provides a stable running environment for subsequent user feedback integration.

[0031] By combining the probability of stutter and the global frame difference intensity to calculate the severity score, it is ensured that the score reflects the dynamic range of the current game scene, and the weight coefficient is optimized by Random Search to balance the probability and frame difference contribution, overcoming the limitations of traditional methods in evaluating the degree of stutter; By setting the low severity threshold and high severity threshold through ROC curve analysis, the score is divided into three levels, providing accurate basis for subsequent rendering parameter adjustment, significantly improving the fine degree and practicality of stutter detection, so that the optimization strategy can dynamically adapt to different degrees of stutter, thereby effectively improving the player's experience in complex game scenes.

[0032] Further, through user feedback and optimized rendering parameters, the model parameters are fine-tuned to improve detection accuracy. By collecting player feedback through game UI (such as "stutter experience" button) and frame time anomalies (frame interval greater than 16.67ms), labeled data is generated in the format of frames and user annotations (0 = normal, 1 = stutter), using 10 groups of sequence data collected per minute (about 50 frames), the online cross-entropy loss is calculated through PyTorch's SGD optimizer (learning rate 0.001) to update the TCN model parameters to improve prediction accuracy, according to the difference between user annotations and TCN model predictions, the CPU is used to calculate the adjustment step (0.01) to update the frame difference threshold weight, based on 50 feedback data, the average error is calculated to adjust (using CPU to traverse feedback data, calculating the difference between user annotations and TCN model predictions, dividing by the data size (50), multiplying by the adjustment step (0.01), adding the result to the current threshold weight to get the updated weight, achieved through NumPy's mean calculation), the updated TCN parameters and threshold weight are applied to the frame difference method preprocessing, the newly collected frame sequence is reprocessed to generate a new mask sequence, which is sequentially processed through CNN feature extraction, TCN time series analysis, severity evaluation, dynamic optimization, and user feedback is collected again, the model and threshold are updated in a loop to form a closed-loop optimization.

[0033] The game UI and frame time anomaly collect player feedback, generate labeled data, calculate online cross-entropy loss using the SGD optimizer of PyTorch, update TCN model parameters to improve prediction accuracy, and overcome the limitations of traditional methods that model parameters are static and cannot adapt to dynamic scenarios. Based on the difference between user labeling and TCN prediction, the model and threshold are continuously improved according to user feedback, significantly enhancing the long-term adaptability and accuracy of stutter detection, making up for the lack of user-driven optimization in the prior art, and providing a more stable and accurate stutter detection and optimization experience for complex game scenarios.

[0034] The embodiment also provides a stutter frame detection system for games, comprising: a multi-modal data acquisition and frame difference calculation module for calculating frame difference images, quantifying picture change intensity, dynamically optimizing frame difference thresholds using DQN reinforcement learning, and generating a binary mask sequence; A feature extraction and fusion module is used to extract semantic feature maps and enhance stutter-related features using a multi-scale SE attention mechanism. A time series modeling and stutter probability mapping module is used to input the fused feature sequence into TCN to generate time series representation, output stutter probability through a linear layer, and generate stutter detection results. A severity evaluation and grading module is used to calculate severity scores based on stutter probability and global frame difference intensity, combined with historical maximum frame difference intensity. A dynamic rendering parameter adjustment module is used to dynamically adjust game rendering parameters to alleviate stuttering based on severity scores and grading results.

[0035] The embodiment also provides a computer device suitable for the stutter frame detection method for games, comprising: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions to realize the stutter frame detection method for games as proposed in the above embodiments.

[0036] The computer device can be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected by a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved by WIFI, a carrier network, NFC (Near Field Communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, a trackball or a touchpad arranged on the shell of the computer device, or an external keyboard, a touchpad or a mouse, etc.

[0037] The embodiment also provides a storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the method and system for detecting a frozen frame for a game. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic storage, a flash memory, a magnetic disk or an optical disk.

[0038] To sum up, by adopting the DQN reinforcement learning to dynamically optimize the frame difference threshold, the application can adapt to the complexity of the game scene, significantly reduce the false alarm and missing detection rate; by fusing multi-modal features through the multi-scale SE attention mechanism and PyTorch, the feature expression ability and model robustness are effectively improved; by combining FocalLoss and TCN time series modeling, the related time series mode of lag is accurately captured and the detection precision under unbalanced data is optimized; based on the severity score and grading mechanism of the lag probability and the frame difference intensity, the basis for dynamically adjusting the rendering parameters is provided, and the lag is relieved in real time; through the closed-loop optimization driven by user feedback, the model parameters and threshold weights are continuously fine-tuned, the long-term detection precision is further improved, the accuracy and practicality of the lag detection in the high dynamic game scene are enhanced, a smoother game experience is provided for the players, and the optimization cost of the game engine is reduced.

[0039] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit it. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application, which should be covered in the scope of the claims of the present application.

Claims

1. A method for detecting stuck frames in games, characterized by: include, Collect multimodal data and calculate frame difference images. Use DQN reinforcement learning to dynamically optimize the frame difference threshold and generate a binary mask sequence. Based on the binary mask sequence, use MobileNetV3-Small to obtain feature maps. Use the multi-scale SE attention mechanism to optimize the feature maps and perform cross-modal feature extraction. Calculate the global change intensity, use PyTorch to fuse multimodal features, generate a time series representation based on the multimodal features through TCN modeling, map the time series representation to a jam probability through a multi-layer perceptron structure, use the Focal Loss function to optimize the jam probability, and generate jam detection results; Based on the jamming detection results, the severity score is calculated according to the jamming probability and frame difference strength, and the game is graded according to the score; Dynamically adjust rendering parameters based on severity scores and grading results. Fine-tune model parameters through user feedback and optimized rendering parameters to improve detection accuracy.

2. The method for detecting jammed frames in a game according to claim 1, wherein: The method of collecting multimodal data and calculating frame difference images, using DQN reinforcement learning to dynamically optimize the frame difference threshold, and generating a binary mask sequence refers to obtaining a frame sequence through the engine API, using bilinear interpolation to reduce the resolution of the original frame to 256x256 pixels, and normalizing the pixel values ​​to obtain , collect cross-modal data, including audio signals and input events, and calculate the pixel difference between the current frame and the previous frame ; Dynamically optimize threshold using reinforcement learning (DQN) , for each frame in the time window to , respectively calculate the frame difference image , and the corresponding threshold Compare, generate binary masks, and form a binary mask sequence .

3. The method for detecting jammed frames in a game according to claim 2, wherein: The feature map is obtained by MobileNetV3-Small, and the multi-scale SE attention mechanism is used to optimize the feature map to calculate the frame after mask processing. , the frame Input MobileNetV3-Small and get the feature map , through the multi-scale SE attention mechanism to the feature map Apply multi-scale attention to obtain weighted feature maps , using the Fast Fourier Transform (FFT) method to extract audio features , use the encoder model (EventEncoder) to extract input event features .

4. The method for detecting jammed frames in a game according to claim 3, wherein: The calculation of global change intensity uses PyTorch to fuse multimodal features, generates time series representation based on multimodal features through TCN modeling, maps the time series representation to the jam probability through the multi-layer perceptron structure, uses the Focal Loss loss function to optimize the jam probability, and generates the jam detection result. This refers to calculating the global change frame difference intensity. , use PyTorch's torch.cat function to splice frame difference features and semantic features to generate a fusion feature vector , TCN processes the fusion feature sequence through multi-layer one-dimensional causal convolution , generating a temporal representation , the timing of TCN output is represented by Enter the first fully connected layer in the multi-layer perceptron (MLP) structure, using weights and bias , expand the feature dimension to twice the original one and obtain the intermediate feature representation ,right Apply Dropout for regularization, and send the activated result to the second fully connected layer to obtain the feature representation after restoring the original dimension. , with the initial input Perform residual connection and use LayerNorm for normalization to obtain enhanced representation ,Will Send it to the linear layer, using weights and bias Transform and output the jamming probability through Softmax ; Based on the Focal Loss loss function, the error between the model prediction and the true label is calculated , updated using the SGD optimizer and b, using the optimized and b to recalculate the new jam probability , set the optimal threshold Q, if > threshold Q, then output the jam detection result ,like < threshold Q, then output the jam detection result 0.

5. The method for detecting jammed frames in a game according to claim 4, wherein: Calculate the severity score based on the stuttering probability and frame difference intensity, and grade according to the score, which means combining the stuttering probability and frame difference intensity to calculate the severity score , set the low severity threshold U and high severity threshold B according to the score grading. If < U, it is of low severity. If U ≤ < B, it is of medium severity. If ≥ B, it is of high severity.

6. The method for detecting jammed frames in a game according to claim 5, wherein: The dynamic adjustment of rendering parameters according to the severity score and grading results refers to the dynamic adjustment of rendering parameters according to the severity score. And the grading results trigger targeted rendering optimization strategies. For high severity, use Unity's Screen.SetResolution to reduce the resolution from 1080p to 720p. For medium severity, use Unity's ParticleSystem API to reduce the number of particle effects. For low severity, use Unity's ShaderLab to switch to low-precision shaders. The optimization strength is calculated based on the score, and the adjustment range is equal to the severity score multiplied by the maximum adjustment range. The context parameters are obtained in real time from the game engine to ensure that the strategy adapts to the current operating environment. The final result is the optimized rendering parameters.

7. The method for detecting jammed frames in a game according to claim 6, wherein: The method of fine-tuning model parameters to improve detection accuracy through user feedback and optimized rendering parameters refers to collecting player feedback through game UI and frame time anomalies, generating labeled data in the format of frames and user annotations, using 10 sets of sequence data collected per minute, calculating online cross entropy loss through PyTorch's SGD optimizer, updating TCN model parameters to improve prediction accuracy, using CPU calculation to adjust the step size to update the frame difference threshold weight based on the difference between user annotations and TCN model predictions, calculating the average error based on 50 sets of feedback data for adjustment, applying the updated TCN parameters and threshold weights to frame difference method preprocessing, reprocessing the newly collected frame sequence, generating a new mask sequence, and sequentially going through CNN feature extraction, TCN timing analysis, severity assessment, dynamic optimization, and collecting user feedback again, cyclically updating the model and threshold to form a closed-loop optimization.

8. A system for detecting stuck frames in games, based on the method for detecting stuck frames in games according to any one of claims 1 to 7, characterized in that: It includes a multimodal data acquisition and frame difference calculation module, which is used to calculate the frame difference image, quantify the intensity of the picture change, use DQN reinforcement learning to dynamically optimize the frame difference threshold, and generate a binary mask sequence; Feature extraction and fusion module, used to extract semantic feature maps and combine multi-scale SE attention mechanism to enhance the jamming-related features; The time series modeling and jamming probability mapping module is used to input the fused feature sequence into the TCN to generate a time series representation, output the jamming probability through the linear layer, and generate the jamming detection result; Severity assessment and grading module, used to calculate severity scores based on the probability of stuck and the global frame difference strength, combined with the historical maximum frame difference strength; The dynamic rendering parameter adjustment module is used to dynamically adjust game rendering parameters to alleviate lag based on severity scores and grading results.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the game freeze frame detection method described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the game freeze frame detection method described in any one of claims 1 to 7 are implemented.

Images