A microservice software optimization method based on reinforcement learning

Abstract

This invention relates to the field of microservice anomaly detection, and more particularly to a microservice software optimization method based on reinforcement learning. This invention acquires historical operation data from the user terminal to extract the user terminal's operation rhythm features; combines these operation rhythm features with the uniformity of the repeated attack interval to analyze the user terminal's operation rhythm representation value, thereby marking the user terminal's rhythm steady-state category; in response to the user terminal being marked as having a low rhythm steady-state category, anomaly assessment analysis of the user terminal's game operations is performed; based on the conditional judgment results, combined with the matching degree between the aiming timestamp and the target's updated position timestamp, and the angle between the screen viewpoint and the firing direction, it is determined whether the user terminal is using unauthorized microservice software. This invention focuses on the accurate identification of unauthorized microservice software tools in terminal scenarios, improving the accuracy of anomaly detection and reducing the probability of false positives and false negatives through multi-dimensional feature fusion, hierarchical detection logic, and cross-validation mechanisms.

Description

Technical Field

[0001] This invention relates to the field of microservice anomaly detection, and more particularly to a microservice software optimization method based on reinforcement learning. Background Technology

[0002] Unauthorized microservice software (commonly known as "plugins" or "cheating software") severely disrupts game balance and user experience by injecting or modifying memory data or automating user operations. Traditional anti-cheating technologies mainly rely on signature detection, signature comparison, and static rule matching. However, these methods suffer from significant drawbacks, such as high detection lag, high countermeasure costs, and high false positive rates, when faced with increasingly modular, service-oriented, and dynamically updated plug-in architectures.

[0003] Chinese Patent Application Publication No. CN120860585A discloses a method and system for protecting players from visual cheating in video games based on machine learning. This application provides a method and system to protect players from visual auto-aiming cheating in video games, comprising: acquiring each frame of the video game; superimposing a pre-generated universal defense perturbation onto each acquired frame; and presenting the frame with the superimposed universal defense perturbation to the player of the video game. The universal defense perturbation superimposed on each frame is visually invisible to the player of the video game and enables the object detection model in the visual auto-aiming system during video game operation to fail to identify target objects, thereby disabling its auto-aiming function. This solution prevents the visual auto-aiming system from accurately identifying game targets, thus ensuring the integrity of the game without affecting legitimate gameplay. It can be effectively scaled across different game settings and player groups and can adapt to the dynamic characteristics of the game environment.

[0004] However, the following problems still exist in the existing technology.

[0005] Traditional methods for detecting anomalies in auxiliary software often rely on single-dimensional feature judgments, such as single operation response time and hit accuracy. These methods are difficult to adapt to the diverse avoidance methods of unauthorized microservice software tools in complex terminal scenarios, and are prone to misjudging normal players or missing hidden violations. Summary of the Invention

[0006] To address this, the present invention provides a microservice software optimization method based on reinforcement learning, which overcomes the shortcomings of existing technologies. Traditional game anomaly detection methods often rely on single-dimensional feature judgments, making them ill-suited for the diverse evasion methods of unauthorized microservice software tools in complex terminal scenarios. This often results in misjudging legitimate players or missing subtle violations.

[0007] To achieve the above objectives, this invention provides a microservice software optimization method based on reinforcement learning, comprising:

[0008] Historical operation data of the user terminal is acquired to learn the operation rhythm characteristics of the user terminal, including the target aiming reaction time and the target hitting aiming deviation amplitude within the field of view.

[0009] By combining the operation rhythm characteristics and the uniformity of the repeated striking interval, the operation rhythm characterization value of the user terminal is analyzed to mark the rhythm steady state category of the user terminal.

[0010] In response to a user client being flagged as belonging to a low-rhythm steady-state category, anomaly assessment and analysis of the user client's game operations is performed, including:

[0011] Identify the tight rhythm phase, extract the operation connection features of the tight rhythm phase and the jitter amplitude of the view at the moment of firing to calculate the connection smoothness characterization value of the user terminal, so as to determine whether the user terminal meets the connection steady state benchmark.

[0012] Based on the concentrated area of ​​the target area and the target switching time of the user terminal corresponding to the fast-paced phase, it is determined whether the operation of the user terminal is abnormal. Furthermore, based on the data transmission characteristics, it is determined whether the user terminal has a tendency to apply unauthorized microservice software.

[0013] Based on the condition determination results, combined with the matching degree between the aiming timestamp of the target and the target's updated position timestamp, as well as the angle between the viewing angle and the firing direction, it is determined whether the user terminal is using unauthorized microservice software.

[0014] The operation connection features include the response time difference between operation and implementation and the uniformity of response time; the data transmission features include the uniformity of data packet reporting time intervals, the reporting frequency of data packets, and the repetition rate of data packets.

[0015] Furthermore, the process of analyzing the user terminal's operation rhythm representation value includes:

[0016] The sum of the ratio of the target aiming reaction time to the aiming reaction time threshold and the ratio of the target hitting aiming deviation amplitude to the target hitting aiming deviation amplitude threshold is taken as the first rhythm steady-state feature.

[0017] The ratio of the uniformity of the repeated hitting interval to the repetition uniformity threshold is used as the second rhythm steady-state feature;

[0018] The weighted sum of the first rhythm steady-state feature and the second rhythm steady-state feature is used to determine the operation rhythm characterization value.

[0019] Further, the rhythm steady-state category of the user terminal is categorized, including:

[0020] If the user's operation rhythm representation value is less than the user's operation rhythm representation threshold, then the user is marked as a low-rhythm steady-state category.

[0021] Furthermore, the process of identifying the fast-paced phase includes:

[0022] If the CPU utilization rate is greater than the CPU utilization rate threshold and the frame rate fluctuation is greater than the frame rate fluctuation threshold, then the game is determined to have entered the fast-paced phase.

[0023] Further, the process of calculating the connection smoothness representation value of the user terminal includes:

[0024] The sum of the ratio of the response time difference between operation and implementation to the response time difference threshold and the ratio of the uniformity of response time to the uniformity threshold is used as the first smooth transition feature.

[0025] The ratio of the camera shake amplitude at the moment of firing to the shake amplitude threshold is used as the second smooth transition feature;

[0026] The first smooth transition feature and the second smooth transition feature are weighted and summed to determine the smooth transition characterization value.

[0027] Further, determining whether the user terminal meets the connection steady-state benchmark includes:

[0028] If the connection smoothness representation value on the user's end is less than the connection smoothness representation threshold, it is determined that it does not meet the connection steady-state benchmark.

[0029] Further, determining whether the user's operation is abnormal includes:

[0030] If the user terminal does not meet the hitting state conditions, it is determined that the operation of the user terminal is abnormal;

[0031] The striking state conditions include the user terminal's concentrated area range for the striking target being greater than a concentrated area range threshold, and the striking target switching time being greater than a striking target switching time threshold.

[0032] Furthermore, the process of determining whether the user client has a tendency to apply unauthorized microservice software includes:

[0033] The sum of the ratio of the interval uniformity threshold to the interval uniformity of the data packet reporting time and the ratio of the data packet reporting frequency to the reporting frequency threshold is used as the first data anomaly feature.

[0034] The ratio of the data packet duplication rate to the duplication rate threshold is used as the second data anomaly feature;

[0035] The first data anomaly feature and the second data anomaly feature are weighted and summed to determine the data anomaly characterization parameter;

[0036] If the data anomaly representation parameter is greater than or equal to the data anomaly representation parameter threshold, it is determined that the user client has a tendency to use unauthorized applications.

[0037] Furthermore, if the conditional judgment result indicates that the user terminal does not meet the connection steady-state benchmark and the user terminal has a tendency to apply unauthorized microservice software, then the matching degree between the aiming timestamp of the target and the target update position timestamp, as well as the angle between the screen view and the firing direction, is used to determine whether the user terminal is applying unauthorized microservice software.

[0038] Further, determining whether the user terminal is using unauthorized microservice software includes:

[0039] If the user terminal meets the hit matching requirements, it is determined that the user terminal application is not authorized microservice software;

[0040] The strike matching requirements include that the matching degree between the aiming timestamp of the strike target and the updated position timestamp of the strike target is greater than a matching degree threshold, and the angle between the viewpoint and the firing direction is less than an angle threshold.

[0041] Compared to existing technologies, this invention extracts the user's operational rhythm characteristics by acquiring historical operation data from the user's terminal. Combining these operational rhythm characteristics with the uniformity of repeated attack intervals, it analyzes the user's operational rhythm representation value to label the user's rhythm steady-state category. In response to the user being labeled as having a low rhythm steady-state category, it performs anomaly assessment analysis of the user's game operations. Based on the conditional judgment results, and combining the matching degree between the aiming timestamp and the target's updated position timestamp, as well as the angle between the screen view and the firing direction, it determines whether the user's terminal is using unauthorized microservice software. This invention focuses on the accurate identification of unauthorized microservice software tools, such as cheats, in terminal scenarios. Through multi-dimensional feature fusion, layered detection logic, and cross-validation mechanisms, it improves the accuracy of anomaly detection and reduces the probability of false positives and false negatives.

[0042] In particular, in practice, the core characteristics of user-side game operations are the coordination of accuracy and consistency. However, the rhythm of human manual operation exhibits reasonable fluctuations, such as slight differences in aiming reaction time and natural intervals between repeated shots. In contrast, operations performed by unauthorized microservice software tools often exhibit excessively high accuracy and abnormal consistency. By combining these two types of features, the essential differences between human and microservice software tool operations can be accurately captured, making the detection logic more aligned with the actual operational patterns of the terminal scenario and improving the effectiveness of anomaly identification. Specifically, the target aiming reaction time within the field of view refers to the time interval from when the target appears in the user's field of view to when the aiming operation is initiated. Since the response time of human operation exhibits natural fluctuations, while the response time of microservice software tools is extremely fast and exhibits low fluctuations—a characteristic of excessively short reaction time—this feature is used to quantify the reasonableness of the reaction speed. The target hitting aiming deviation refers to the degree of deviation between the aiming point and the target area during the "aiming at the target - executing the shot" process. Because unauthorized microservice software tools forcibly aim at the core target area, the deviation can approach zero, while human operation exhibits random fluctuations. This feature is used to quantify the naturalness of aiming accuracy. Combining the uniformity of the repeated hit interval length quantifies the continuity of continuous hitting operations performed by the user, unauthorized microservice software tools trigger hits at a fixed frequency, resulting in a highly uniform repeated hit interval length—a mechanically uniform interval—used to quantify the rhythmic rationality of continuous operations. Therefore, this invention comprehensively characterizes the overall rhythmic state of the user's game operations through the aforementioned features, jointly quantifying the naturalness, rationality, and authenticity of the user's game operations. It then analyzes the operation rhythm representation value to characterize the degree of abnormal tendencies in the user's operation rhythm, providing data support for subsequent user labeling. This invention achieves quantitative grading of user operation rhythm, accurately screening out potential abnormal users with low rhythm stability, avoiding misjudgment of normal user terminals, and improving the targeting of anomaly detection.

[0043] In particular, this invention first targets high-load scenarios with tight timeframes, such as team battles and intensive monster-hunting in games. These scenarios are core applications of unauthorized microservice software tools. In such scenarios, the advantages of microservice software tools—"automatic aiming, rapid connection, and mechanical operation"—are more prominent, while human manual operation is prone to natural fluctuations in operation connection due to high load. By performing connection smoothness detection for these high-risk scenarios, the resource waste of "indiscriminate detection across all scenarios" is avoided, making anomaly detection more closely match the risk distribution of actual game operation scenarios and significantly improving scenario adaptability. In reality, the "command issuance-operation execution" of human manual operation has natural time difference fluctuations and low response time uniformity. Microservice software tools, on the other hand, trigger operations through programs, resulting in extremely small response time differences and extremely high uniformity, exhibiting a mechanical and unfluctuating characteristic. By quantifying the connection efficiency of "command-execution" and the rhythmic regularity of continuous user operations through operation connection characteristics, the difference between "natural fluctuations" and "mechanical consistency" is reflected. Furthermore, when humans manually fire, slight, unconscious camera shake is inevitable. However, microservice software tools directly modify the game's camera control logic, so the sudden change in camera shake after locking onto a target and firing can approach zero. This further quantifies the physical naturalness of the user's action at the moment of firing by combining the camera shake amplitude with the actual camera shake. Therefore, this invention calculates the smoothness characteristic value of the user's actions from three dimensions: immediacy, rhythmic regularity, and physical naturalness. This reflects the core state of actions during fast-paced phases, distinguishing between the "natural limitations and fluctuations of human manual operation" and the "stability and regularity of the mechanical operation of microservice software tools," providing data support for subsequent determination of whether the user's actions meet the steady-state benchmark. This invention improves the accuracy of anomaly detection by using both smoothness indicators and the rationality of the firing behavior for dual verification.

[0044] In particular, this invention establishes a correlation verification logic between operational behavior and data transmission characteristics. In practice, unauthorized microservice software tools not only cause operational anomalies but also trigger regular anomalies in data packet transmission. The correlation verification logic designed in this invention achieves comprehensive detection of unauthorized microservice software, reducing the false negative rate. In terms of operational behavior, the invention determines whether the operation conforms to the patterns of human manual operation from two perspectives: "spatial distribution rationality" and "time efficiency naturalness," by considering the range of the concentrated area of ​​the striking target and the time spent switching between striking targets. Even with precise human manual operation, factors such as target movement and operational adjustments can lead to a reasonable dispersion of the striking area, resulting in a relatively large concentrated striking range. Unauthorized microservice software tools, however, forcibly lock onto the core area of ​​the target, resulting in an extremely small concentrated striking area, exceeding the limits of human operation. Based on this, "overly concentrated mechanical striking" is identified by analyzing the concentrated area of ​​the striking target. Furthermore, when humans switch striking targets, they undergo a physiological process of "identifying a new target - adjusting aim - executing the strike," which naturally takes time. Microservice software tools, on the other hand, can instantly switch and lock onto targets, with extremely short switching times. Based on this, the "rapid switching beyond physiological limits" can be accurately captured by measuring the time taken to switch targets. The combination of the two aforementioned characteristics can accurately identify the degree of abnormality in mechanical operations. In terms of data transmission, since unauthorized microservice software tools are essentially third-party programs, their data packet interactions with the server must follow program logic, rather than random triggering by human operation. Therefore, if a user applies an unauthorized microservice software tool, it will exhibit extremely high uniformity in the intervals between data packet reporting times (i.e., fixed-time reporting), abnormal data packet reporting frequency (i.e., exceeding the triggering frequency of normal operation), and high data packet repetition rate (i.e., the program repeatedly sends the same instruction packets). Furthermore, this invention relies on data transmission characteristics to capture the program operation characteristics of unauthorized microservice software tools. For example, plug-ins send data packets through fixed logic, leading to abnormal transmission patterns. Therefore, this invention determines data anomaly characterization parameters to quantify the degree of mechanical deviation in data transmission time, the degree of reasonable deviation in transmission frequency, and the degree of redundancy deviation in transmitted content. This comprehensively characterizes the degree of deviation from the normal mode during data transmission, enabling accurate identification of anomalies at the data transmission level, avoiding misjudgments caused by normal factors such as network fluctuations, and improving the robustness of detection.

[0045] In particular, based on preliminary screening, this invention conducts final confirmation of unauthorized microservice software applications. Two indicators are selected: the matching degree between the aiming timestamp and the target's updated position timestamp, and the angle between the viewing angle and the firing direction. These are used to identify the core technical features of the unauthorized microservice software tool: forced synchronization of dynamic position and mechanical alignment of operational logic. In reality, human manual operation has physiological delays, making it difficult to perfectly synchronize aiming actions with the real-time target position update. However, the unauthorized microservice software tool tracks the target position in real time through its program, achieving a high degree of synchronization between the aiming timestamp and the target's updated position timestamp. Based on this, the matching degree indicator quantifies the "reasonableness of synchronization between aiming actions and the target's dynamic position," distinguishing between "natural delays in manual aiming" and "real-time tracking by the tool." Furthermore, during manual operation, there may be slight deviations between the viewing angle (observation direction) and the firing direction (attack direction). The unauthorized microservice software tool forces the firing direction to align with the target position, causing the angle between the viewing angle and the firing direction to approach zero. Based on this, the "logical consistency between the observation direction and the attack direction" is quantified by the angle between the screen view and the firing direction, accurately capturing the "forced alignment" feature of tool operations. This invention cross-validates from two dimensions—dynamic synchronization and logical consistency—effectively distinguishing between "high-level human operation" and "unauthorized microservice software tool operation," significantly improving the accuracy of anomaly identification. Therefore, this invention incorporates a timestamp matching index to effectively eliminate operational deviations caused by network latency, ensuring that the judgment result is not affected by the network environment. Simultaneously, the angle between the screen view and the firing direction verifies the consistency of the operational logic, avoiding misjudging "high-level normal operation" as unauthorized microservice software. Based on a multi-stage, multi-indicator, progressive judgment, a complete detection chain of "screening-verification-confirmation" is formed, ensuring both the comprehensiveness of anomaly detection and the accuracy of the final judgment, significantly reducing the false positive and false negative rates, and providing reliable technical protection for the fairness of the game environment. Attached Figure Description

[0046] Figure 1 This is a schematic diagram illustrating the steps of a reinforcement learning-based microservice software optimization method according to an embodiment of the invention.

[0047] Figure 2 A logic decision diagram for marking the steady-state category of the user terminal in an embodiment of the invention;

[0048] Figure 3 A logic diagram for determining whether a user terminal conforms to a connection steady-state benchmark in an embodiment of the invention;

[0049] Figure 4 This is a logic diagram for determining whether a user client has a tendency to use unauthorized microservice software, as described in an embodiment of the invention. Detailed Implementation

[0050] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0051] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0052] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0053] Please see Figure 1 The diagram illustrates the steps of a reinforcement learning-based microservice software optimization method according to an embodiment of the present invention. The reinforcement learning-based microservice software optimization method according to an embodiment of the present invention includes:

[0054] Step S1: Obtain historical operation data from the user terminal to learn the operation rhythm characteristics of the user terminal, including the target aiming reaction time within the field of view and the target hitting aiming deviation amplitude.

[0055] Step S2: Combine the operation rhythm characteristics and the uniformity of the repeated striking interval duration to analyze the operation rhythm representation value of the user terminal, so as to mark the rhythm steady state category of the user terminal.

[0056] Step S3, in response to the user terminal being marked as belonging to the low-rhythm steady-state category, performs anomaly assessment and analysis of the user terminal's game operations, including,

[0057] Identify the tight rhythm phase, extract the operation connection features of the tight rhythm phase and the jitter amplitude of the view at the moment of firing to calculate the connection smoothness characterization value of the user terminal, so as to determine whether the user terminal meets the connection steady state benchmark.

[0058] Based on the concentrated area of ​​the target area and the target switching time of the user terminal corresponding to the fast-paced phase, it is determined whether the operation of the user terminal is abnormal. Furthermore, based on the data transmission characteristics, it is determined whether the user terminal has a tendency to apply unauthorized microservice software.

[0059] Step S4: Based on the condition determination result, combined with the matching degree between the aiming timestamp of the target and the target update position timestamp, as well as the angle between the screen view and the firing direction, determine whether the user terminal is using unauthorized microservice software.

[0060] The operation connection features include the response time difference between operation and implementation and the uniformity of response time; the data transmission features include the uniformity of data packet reporting time intervals, the reporting frequency of data packets, and the repetition rate of data packets.

[0061] The historical operation data includes operation rhythm characteristics and the uniformity of repeated striking intervals, etc.

[0062] When determining whether a user is using unauthorized microservice software, several measures can be taken, such as sending a warning and then closing the current game process, or other measures that have been authorized in advance by the user agreement. These will not be elaborated further.

[0063] Specifically, there are no specific limitations on the methods for collecting and acquiring relevant feature data. Some feature data are raw data generated during game operations or system operation, which can be directly captured through game engine interfaces, operation logs, or server-side data points. These include: operation rhythm features, CPU usage, frame rate fluctuations, response time difference between operation and implementation, data transmission features, aiming timestamps for targets, timestamps for target position updates, and the angle between the screen view and the firing direction. These will not be elaborated further.

[0064] Furthermore, based on the original data, other relevant feature data are obtained through statistical analysis, mathematical modeling, or algorithmic calculation. Timestamps of consecutive striking operations can be extracted from the operation logs, the time interval between two adjacent strikes can be calculated, and the uniformity of the repeated striking interval duration can be quantified using the coefficient of variation, where the coefficient of variation is the ratio of the standard deviation to the mean of the striking interval duration. Correspondingly, the uniformity of the response time can be determined in the same way.

[0065] Based on the viewpoint coordinates of the user's viewpoint before and after the firing time, such as from 50ms before firing to 50ms after firing, the maximum fluctuation value of the viewpoint coordinates within this time window is calculated, which is the jitter amplitude of the viewpoint at the firing time.

[0066] Extract the hit coordinates of the strike operations performed by the user terminal during the fast-paced phase from the operation log, that is, the coordinates of the hit points on the target. Calculate the coverage area of ​​the hit coordinates using the "minimum circumcircle" or "bounding box" algorithm, such as the radius of the circumcircle and the side length of the bounding box, which is the concentrated area range of the user terminal's strikes on the target.

[0067] The key time points recorded in the raw data for the user terminal switching from the current target to a new target are obtained, including the first timestamp of stopping shooting at the current target (i.e., the timestamp corresponding to the last shooting command for the current target) and the second timestamp of starting shooting at the new target (i.e., the timestamp corresponding to the first shooting command for the new target). The difference between the second timestamp and the first timestamp is then used as the target switching time.

[0068] By extracting the aiming timestamp T1 from the user's terminal and the nearest position update timestamp T2 of the target before and after time T1, the absolute value of the difference between the aiming timestamp and the position update timestamp is calculated and normalized to convert it into the matching degree between the aiming timestamp and the target's updated position timestamp. The smaller the absolute value of the difference, the greater the matching degree.

[0069] Specifically, the process of analyzing the user's operational rhythm representation value includes:

[0070] The sum of the ratio of the target aiming reaction time to the aiming reaction time threshold and the ratio of the target hitting aiming deviation amplitude to the target hitting aiming deviation amplitude threshold is taken as the first rhythm steady-state feature.

[0071] The ratio of the uniformity of the repeated hitting interval to the repetition uniformity threshold is used as the second rhythm steady-state feature;

[0072] The weighted sum of the first rhythm steady-state feature and the second rhythm steady-state feature is used to determine the operation rhythm characterization value.

[0073] Specifically, the core of unauthorized microservice software tools lies in exceeding the physiological limits of human operation. Operation rhythm characteristics can quantify this limit-breaking, serving as the most crucial dimension distinguishing normal operation from unauthorized microservice software tool operation. In reality, humans are limited by physiological neural conduction, resulting in a minimum reaction delay. Microservice software tools, however, can achieve zero-latency aiming. Simultaneously, human manual operation is affected by device control, such as mouse input and physiological jitter, leading to a certain aiming deviation. Microservice software tools, on the other hand, can achieve zero-deviation locking. These two characteristics reflect whether the user's operation exceeds human physiological and physical limits. This is the most core and difficult-to-imitate abnormal characteristic of unauthorized microservice software tools, serving as the core anchor point for anomaly detection. The uniformity of the repeated hitting interval reflects the mechanical regularity of continuous operation, such as the fixed-interval hitting of cheats. However, this characteristic carries a certain "risk of overlap with normal operation," thus it is used as a dimension for microservice software detection. Therefore, in the weighted summation, the first rhythm steady-state characteristic calculated based on the operation rhythm characteristics—namely, the target aiming reaction time within the field of view and the target hitting aiming deviation—is given a higher weight coefficient, set to 0.65. Accordingly, the weighting coefficient of the second rhythm steady-state feature calculated based on the uniformity of the repeated striking interval is set to 0.35.

[0074] In this embodiment, the purpose of setting the aiming reaction time threshold, the target hitting aiming deviation amplitude threshold, and the repetition uniformity threshold is to characterize situations where the user's operation rhythm exhibits a high degree of abnormal tendency. By acquiring historical operation data corresponding to several sessions completed by the user, and calling the target aiming reaction time data, target hitting aiming deviation amplitude data, and repetition interval uniformity data within the field of view, the average aiming reaction time, the average target hitting aiming deviation amplitude, and the average repetition uniformity are calculated and used as the baseline values ​​under normal conditions. Based on the purpose of setting the above three thresholds, the aiming... The reaction time threshold is determined as the product of the average aiming reaction time and the reaction deviation coefficient. The target hitting aiming deviation amplitude threshold is determined as the product of the average target hitting aiming deviation amplitude and the aiming deviation coefficient. The repeatability uniformity threshold is determined as the product of repeatability uniformity and repeatability deviation coefficient. The reaction deviation coefficient is selected within the interval [1.1, 1.15], preferably 1.1 in practice. The aiming deviation coefficient is selected within the interval [1.1, 1.2], preferably 1.1 in practice. The repeatability deviation coefficient is selected within the interval [1.2, 1.3], preferably 1.2 in practice.

[0075] Specifically, in practice, the core characteristics of user-side game operations are the coordination of accuracy and consistency. However, the rhythm of human manual operation exhibits reasonable fluctuations, such as slight differences in aiming reaction time and natural intervals between repeated shots. In contrast, operations performed by unauthorized microservice software tools often exhibit excessively high accuracy and abnormal consistency, such as aiming without delay and mechanically repetitive shots. By combining these two types of characteristics, the essential differences between human operation and microservice software tool operation can be accurately captured, making the detection logic more aligned with the actual operational patterns of the terminal scenario and improving the effectiveness of anomaly identification. Specifically, the target aiming reaction time within the field of view refers to the time interval from when the target appears in the user's field of view to when the aiming operation is initiated. Since the response time of human operation exhibits natural fluctuations, while the response time of microservice software tools is extremely fast and exhibits low fluctuations, i.e., an abnormal characteristic of excessively short reaction time, this feature is used to quantify the reasonableness of the reaction speed. The target aiming deviation amplitude refers to the degree of deviation between the aiming point and the target area during the "aiming at the target - executing the shot" process. Because unauthorized microservice software tools, such as aimbots / lock-on-health cheats, force aiming at the core area of ​​the target, the deviation can approach zero, while human operation exhibits random fluctuations. This feature is used to quantify the naturalness of aiming accuracy. Combining the uniformity of the repetitive attack interval, the consistency of continuous attack operations performed by the user is quantified. Unauthorized microservice software tools, such as automatic rapid-fire cheats, trigger attacks at a fixed frequency, resulting in a highly uniform repetitive attack interval, i.e., a mechanically uniform interval, used to quantify the rationality of the rhythm of continuous operation. Therefore, this invention comprehensively characterizes the overall rhythm state of the user's game operation through the aforementioned features, jointly quantifying the naturalness, rationality, and authenticity of the user's game operation, and then analyzing the operation rhythm representation value to characterize the degree of abnormal tendency of the user's operation rhythm, providing data support for subsequent user labeling. This invention achieves quantitative classification of user operation rhythm, accurately screening out potential abnormal users with low rhythm stability, avoiding misjudgment of normal users, and improving the targeting of anomaly detection.

[0076] Specifically, please refer to Figure 2 As shown, it is a logic decision diagram for marking the rhythm steady-state category of the user terminal according to an embodiment of the present invention. Marking the rhythm steady-state category of the user terminal includes:

[0077] If the user's operation rhythm representation value is less than the user's operation rhythm representation threshold, then the user is marked as a low-rhythm steady-state category.

[0078] If the user terminal's operation rhythm representation value is greater than or equal to the user terminal's operation rhythm representation threshold, then there is no need to mark the user terminal.

[0079] The operation rhythm characterization threshold is predetermined. The operation rhythm characterization value calculated under the following conditions is equal to the target aiming reaction time threshold, the target hitting aiming deviation amplitude threshold, and the uniformity of the repeated hitting interval duration threshold.

[0080] Specifically, the process of identifying the fast-paced phases includes:

[0081] If the CPU utilization rate is greater than the CPU utilization rate threshold and the frame rate fluctuation is greater than the frame rate fluctuation threshold, then the game is determined to have entered the fast-paced phase.

[0082] In this embodiment, the purpose of setting the CPU utilization threshold and the frame rate fluctuation threshold is to identify high-load game phases where unauthorized microservice software tools are frequently used. By obtaining system resource data corresponding to several game sessions completed by the user, calling the CPU utilization data and frame rate fluctuation data, the average CPU utilization and the average frame rate fluctuation are calculated and used as the baseline values ​​under normal conditions. Based on the purpose of setting the above two thresholds, the CPU utilization threshold is determined as the product of the average CPU utilization and the utilization deviation coefficient, and the frame rate fluctuation threshold is determined as the product of the average frame rate fluctuation and the fluctuation deviation coefficient. The utilization deviation coefficient is selected in the interval [1.2, 1.4], preferably 1.2 in practice, and the fluctuation deviation coefficient is selected in the interval [1.3, 1.5], preferably 1.3 in practice.

[0083] The system resource data includes CPU utilization and frame rate fluctuations, among other things.

[0084] Specifically, the process of calculating the connection smoothness representation value of the user terminal includes:

[0085] The sum of the ratio of the response time difference between operation and implementation to the response time difference threshold and the ratio of the uniformity of response time to the uniformity threshold is used as the first smooth transition feature.

[0086] The ratio of the camera shake amplitude at the moment of firing to the shake amplitude threshold is used as the second smooth transition feature;

[0087] The first smooth transition feature and the second smooth transition feature are weighted and summed to determine the smooth transition characterization value.

[0088] Specifically, the core purpose of setting a smooth transition characteristic value is to quantify the naturalness and physical authenticity of the operation transition. However, the ability of operation transition features and the jitter amplitude of the viewpoint at the moment of firing to identify "abnormal operations" differs fundamentally. The jitter amplitude of the viewpoint at the moment of firing directly corresponds to the core of unauthorized microservice software tools breaking through human physiological / physical limitations, and is a rigid basis for anomaly judgment. The core technical logic of unauthorized microservice software tools is to force a stable viewpoint, creating a clear boundary between abnormal and normal operations on the regional user end. The jitter amplitude of the viewpoint at the moment of firing can approach 0, a characteristic that is the essential attribute of unauthorized microservice software tools that is "unfakeable and cannot be imitated by normal operations." Operation transition features, on the other hand, are used to quantify the rhythmic regularity of operations. Skilled players can infinitely approach a "mechanical rhythm" through long-term training, but this differs from the "zero latency" of unauthorized microservice software tools. Therefore, in the weighted summation, the second smooth transition feature calculated based on the jitter amplitude of the viewpoint at the moment of firing is given a higher weight coefficient, set to 0.6. Accordingly, the weighting coefficient of the first smoothness feature, calculated based on the operational connection characteristics, namely the response time difference between operation and implementation and the uniformity of response time, is set to 0.4.

[0089] In this embodiment, the purpose of setting the response time difference threshold, uniformity threshold, and jitter amplitude threshold is to characterize situations where the operation connection state presented by the user terminal is relatively stable and regular, and the possibility of using unauthorized microservice software tools is high. This is achieved by obtaining historical operation data corresponding to several sessions completed by the user terminal, calling the response time difference data between operation and implementation, the uniformity data of response time, and the jitter amplitude data from the firing moment perspective, calculating the mean response time difference, the mean uniformity, and the mean jitter amplitude, and using these as the baseline values ​​under normal circumstances. Based on setting the above three thresholds... The objective is to determine the response time difference threshold as the product of the mean response time difference and the response deviation coefficient, the uniformity threshold as the product of the mean uniformity and the uniformity deviation coefficient, and the jitter amplitude threshold as the product of the mean jitter amplitude and the jitter deviation coefficient. Specifically, the response deviation coefficient is selected within the interval [1.2, 1.3], preferably 1.2; the uniformity deviation coefficient is selected within the interval [1.1, 1.2], preferably 1.1; and the jitter deviation coefficient is selected within the interval [1.15, 1.25], preferably 1.15.

[0090] Specifically, this invention first targets high-load scenarios with tight timeframes, such as team battles and intensive monster-hunting in games. These scenarios are core applications of unauthorized microservice software tools. In such scenarios, the advantages of microservice software tools—"automatic aiming, rapid connection, and mechanical operation"—are more prominent, while human manual operation is prone to natural fluctuations in operation connection due to high load. By performing connection smoothness detection for these high-risk scenarios, the resource waste of "indiscriminate detection across all scenarios" is avoided, making anomaly detection more closely match the risk distribution of actual game operation scenarios and significantly improving scenario adaptability. In reality, the "command issuance-operation execution" of human manual operation has natural time difference fluctuations, such as occasional stutters and reaction adjustments during continuous operation, resulting in low response time uniformity. Microservice software tools, on the other hand, trigger operations through programs, exhibiting extremely small and highly uniform response time differences, displaying a mechanical and unfluctuating characteristic. By quantifying the connection efficiency of "command-execution" and the rhythmic regularity of continuous user operations through operation connection characteristics, the difference between "natural fluctuations" and "mechanical consistency" is reflected. Furthermore, when humans manually fire, slight, unconscious camera shake is inevitable. However, microservice software tools directly modify the game's camera control logic, so the sudden change in camera shake after locking onto a target and firing can approach zero. This further quantifies the physical naturalness of the user's action at the moment of firing by combining the camera shake amplitude with the actual camera shake. Therefore, this invention calculates the smoothness characteristic value of the user's actions from three dimensions: immediacy, rhythmic regularity, and physical naturalness. This reflects the core state of actions during fast-paced phases, distinguishing between the "natural limitations and fluctuations of human manual operation" and the "stability and regularity of the mechanical operation of microservice software tools," providing data support for subsequent determination of whether the user's actions meet the steady-state benchmark. This invention improves the accuracy of anomaly detection by using both smoothness indicators and the rationality of the firing behavior for dual verification.

[0091] Specifically, please refer to Figure 3 As shown, this is a logic diagram for determining whether a user terminal conforms to the connection steady-state benchmark in an embodiment of the present invention. Determining whether the user terminal conforms to the connection steady-state benchmark includes:

[0092] If the connection smoothness representation value on the user side is less than the connection smoothness representation threshold, it is determined that it does not meet the connection steady-state benchmark.

[0093] If the connection smoothness representation value on the user side is greater than or equal to the connection smoothness representation threshold, it is determined that the connection steady-state benchmark is met.

[0094] The smoothness of connection characterization threshold is predetermined. The smoothness of connection characterization value calculated is determined when the response time difference between operation and implementation is equal to the response time difference threshold, the uniformity of response time is equal to the uniformity threshold, and the jitter amplitude of the viewpoint at the moment of firing is equal to the jitter amplitude threshold.

[0095] Specifically, determining whether the user's operation is abnormal includes:

[0096] If the user terminal does not meet the hitting state conditions, it is determined that the operation of the user terminal is abnormal;

[0097] The striking state conditions include the user terminal's concentrated area range for the striking target being greater than a concentrated area range threshold, and the striking target switching time being greater than a striking target switching time threshold.

[0098] In this embodiment, the purpose of setting the concentrated area range threshold and the target switching time threshold is to characterize situations where the operation performed by the user terminal is highly mechanical and abnormal. By acquiring historical operation data corresponding to several sessions completed by the user terminal, the concentrated area range data and target switching time data of the user terminal for the target are called, and the average value of the concentrated area range and the average value of the target switching time are calculated. These are then used as the benchmark values ​​under normal circumstances. Based on the purpose of setting the above two thresholds, the concentrated area range threshold is determined to be the product of the average value of the concentrated area range and the range deviation coefficient, and the target switching time threshold is determined to be the product of the average target switching time and the time deviation coefficient. The range deviation coefficient is selected within the interval [1.1, 1.2], preferably 1.1 in the implementation, and the time deviation coefficient is selected within the interval [1.2, 1.3], preferably 1.2 in the implementation.

[0099] Specifically, please refer to Figure 4 As shown, this is a logic diagram for determining whether a user client has a tendency to use unauthorized microservice software, according to an embodiment of the present invention. The process of determining whether a user client has a tendency to use unauthorized microservice software includes:

[0100] The sum of the ratio of the interval uniformity threshold to the interval uniformity of the data packet reporting time and the ratio of the data packet reporting frequency to the reporting frequency threshold is used as the first data anomaly feature.

[0101] The ratio of the data packet duplication rate to the duplication rate threshold is used as the second data anomaly feature;

[0102] The first data anomaly feature and the second data anomaly feature are weighted and summed to determine the data anomaly characterization parameter;

[0103] If the data anomaly representation parameter is greater than or equal to the data anomaly representation parameter threshold, it is determined that the user client has a tendency to use unauthorized applications.

[0104] If the data anomaly representation parameter is less than the data anomaly representation parameter threshold, it is determined that there is no tendency for the user to use the application without authorization.

[0105] Specifically, the program-level transmission characteristics of unauthorized microservice software tools are captured through data anomaly characterization parameters. In reality, unauthorized microservice software tools trigger operations through fixed program logic, resulting in a "mechanically uniform" data packet reporting interval. In contrast, normal user operations are influenced by human physiological rhythms and scenario judgments, leading to natural fluctuations in the interval. This difference is the essential distinction between "program-triggered" and "human-triggered" operations, and cannot be imitated through normal operation. Furthermore, unauthorized microservice software tools can exceed the physiological limits of human operation, resulting in a high actual reporting frequency; while the reporting frequency of normal users is limited by hand keystroke speed and operational logic, and will not exceed the limit. This characteristic directly reflects "superhuman operation frequency," one of the core characteristics of microservice software tools, and carries no risk of overlapping with normal operations. Both of these characteristics directly anchor the "program-controlled nature" of microservice software tools. Furthermore, by combining the data packet repetition rate, a comprehensive identification of various unauthorized microservice software tools is achieved, avoiding the omission of cheating scenarios involving "non-repeated packet sending." Therefore, in the weighted summation, the first data anomaly feature, calculated based on the uniformity of the data packet reporting time interval and the reporting frequency of the data packets, is given a higher weight coefficient, set to 0.6. Correspondingly, the weight coefficient of the second data anomaly feature, calculated based on the data packet repetition rate, is set to 0.4.

[0106] In this embodiment, the purpose of setting the interval uniformity threshold, reporting frequency threshold, and repetition rate threshold is to characterize the degree of mechanical deviation in data transmission time, the degree of reasonable deviation in transmission frequency, and the degree of redundancy deviation in transmitted content. This comprehensively characterizes situations where the deviation from the normal mode is significant during data transmission. By acquiring historical operation data corresponding to several sessions completed by the user terminal, and calling the interval uniformity data, reporting frequency data, and repetition rate data of data packet reporting time, the average interval uniformity, average reporting frequency, and average repetition rate are calculated and used as the corresponding values ​​under normal conditions. Based on the purpose of setting the above three thresholds, the interval uniformity threshold is determined as the product of the mean interval uniformity and the first deviation coefficient, the reporting frequency threshold is determined as the product of the mean reporting frequency and the second deviation coefficient, and the repetition rate threshold is determined as the product of the mean repetition rate and the third deviation coefficient. The first deviation coefficient is selected within the interval [0.95, 0.98], preferably 0.95 in practice; the second deviation coefficient is selected within the interval [1.25, 1.35], preferably 1.25 in practice; and the third deviation coefficient is selected within the interval [1.2, 1.4], preferably 1.2 in practice.

[0107] The threshold for data anomaly characterization parameters is predetermined. The data anomaly characterization parameters calculated under the following conditions are equal to the interval uniformity threshold, the data packet reporting frequency, and the data packet duplication rate.

[0108] Specifically, this invention establishes a correlation verification logic between operational behavior and data transmission characteristics. In reality, unauthorized microservice software tools not only cause operational anomalies but also trigger regular anomalies in data packet transmission. The correlation verification logic designed in this invention achieves comprehensive detection of unauthorized microservice software, reducing the false negative rate. In terms of operational behavior, by considering the range of the concentrated area of ​​the target being struck and the time spent switching targets, the invention determines whether the operation conforms to the patterns of human manual operation from two perspectives: "rationality of spatial distribution" and "naturalness of time efficiency." For example, the area of ​​human target striking will naturally disperse, and target switching involves physiological reaction time. Even with precise human manual operation, factors such as target movement and operational adjustments will result in a reasonable dispersion of the striking area, meaning a relatively large concentrated striking range. Unauthorized microservice software tools, however, will forcibly lock the core area of ​​the target, resulting in an extremely small concentrated striking area, exceeding the limits of human operation. Based on this, "overly concentrated mechanical striking" is identified by considering the concentrated area of ​​the target being struck. Furthermore, when humans switch targets, they undergo a physiological process of "identifying a new target - adjusting aim - executing the attack," which naturally takes time. Microservice software tools, on the other hand, can switch targets instantly, with extremely short switching times. Based on this, the target switching time can accurately capture "rapid switching beyond physiological limits," and the combination of these two characteristics can accurately identify the degree of abnormality in mechanical operations. In terms of data transmission, since unauthorized microservice software tools are essentially third-party programs, their data packet interactions with the server must follow program logic, rather than random triggers from human operation. Therefore, if a user applies an unauthorized microservice software tool, it will exhibit extremely high uniformity in data packet reporting intervals (i.e., fixed-time reporting), abnormal data packet reporting frequency (i.e., exceeding the trigger frequency of normal operation), and high data packet repetition rate (i.e., the program repeatedly sends the same instruction packets). Furthermore, this invention relies on data transmission characteristics to capture the program operation characteristics of unauthorized microservice software tools, such as how plug-ins send data packets through fixed logic, leading to abnormal transmission patterns. Therefore, this invention determines data anomaly characterization parameters to quantify the mechanical deviation of data transmission time, the reasonable deviation of transmission frequency, and the redundancy deviation of transmitted content. This comprehensively characterizes the deviation from the normal mode during data transmission, enabling anomalies at the data transmission level to be accurately identified, avoiding misjudgments caused by normal factors such as network fluctuations, and improving the robustness of detection.

[0109] Specifically, if the user terminal does not meet the connection steady-state benchmark and the user terminal has a tendency to use unauthorized microservice software, then the matching degree between the aiming timestamp of the target and the target update position timestamp, as well as the angle between the screen view and the firing direction, is used to determine whether the user terminal is using unauthorized microservice software.

[0110] Specifically, determining whether the user client is using unauthorized microservice software includes:

[0111] If the user terminal meets the hit matching requirements, it is determined that the user terminal application is not authorized microservice software;

[0112] The strike matching requirements include that the matching degree between the aiming timestamp of the strike target and the updated position timestamp of the strike target is greater than a matching degree threshold, and the angle between the viewpoint and the firing direction is less than an angle threshold.

[0113] In this embodiment, the purpose of setting a matching degree threshold is to characterize the situation where the synchronization degree between the aiming timestamp and the update position timestamp is high, and the purpose of setting an angle threshold is to characterize the situation where the logical consistency between the observation direction and the attack direction is strong. The two together characterize the situation where the rationality of the user's operation is low. By acquiring historical operation data corresponding to several sessions completed by the user, the matching degree data of the aiming timestamp and the update position timestamp of the target, as well as the angle data between the screen view and the firing direction, are called to solve for the mean matching degree and the mean angle, and the corresponding values ​​are used as the benchmark values ​​under normal circumstances. Based on the purpose of setting the above two thresholds, the matching degree threshold is determined to be the product of the mean matching degree and the matching deviation coefficient, and the angle threshold is determined to be the product of the mean angle and the angle deviation coefficient. The matching deviation coefficient is selected in the interval [1.15, 1.3], preferably 1.15 in the implementation, and the angle deviation coefficient is selected in the interval [0.95, 0.98], preferably 0.95 in the implementation.

[0114] Specifically, based on preliminary screening, this invention conducts final confirmation of unauthorized microservice software applications. Two indicators are selected: the matching degree between the aiming timestamp and the target's updated position timestamp, and the angle between the viewing angle and the firing direction. These are used to identify the core technical features of the unauthorized microservice software tool: forced synchronization of dynamic position and mechanical alignment of operational logic. In reality, human manual operation has physiological delays, making it difficult to perfectly synchronize aiming actions with the real-time target position update. However, the unauthorized microservice software tool tracks the target position in real time through its program, achieving a high degree of synchronization between the aiming timestamp and the target's updated position timestamp. Based on this, the matching degree indicator quantifies the "reasonableness of synchronization between aiming actions and the target's dynamic position," distinguishing between "natural delays in manual aiming" and "real-time tracking by the tool." Furthermore, during manual operation, there may be slight deviations between the viewing angle (observation direction) and the firing direction (attack direction). The unauthorized microservice software tool forces the firing direction to align with the target position, causing the angle between the viewing angle and the firing direction to approach zero. Based on this, the "logical consistency between the observation direction and the attack direction" is quantified by the angle between the screen view and the firing direction, accurately capturing the "forced alignment" feature of tool operations. This invention cross-validates from two dimensions—dynamic synchronization and logical consistency—effectively distinguishing between "high-level human operation" and "unauthorized microservice software tool operation," significantly improving the accuracy of anomaly identification. Therefore, this invention incorporates a timestamp matching index to effectively eliminate operational deviations caused by network latency, ensuring that the judgment result is not affected by the network environment. Simultaneously, the angle between the screen view and the firing direction verifies the consistency of the operational logic, avoiding misjudging "high-level normal operation" as unauthorized microservice software. Based on a multi-stage, multi-indicator, progressive judgment, a complete detection chain of "screening-verification-confirmation" is formed, ensuring both the comprehensiveness of anomaly detection and the accuracy of the final judgment, significantly reducing the false positive and false negative rates, and providing reliable technical protection for the fairness of the game environment.

[0115] If the reinforcement learning-based microservice software optimization method of the present invention is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present invention. The aforementioned storage medium includes various media that can store program code, such as USB flash drives, mobile hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0116] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A microservice software optimization method based on reinforcement learning, characterized in that, include: Historical operation data of the user terminal is acquired to learn the operation rhythm characteristics of the user terminal, including the target aiming reaction time within the field of view and the target hitting aiming deviation amplitude. By combining the operation rhythm characteristics and the uniformity of the repeated striking interval, the operation rhythm characterization value of the user terminal is analyzed to mark the rhythm steady state category of the user terminal. In response to the user client being marked as belonging to the low-steady-state rhythm category, anomaly assessment and analysis of the game operation of the user client is performed. include, Identify the tight rhythm phase, extract the operation connection features of the tight rhythm phase and the jitter amplitude of the view at the moment of firing to calculate the connection smoothness characterization value of the user terminal, so as to determine whether the user terminal meets the connection steady state benchmark. Based on the concentrated area of ​​the target area and the target switching time of the user terminal corresponding to the fast-paced phase, it is determined whether the operation of the user terminal is abnormal. Furthermore, based on the data transmission characteristics, it is determined whether the user terminal has a tendency to apply unauthorized microservice software. Based on the condition determination results, combined with the matching degree between the aiming timestamp of the target and the target's updated position timestamp, as well as the angle between the viewing angle and the firing direction, it is determined whether the user terminal is using unauthorized microservice software. The operation connection features include the response time difference between operation and implementation and the uniformity of response time; the data transmission features include the uniformity of data packet reporting time interval, data packet reporting frequency, and data packet repetition rate. The process of analyzing the user terminal's operation rhythm representation value includes: The sum of the ratio of the target aiming reaction time to the aiming reaction time threshold and the ratio of the target hitting aiming deviation amplitude to the target hitting aiming deviation amplitude threshold is taken as the first rhythm steady-state feature. The ratio of the uniformity of the repeated hitting interval to the repetition uniformity threshold is used as the second rhythm steady-state feature; The weighted sum of the first rhythm steady-state feature and the second rhythm steady-state feature is used to determine the operation rhythm characterization value; The process of determining whether the user client has a tendency to apply unauthorized microservice software includes: The sum of the ratio of the interval uniformity threshold to the interval uniformity of the data packet reporting time and the ratio of the data packet reporting frequency to the reporting frequency threshold is used as the first data anomaly feature. The ratio of the data packet duplication rate to the duplication rate threshold is used as the second data anomaly feature; The first data anomaly feature and the second data anomaly feature are weighted and summed to determine the data anomaly characterization parameter; If the data anomaly representation parameter is greater than or equal to the data anomaly representation parameter threshold, it is determined that the user client has a tendency to use unauthorized applications.

2. The microservice software optimization method based on reinforcement learning according to claim 1, characterized in that, The rhythm steady-state category of the user terminal is marked, including: If the user's operation rhythm representation value is less than the user's operation rhythm representation threshold, then the user is marked as a low-rhythm steady-state category.

3. The microservice software optimization method based on reinforcement learning according to claim 1, characterized in that, The process of identifying the fast-paced phases includes: If the CPU utilization rate is greater than the CPU utilization rate threshold and the frame rate fluctuation is greater than the frame rate fluctuation threshold, then the game is determined to have entered the fast-paced phase.

4. The microservice software optimization method based on reinforcement learning according to claim 1, characterized in that, The process of calculating the connection smoothness representation value of the user terminal includes: The sum of the ratio of the response time difference between operation and implementation to the response time difference threshold and the ratio of the uniformity of response time to the uniformity threshold is used as the first smooth transition feature. The ratio of the camera shake amplitude at the moment of firing to the shake amplitude threshold is used as the second smooth transition feature; The first smooth transition feature and the second smooth transition feature are weighted and summed to determine the smooth transition characterization value.

5. The microservice software optimization method based on reinforcement learning according to claim 4, characterized in that, Determining whether the user terminal meets the connection steady-state benchmark includes: If the connection smoothness representation value on the user's end is less than the connection smoothness representation threshold, it is determined that it does not meet the connection steady-state benchmark.

6. The microservice software optimization method based on reinforcement learning according to claim 1, characterized in that, Determining whether the user's operation is abnormal includes: If the user terminal does not meet the hitting state conditions, it is determined that the operation of the user terminal is abnormal; The striking state conditions include the user terminal's concentrated area range for the striking target being greater than a concentrated area range threshold, and the striking target switching time being greater than a striking target switching time threshold.

7. The microservice software optimization method based on reinforcement learning according to claim 1, characterized in that, If the user terminal does not meet the connection steady-state benchmark and the user terminal has a tendency to use unauthorized microservice software, then the matching degree between the aiming timestamp of the target and the target update position timestamp, as well as the angle between the screen view and the firing direction, is used to determine whether the user terminal is using unauthorized microservice software.

8. The microservice software optimization method based on reinforcement learning according to claim 1, characterized in that, Determining whether the user client is using unauthorized microservice software includes: If the user terminal meets the hit matching requirements, it is determined that the user terminal application is not authorized microservice software; The strike matching requirements include that the matching degree between the aiming timestamp of the strike target and the updated position timestamp of the strike target is greater than a matching degree threshold, and the angle between the viewpoint and the firing direction is less than an angle threshold.

Images