Communication module memory leakage test method, system, device, medium and equipment

Abstract

The application discloses a communication module memory leakage test method, system, device, medium and equipment. The method comprises the following steps: constructing a reinforcement learning agent, and configuring a state space, an action space and a reward function corresponding to the reinforcement learning agent; and testing the communication module based on a target test action instruction output by the reinforcement learning agent. The state space maps real-time collected multi-dimensional running state data of the communication module and provides panoramic observation basis; the action space maps test action instructions for inducing memory leakage and covers complex extreme scenes; and the reward function is oriented to memory leakage and guides the optimization strategy of the agent. The method realizes comprehensive and accurate testing of the memory leakage of the communication module, can effectively cover the blind area of the memory leakage of deep layers and time sequence correlation, improves the test efficiency and the utilization rate of hardware resources, is adaptive to version iteration and protocol update of the module, reduces the test and maintenance cost, and is favorable for guaranteeing long-term stable operation of the communication module.

Description

Technical Field

[0001] This application relates to the field of embedded software testing technology, and in particular to a method, system, device, medium and equipment for testing memory leaks in communication modules. Background Technology

[0002] As a core communication component of embedded terminal devices, the long-term stable operation of communication modules is crucial to system reliability. Memory leaks are a typical and persistent problem affecting the long-term performance of communication modules. Currently, industry testing for memory leaks in communication modules commonly employs methods such as manually designing test cases, executing automated scripts in fixed business scenarios, or random stress testing. These methods periodically monitor trends in metrics such as module memory usage and system resource consumption, and use preset thresholds to determine the presence of memory leaks. These methods typically cover common interface operations, routine business processes, and simple environmental change scenarios. They rely on testers' experience and understanding of module functionality to design test content and represent the mainstream solutions in the current field.

[0003] However, existing testing methods have the following limitations: First, they struggle to simulate the abnormal operating states of communication modules under harsh real-world network environments (such as frequent base station switching and momentary signal interruptions) and complex concurrent services (such as processing abnormal AT commands simultaneously during data transmission). This results in insufficient test coverage for deep, time-dependent memory leaks, creating significant test blind spots. Second, a large number of invalid operations exist in the massive random test cases, causing unnecessary consumption of the module's CPU, memory, and other hardware resources. This not only prolongs the testing cycle but also reduces hardware resource utilization, leading to low resource efficiency. Third, when the communication module's software version iterates or the communication protocol is updated, test cases need to be manually redesigned and verified. They cannot automatically adapt to new functionalities or hardware drivers, and the test strategies have poor adaptability to the real-time hardware status and operating environment of the communication module, resulting in maintenance lag. These problems make it difficult for existing testing methods to comprehensively, efficiently, and adaptively detect memory leaks in communication modules, which is detrimental to the long-term stable operation of communication modules. Summary of the Invention

[0004] This application provides a method, system, device, medium, and equipment for testing memory leaks in communication modules, aiming to solve the technical problem that existing methods for testing memory leaks in communication modules have poor adaptability to the real-time hardware status and operating environment of communication modules.

[0005] To address the aforementioned technical problems, this application adopts the following technical solution: A method for testing memory leaks in communication modules is provided. This method includes: constructing a reinforcement learning agent and configuring its corresponding state space, action space, and reward function; wherein the state space maps state vectors corresponding to multi-dimensional operational state data of the communication module in a physical execution environment; the action space maps multiple test action instructions that induce memory leaks in the communication module; the reinforcement learning agent observes the current state of the state space and selects and outputs a target test action instruction from the action space based on the current state; the reward function calculates an immediate reward based on the new state of the state space after executing the target test action instruction, and feeds the immediate reward back to the reinforcement learning agent for policy optimization; the immediate reward is positively correlated with the probability of memory leaks in the communication module; and the communication module is tested based on the target test action instruction output by the reinforcement learning agent.

[0006] In some embodiments, constructing a reinforcement learning agent and configuring the state space, action space, and reward function corresponding to the reinforcement learning agent includes: constructing the reinforcement learning agent based on a preset reinforcement learning algorithm; collecting multi-dimensional operating state data of the communication module in the physical execution environment in real time, and converting the multi-dimensional operating state data into a state vector to construct the state space; acquiring multiple test action instructions that induce memory leaks in the communication module, and converting the test action instructions into executable instructions to construct the action space; setting several preset reward items and several preset penalty items to construct the reward function; and associating the state space, the action space, and the reward function with the reinforcement learning agent.

[0007] In some embodiments, testing the communication module based on the target test action instruction output by the reinforcement learning agent includes: converting the target test action instruction output by the reinforcement learning agent into a hardware operation instruction; sending the hardware operation instruction to the hardware interface of the communication module in the physical execution environment to drive the communication module to execute the corresponding test action instruction and perform a status update.

[0008] In some embodiments, the multidimensional operational status data of the communication module includes at least one of memory-dimensional operational status data, system resource-dimensional operational status data, communication-dimensional operational status data, data flow-dimensional operational status data, and test context-dimensional operational status data.

[0009] In some embodiments, the test action instructions for inducing memory leaks in the communication module include at least one of atomic-level interface operations, scenario-level event simulations, and system-level stress operations.

[0010] In some embodiments, the reward function includes a memory consumption reward, an abnormal state combination reward, a problem discovery reward, and an inefficiency penalty.

[0011] To address the aforementioned technical problems, another technical solution adopted in this application is: providing a communication module memory leak testing system. This system includes: an intelligent decision-making and control module, comprising a reinforcement learning agent configured with a corresponding state space and action space, used to observe the current state of the corresponding state space, and select and output target test action instructions from the corresponding action space based on the current state; and a state perception and evaluation module, comprising a data acquisition unit, a state processor, and a reward calculator. The data acquisition unit is used to collect multi-dimensional operating state data of the communication module under physical execution environment, and the state processor is used to process the multi-dimensional operating state data... The row state data is converted into a state vector and mapped to the state space. The state vector corresponding to the new state is then sent to the reward calculator. The reward calculator is configured with a reward function corresponding to the reinforcement learning agent. It is used to calculate the immediate reward corresponding to the new state using the reward function and feeds the immediate reward back to the reinforcement learning agent so that the agent can optimize its policy based on the immediate reward. The test execution and interaction module is used to map multiple test action instructions that induce memory leaks in the communication module to the action space and test the communication module based on the target test action instructions output by the reinforcement learning agent.

[0012] To solve the above-mentioned technical problems, another technical solution adopted in this application is: to provide a communication module memory leak testing device, the device comprising: a test system hardware platform integrating the communication module memory leak testing system as described above; a unit under test, including a communication module, for connecting the communication module to the test system hardware platform and providing a physical execution environment to the communication module; and a peripheral simulation device, communicatively connected to the communication module, for simulating the communication peer of the communication module.

[0013] To solve the above-mentioned technical problems, another technical solution adopted in this application is: to provide a storage medium on which program data is stored, characterized in that the program data, when executed by a processor, implements the steps of the communication module memory leak test method described above.

[0014] To solve the above-mentioned technical problems, another technical solution adopted in this application is: to provide a computer device, which includes a processor and a memory connected to each other, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the steps of the communication module memory leak testing method described above.

[0015] The beneficial effects of this application are as follows: Unlike existing technologies, this application discloses a method, system, apparatus, medium, and device for testing memory leaks in communication modules. This application constructs a reinforcement learning agent and configures its corresponding state space, action space, and reward function to test the communication module based on the target test action instructions output by the reinforcement learning agent. Specifically, by collecting multi-dimensional operational status data of the communication module in the physical execution environment in real time and constructing a state space, the actual operating status of the communication module can be comprehensively and accurately reflected, providing richer and more realistic observation data for the reinforcement learning agent, significantly improving the comprehensiveness and accuracy of the test. By acquiring multiple test action instructions that induce memory leaks in the communication module and constructing an action space, the reinforcement learning agent is provided with a diverse range of test action instructions to choose from, enabling it to flexibly simulate various complex and extreme test scenarios. This effectively covers the blind spots of deep and time-dependent memory leaks that are difficult to reach by traditional testing, discovering deep memory leak paths. Furthermore, since the test action instructions are effective actions designed to induce memory leaks, and the selected target test action instructions are the optimal actions based on the current state, the repeated execution of invalid operations can be avoided, reducing unnecessary consumption of hardware resources, shortening the test cycle, and improving test efficiency and hardware resource utilization. By setting a reward function, a clear optimization goal is provided for the reinforcement learning agent, enabling it to continuously learn and improve the test strategy, thereby better discovering deep and time-dependent memory leak problems. This dynamic learning mechanism based on real-time interactive feedback enables reinforcement learning agents to adapt to software version iterations or protocol updates of the module. Without the need for manual redesign of test cases, they can automatically adjust target test action instructions to adapt to new functionalities or hardware drivers, reducing test maintenance costs and improving the real-time performance, compatibility, and adaptability of test strategies, thus providing a strong guarantee for the long-term stable operation of the communication module.

[0016] Therefore, by implementing this reinforcement learning-based communication module memory leak testing scheme, a comprehensive, efficient, adaptive, and continuously optimized testing process for communication module memory leaks has been achieved. The real-time interaction loop between the agent and the communication module state enables the testing process to automatically adapt to the real-time hardware state and operating environment of the communication module. This effectively solves the problems of poor adaptability, insufficient test coverage, low resource efficiency, and maintenance lag in existing testing methods, which are conducive to the long-term stable operation and iterative optimization of the communication module. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein: Figure 1 This is a flowchart illustrating an embodiment of the communication module memory leak testing method provided in this application; Figure 2 This is a schematic diagram of the interaction logic of a reinforcement learning framework; Figure 3 yes Figure 1 A flowchart illustrating an embodiment of a memory leak test for a communication module. Figure 4 This is a flowchart illustrating an embodiment of the communication module memory leak testing system provided in this application; Figure 5 This is a flowchart illustrating an embodiment of the communication module memory leak testing device provided in this application; Figure 6 This is a schematic diagram of the structure of an embodiment of the storage medium provided in this application; Figure 7 This is a schematic diagram of the structure of an embodiment of the computer device provided in this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0019] The terms "first," "second," and "third" used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0020] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0021] This application provides a method for testing memory leaks in communication modules. (See attached document.) Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the communication module memory leak testing method provided in this application. The communication module memory leak testing method includes: Step 10: Construct a reinforcement learning agent and configure its state space, action space, and reward function. The state space maps the state vectors corresponding to the multi-dimensional operational state data of the communication module in the physical execution environment. The action space maps multiple test action instructions that induce memory leaks in the communication module. The reinforcement learning agent observes the current state of the state space and selects and outputs target test action instructions from the action space based on the current state. The reward function calculates an immediate reward based on the new state of the state space after executing the target test action instruction and feeds the immediate reward back to the reinforcement learning agent for policy optimization. The immediate reward is positively correlated with the probability of memory leaks in the communication module.

[0022] In this embodiment, reinforcement learning (RL) is a machine learning method that uses an agent to interact with its environment and optimizes its strategy based on reward signals to achieve a goal. Its core lies in constructing a closed-loop feedback mechanism of "state-action-reward" to guide the agent to autonomously learn the optimal strategy. (See also...) Figure 2The reinforcement learning framework primarily consists of an agent and an environment, which interact through a closed-loop feedback mechanism involving state, action, and reward. The agent generates an action probability distribution based on its internal policy, selects action commands based on the observed environmental state, and sends these commands to the execution module within the environment. After executing one or more action commands, the environment transitions to a new state and provides a reward signal. Subsequently, the agent optimizes its policy based on the new state and the reward feedback, and selects new action commands to execute according to a specific policy. Through reinforcement learning, the agent learns which actions to take in which state to maximize its reward.

[0023] In this embodiment, the physical execution environment of the communication module is specifically used as the reinforcement learning environment. A reinforcement learning agent is constructed (e.g., using a preset reinforcement learning algorithm described later, an existing mature reinforcement learning framework, or a newly developed and customized reinforcement learning model based on reinforcement learning principles). Multi-dimensional operational state data of the communication module is configured to be mapped to the agent's state space. Test action instructions that induce memory leaks are configured to be mapped to the agent's action space. An immediate reward function is configured to calculate a reward positively correlated with the probability of memory leaks, optimizing the reinforcement learning agent's strategy. This achieves the construction of the reinforcement learning framework. Within this framework, the reinforcement learning agent establishes real-time dynamic interaction with the physical execution environment of the communication module, enabling the agent to autonomously perceive the state of the communication module, intelligently decide on test action instructions, and learn optimization strategies from the interaction to achieve adaptive testing.

[0024] In this embodiment, within the reinforcement learning framework, the multi-dimensional operational state data of the physical execution environment corresponding to the communication module is specifically used as the state observed by the agent. The physical execution environment of the communication module refers to the actual hardware testing environment containing the communication module under test, capable of simulating complex scenarios of actual operation and providing realistic interactive feedback. Simultaneously, this physical execution environment may also include network impairment devices configured to simulate harsh network environments (realizing scenarios such as network interruptions and switching), and programmable power supplies configured to simulate power fluctuations (realizing operations such as power jitter and cycling). Including these environmental devices as part of the physical execution environment and collecting their operational state data covers more dimensions of the communication module's operational state data. Subsequently, actions output by the agent will control these environmental devices to achieve simulation testing of more environmental conditions for the communication module.

[0025] In this embodiment, the communication module and its environmental devices can include multi-dimensional operational state data in the physical execution environment, such as memory dimension (physical memory, heap memory), system resource dimension (number of active threads, number of sockets), and communication dimension (network registration status, signal strength), etc. The operational state data to be mapped to the state space in this embodiment can be dynamically adjusted according to the test objectives, hardware resource limitations, and test efficiency requirements to balance the comprehensiveness and efficiency of the test. For example, in the early stage of the test, more dimensions of the state are included to make the observation more comprehensive, while in the later stage of the test, key dimensions are focused to make the test more efficient. These multi-dimensional operational state data will be mapped to the state space for the agent to observe and use as a basis for decision-making to achieve state perception. In addition, since the reinforcement learning model is trained based on vector input (that is, the corresponding vector space specifically represents the state in vector form), these operational state data need to be converted into vector form before being mapped to the state space to adapt to the input requirements of the reinforcement learning model. This application does not specifically limit the method of vector conversion, as long as it can accurately reflect the state characteristics and meet the model training requirements.

[0026] In this embodiment, within the reinforcement learning framework, test action instructions that induce memory leaks in the communication module are specifically used as the actions executed by the agent. These test action instructions are designed to trigger the risk of memory leaks in the communication module, such as atomic-level interface operations (e.g., sending abnormal commands), scenario-level event simulations (e.g., network interruptions), and system-level stress operations (e.g., concurrent high-load services). These test action instructions need to actively manipulate the operating environment and input of the communication module to stimulate potential memory leak defects and increase the probability of leak detection. These test action instructions can be dynamically adjusted according to module type, test objectives, and hardware capabilities to cover more potential leak scenarios and improve test targeting. These test action instructions will be incorporated into the action space of the reinforcement learning agent as a set of executable operations. At each decision step, the reinforcement learning agent will select the optimal action to execute as the target test action instruction based on the current state. This target test action instruction matches the agent's internal policy, which can maximize the expected reward and improve testing efficiency. Furthermore, through subsequent policy optimization, the selected actions will be more accurate, better able to discover deep, time-series related memory leaks, reduce invalid operations, and improve test targeting.

[0027] In this embodiment, within the reinforcement learning framework, the agent receives an immediate reward positively correlated with the memory leak probability. The reward function used to calculate this reward is a composite function designed with memory leak detection in mind. It can cover multiple reward and penalty items to guide the agent in optimizing its strategy and balancing exploration and exploitation. Furthermore, to effectively guide the detection of memory leaks, the reward function needs to comprehensively consider factors such as memory changes, abnormal states, and testing efficiency to calculate an immediate reward positively correlated with the memory leak probability. This encourages the agent to choose actions that trigger memory leaks, increasing the probability of leak detection. For example, a high reward can be given when memory usage increases significantly in a new state, an abnormal state combination occurs, or a system alarm is triggered, encouraging the agent to choose such actions. Conversely, a penalty can be given for new states with no memory changes or invalid operations, reducing the agent's choice of corresponding actions, avoiding repeated invalid operations, and optimizing the time and space efficiency of the testing process. Through the design of this reward function, the effectiveness of actions can be accurately reflected, guiding the agent to evolve towards maximizing the probability of memory leak detection, achieving adaptive optimization.

[0028] This embodiment, through the aforementioned construction and configuration steps, effectively builds a deeply coupled framework for reinforcement learning and communication module testing, forming a closed loop of state awareness, intelligent decision-making, and execution feedback. This addresses the testing blind spots, low resource efficiency, and maintenance lag issues of existing testing methods, achieving comprehensive coverage of communication module memory leaks. Within this framework, reinforcement learning agents can autonomously explore test scenarios, optimize test strategies, discover deep-seated temporal-related memory leak paths, and output optimal target test action instructions. This covers complex scenarios, discovers deep leaks, and improves testing efficiency. Subsequently, by executing the target test action instructions, new states can be collected and immediate rewards calculated, optimizing the reinforcement learning agent's strategy. This achieves comprehensive, efficient, and adaptive communication module memory leak testing, improving the long-term stability of the module and reducing testing and maintenance costs.

[0029] Optionally, in some embodiments, a reinforcement learning agent is constructed, and a corresponding state space, action space, and reward function are configured for the reinforcement learning agent. This includes: constructing a reinforcement learning agent based on a preset reinforcement learning algorithm; collecting multi-dimensional operational state data of the communication module in a physical execution environment in real time, and converting the multi-dimensional operational state data into state vectors to construct a state space; acquiring multiple test action instructions that induce memory leaks in the communication module, and converting the test action instructions into executable instructions to construct an action space; setting several preset reward items and several preset penalty items to construct a reward function; and associating the state space, action space, and reward function with the reinforcement learning agent.

[0030] This optional embodiment provides a specific implementation method for constructing a reinforcement learning agent and configuring its related space. Specifically, when constructing the reinforcement learning agent, the agent's policy network and value network are first initialized based on a preset reinforcement learning algorithm. This preset reinforcement learning algorithm is the core algorithm used to construct the agent. Specifically, algorithms such as PPO (Proximal Policy Optimization), DDPG (Deep Deterministic Policy Gradient), A2C (Advantage Actor-Critic), or A3C (Asynchronous Advantage Actor-Critic) can be used, as long as the algorithm can achieve the core function of optimizing the test policy based on interactive rewards. When constructing this agent using the PPO algorithm, the parameters of the policy network and value network, unique to the PPO algorithm, can be initialized first. Then, experience is collected through interaction with the environment. Experience replay and proximal policy optimization are used to update the network parameters, ultimately completing the construction of the agent and enabling it to make autonomous decisions. The PPO algorithm has advantages such as stable training, insensitivity to hyperparameters, and high sampling efficiency, allowing it to quickly learn effective policies within a limited number of interactions. Furthermore, during the agent construction process, optimization techniques such as experience replay mechanisms, policy gradient optimization, and reward function design can be combined to improve the agent's training stability and sampling efficiency, enabling the agent to converge quickly, learn efficiently, and output optimal test action instructions.

[0031] In this optional embodiment, the construction of the state space and action space is designed based on the aforementioned multi-dimensional operational state data and test action instructions that induce memory leaks. The state space, based on real-time collected multi-dimensional operational state data, is converted into state vectors and mapped to the agent's observation space, providing panoramic state perception and a basis for agent decision-making, covering the entire operation scenario of the communication module. The action space, based on test action instructions that induce memory leaks, is converted into executable instructions and mapped to the agent's action space, providing diverse test action instruction choices and a set of operations for agent decision-making, covering complex and extreme test scenarios. The reward function covers several preset reward items (such as functions that reward memory growth, abnormal state combinations, and problem discovery) and several preset penalty items (such as functions that penalize invalid operations, repetitive actions, and inefficient testing) to guide the agent in optimizing its strategy, balancing exploration and utilization, and feeding back immediate rewards to the agent for strategy updates, achieving adaptive testing.

[0032] In this optional embodiment, after constructing the state space, action space, and reward function, and completing the initialization of the reinforcement learning agent, these spaces and functions can be associated with the reinforcement learning agent. For example, the input, output, and feedback mechanisms of the agent can be associated through software programming, with the state space serving as the agent's observation input, the action space as the agent's output options, and the reward function as the agent's feedback mechanism. This enables the reinforcement learning agent to autonomously perceive the state, intelligently decide on actions, learn from interactions, optimize testing strategies, and achieve adaptive testing.

[0033] Optionally, in some embodiments, the multidimensional operational status data of the communication module includes at least one of memory-dimensional operational status data, system resource-dimensional operational status data, communication-dimensional operational status data, data flow-dimensional operational status data, and test context-dimensional operational status data.

[0034] In this optional embodiment, the composition of the multi-dimensional operational status data of the communication module and the core role of each dimension are further specified. Among them, the memory dimension operational status data refers to the usage of various memory resources during the operation of the communication module, used to characterize the dynamic changes in module memory allocation and release. Specifically, it can include at least one of the following memory resource indicators: physical memory (reflecting the overall memory consumption level of the module), heap memory (reflecting the health status of dynamic memory allocation), and protocol stack memory (reflecting the private memory occupancy of the communication protocol processing module). Since changes in the memory dimension are directly related to the risk of memory leaks, it is usually the core observation object of memory leaks. These indicators have high sensitivity and can directly reflect the occurrence and development of leaks. Therefore, by incorporating this memory dimension operational status data into the state space, signs of memory leaks can be accurately captured, providing a core basis for intelligent agent decision-making.

[0035] In this optional embodiment, the system resource dimension operation status data refers to the resource usage at the kernel level of the communication module's operating system, which reflects the load and health of the module system. Specifically, it may include at least one of the system resource indicators maintained by the module's operating system kernel, such as the number of active threads (reflecting the number of tasks executed concurrently by the system), the number of sockets (reflecting the activity level of network connections), and the number of file handles (reflecting the resource usage of file operations). Since abnormal system resource usage is often accompanied by memory leaks and is an indirect manifestation of memory leaks, incorporating this dimension into the state space can help to judge the risk of memory leaks at the system level and improve the comprehensiveness of state awareness.

[0036] In this optional embodiment, the communication dimension operation status data refers to the status information of the communication module's interaction with the external network, which is used to reflect the module's communication environment and connection quality. Specifically, it may include at least one of the following communication status indicators: network registration status (reflecting whether the module is normally connected to the network) read from the module's baseband chip through AT command interaction, real-time collected signal strength (reflecting the quality of the communication link), currently used cellular network frequency band (reflecting the characteristics of the network environment), and network standard (reflecting the type of communication protocol). Since a poor communication environment can easily trigger the module's abnormal processing flow and increase the probability of memory leaks, this dimension is included in the state space, which allows the intelligent agent to perceive the impact of the communication environment on memory leaks and adjust the test action commands accordingly.

[0037] In this optional embodiment, the data flow dimension running status data refers to the dynamic state during the data transmission process of the communication module, which is used to reflect the load and efficiency of the module's data processing. Specifically, it may include at least one of the following data flow indicators: queue length of the hardware buffer of the module's serial port / Universal Serial Bus (USB) (reflecting the degree of congestion in data transmission), uplink data flow (reflecting the load of the module sending data), and downlink data flow (reflecting the load of the module receiving data). Since drastic changes in data flow can easily cause memory management anomalies and are a potential cause of memory leaks, incorporating this dimension into the state space can capture the impact of data flow changes on memory and improve the targeting of testing.

[0038] In this optional embodiment, the test context dimension running state data refers to the action sequence and execution results in the historical test process, which is used to reflect the historical trajectory and experience of the test. Specifically, it may include at least one of the test context information such as the historical test action instruction sequence (reflecting the previously executed test operation) and the execution result of the historical action (reflecting the impact of the action on the module state). This dimension can help the agent avoid repeating invalid operations, optimize decision-making by utilizing historical experience, and improve test efficiency.

[0039] This optional embodiment constructs a panoramic state awareness system for communication module memory leak testing through the collaborative integration of operational state data from multiple dimensions. This system features multi-dimensional collaboration, real-time synchronization, and digital twin-driven capabilities. It achieves full-level state mapping from hardware to software, and from real-time state to historical experience, enabling reinforcement learning agents to accurately identify complex abnormal scenarios involving multiple coupled factors (such as a combination of communication interruption, high load, and memory growth), thus overcoming the limitations of traditional single-indicator testing. Simultaneously, the panoramic state awareness provides rich feedback for the agent's strategy updates, driving the system to autonomously explore time-series-related deep leak paths (such as protocol stack memory leaks caused by consecutive abnormal operations), significantly improving test coverage depth and efficiency. Furthermore, the system supports dynamic addition, deletion, and adjustment of dimensions, allowing the testing system to adapt to software version iterations or hardware changes in the communication module. This effectively solves the problem of lagging maintenance in existing technologies and provides core technical support for intelligent and adaptive testing of communication module memory leaks.

[0040] Optionally, in some embodiments, the test action instructions that induce memory leaks in the communication module include at least one of atomic-level interface operations, scenario-level event simulations, and system-level stress operations.

[0041] In this optional embodiment, the types of test action instructions for inducing memory leaks in the communication module and the core functions of each type are further specified. Among them, atomic interface operations are basic operations that directly act on the hardware interface or protocol layer of the communication module. They are test action instruction types used to trigger anomalies in the module's underlying processing logic. Specifically, they can include at least one of the following test action instructions: sending abnormal formatting commands, AT commands with excessively long parameters (used to test the module's fault tolerance capability for illegal input and memory release), quickly resetting the communication interface (used to test whether resource release is complete when the interface restarts), injecting erroneous protocol data units (used to test whether the protocol stack's handling of erroneous data leads to memory leaks), and quickly switching the communication interface (used to test resource management defects during interface switching). These test action instructions can directly expose memory management defects in the module's underlying code, incorporate them into the action space, and enable the intelligent agent to autonomously explore the module's memory behavior under extreme interface operations, providing a basis for discovering deep leaks.

[0042] In this optional embodiment, scenario-level event simulation is a test action instruction type that simulates abnormal events in real application scenarios to trigger the abnormal handling process of the module in complex environments. Specifically, it may include at least one of the following test action instructions: simulating cellular network switching, instantaneous signal fading and interruption through a network impairment tester (used to test memory leaks when the network fluctuates), simulating unexpected disconnection of cloud platform connection (used to test whether resource release is complete when the connection is abnormal), and simulating module power jitter and cycling through a programmable power supply (used to test memory management defects when the power supply is unstable). These test action instructions can reproduce extreme situations in real scenarios, induce the module to enter a potential leakage state, and improve the authenticity and effectiveness of the test.

[0043] In this optional embodiment, system-level stress operation is a test action instruction type that applies a high load to the module system level to trigger memory leaks in the module when resources are scarce. Specifically, it may include at least one of the test action instructions such as concurrently triggering multiple high-load services of the module (such as simultaneously performing data transmission, positioning, and voice calls to test memory management under system resource contention) and triggering the firmware upgrade process (to test memory leaks during the upgrade process). These test action instructions can simulate the state of the module running under high load and expose system-level memory leak problems.

[0044] This optional embodiment constructs a composite action space for inducing memory leaks in communication modules through the combined design of multiple types of test action instructions. This action space covers multi-dimensional test scenarios from the underlying interface to the system level, enabling reinforcement learning agents to autonomously combine diverse test sequences, effectively stimulating potential memory leak defects in the module, improving the effectiveness and comprehensiveness of testing, and providing a rich selection of actions for intelligent testing of communication module memory leaks.

[0045] Optionally, in some embodiments, the reward function includes a memory consumption reward, an abnormal state combination reward, a problem discovery reward, and an inefficiency penalty.

[0046] In this optional embodiment, the composition of the reward function and the core role of each reward item are further specified. Through the collaborative design of multi-dimensional rewards and penalties, a closed-loop feedback mechanism is formed to guide the reinforcement learning agent to optimize the testing strategy. Among them, the memory consumption reward item (denoted as R) memory This refers to the core reward item used to guide the agent to prioritize test action instructions that may trigger memory growth. It establishes a direct mapping relationship between memory leak risk and reward, causing the agent to tend to choose test action instructions that increase memory consumption, thus increasing the probability of detecting memory leaks. It directly correlates the causal relationship between test action instructions and memory changes, quantifies the dynamic changes in memory consumption, and provides the agent with a clear optimization direction. For example, it can be calculated using formula R...memory =λ×(Memory t+1 Memory t ) Calculate the memory consumption reward, where Memory t This refers to the physical memory usage of the communication module at time t. t+1 This refers to the physical memory usage of the communication module at time t+1, where λ is an adjustable weighting coefficient that can be set based on offline simulation experiments, grid search, and testing experience. This formula directly reflects the impact of test action commands on memory consumption. A positive reward is given when memory increases, and a zero or negative reward is given when memory does not increase or decreases. By introducing this memory consumption reward, the agent's decision-making can be focused on the core leakage characteristic of memory growth, providing direct guidance for discovering deep memory leaks, avoiding ineffective low-memory-consumption actions, and improving the targeting of tests.

[0047] In this optional embodiment, the abnormal state combination reward item (denoted as R) novelty This refers to a reward system that encourages agents to explore uncovered complex and abnormal scenarios. It uses state clustering algorithms or hash tables to record existing state combinations. When an agent's test action command drives the communication module into an unprecedented abnormal state combination (such as a combination of communication interruption + high load + memory growth), a high reward (e.g., a fixed R) is given. novelty =+50), which stimulates the intelligent agent's desire to explore, autonomously discovers more potential memory leak triggering conditions, effectively breaks through the blind spots of traditional testing, and covers complex abnormal scenarios with deep and time-series correlations.

[0048] In this optional embodiment, the problem discovery reward item (denoted as R) anomaly This refers to the core reward for directly discovering serious memory leaks. It can be determined by monitoring the system logs or hardware status of the communication module. A substantial reward (e.g., fixed at R) is given when test commands cause observable hardware-level faults such as kernel alarms, watchdog resets, or system crashes. anomaly =+100), which enables rapid location of critical memory leak paths, significantly improves the effectiveness of testing, guides the agent's strategy to discover high-value leak problems, and shortens the problem localization cycle.

[0049] In this optional embodiment, the inefficiency penalty term (denoted as R) penalty This refers to a negative stimulus term that optimizes the time and space efficiency of the testing process. It can be determined by judging the memory changes or state updates after the action is executed. When there is no memory change after the action is executed (Memory... t+1 Memory t =0) or when the same action is performed repeatedly, a penalty (such as being fixed at R) is imposed.penalty =-0.1), thereby avoiding the agent from repeatedly executing invalid actions, reducing unnecessary consumption of hardware resources, shortening the test cycle, and improving the utilization rate of test resources.

[0050] This optional embodiment constructs a composite reward function centered on memory leaks through the aforementioned multi-dimensional reward and punishment synergistic design. Based on the above, it can be implemented using formula R... total = a×R memory + b×R novelty + c×R anomaly + d×R penalty Calculate the total reward R total Where a, b, c, and d are adjustable weighting coefficients, which can be set according to the test objectives, module characteristics, and experimental experience. Typically, |R| is set to... anomaly |>|R memory Single-step change |>|R penalty This mechanism ensures that the discovery of serious memory leaks takes precedence over the observation of single-step memory growth, while balancing the optimal use of exploration and testing resources through appropriate efficiency penalties. Exploration and efficiency are considered long-term optimization goals. This multi-layered, differentiated reward and penalty mechanism effectively guides the reinforcement learning agent to achieve a dynamic balance between exploring new anomaly scenarios, focusing on core memory leak characteristics, and improving testing efficiency. Ultimately, it aims to autonomously discover deep, time-dependent memory leak paths that are difficult to cover with traditional testing, providing precise feedback guidance for intelligent testing of communication module memory leaks.

[0051] Step 20: Test the communication module based on the target test action instructions output by the reinforcement learning agent.

[0052] In this embodiment, based on the aforementioned reinforcement learning agent, the communication module can be tested using the target test action instructions output by the agent. This triggers state changes in the communication module and its operating environment, providing input for subsequent state acquisition and reward calculation. Since the target test action instructions are the optimal actions decided by the reinforcement learning agent based on its internal policy and the current state of the communication module, the actions can maximize the stimulation of potential memory leaks in the communication module, effectively covering complex scenarios that are difficult to reach with traditional testing. In the testing process based on target test action instructions, the target test action instructions can be converted into hardware operation instructions through a software logic module (such as an action mapper). The hardware operation instructions are then obtained and executed by the test execution engine to test the communication module and the test environment. Alternatively, network interruptions can be controlled via SNMP (Simple Network Management Protocol) or dedicated APIs (Application Programming Interfaces) of the network impairment instrument, power jitter can be simulated via SCPI (Standard Commands for Programmable Instruments) of the programmable power supply, or abnormal AT commands can be sent via the pySerial library. These hardware operation instructions drive the physical device to perform corresponding actions, triggering state changes in the communication module. Through this test execution based on target test action instructions, the module's state feedback in a real-world scenario can be obtained, effectively identifying potential memory leak defects and providing data support for subsequent reward calculations and strategy optimization.

[0053] In this embodiment, the execution of the test action command can refer to the execution of the target test command output during the iterative policy optimization process of the reinforcement learning agent, or it can refer to the test verification based on the converged optimal policy after the reinforcement learning agent completes the iteration. See also... Figure 3 , Figure 3This diagram illustrates the process of testing memory leaks in a communication module within the aforementioned reinforcement learning architecture. During the test, the test environment and reinforcement learning agent are first initialized (e.g., connecting hardware devices, initializing agent parameters to construct the agent, and configuring the corresponding state space, action space, and reward function). Based on this, the reinforcement learning agent observes the current state St in the state space and selects action At to execute according to policy π. The state processor observes the new state S_{t+1}, and the reward function calculates the immediate reward Rt based on the state transition. The experience replay buffer of the reinforcement learning agent can store the experiences St, At, Rt, and S_{t+1} for subsequent policy updates. Then, it determines whether the reinforcement learning agent has met the learning conditions (e.g., in the PPO algorithm, it checks if the experience buffer has accumulated a sufficient amount of experience, while in other algorithms, it might check if a preset number of interaction steps has been completed). If the learning conditions are met, policy optimization can be performed, and the stored experience can be sampled. The policy is updated through the policy network and value network of the reinforcement learning agent, thereby improving the agent's decision-making ability. However, if... If the learning conditions are not met or the strategy update is completed, the system will determine whether the current test round meets the termination conditions (e.g., terminating the current round upon memory overrun, system crash, or at the end of a preset number of interaction steps to prevent continuous abnormal module operation). If the termination conditions are not met, the test needs to continue, and the next round of interaction can be started. Here, the current state St in the state space is observed to execute a new action At. If the termination conditions are met, the current round ends, and the test environment can be reset (e.g., restarting the communication module, restoring network connection, resetting power parameters). After resetting, the system will determine whether the current test round meets the termination conditions. If the required number of test rounds has been reached (this number can be set based on test objectives, module characteristics, and hardware resources to ensure sufficient scenario coverage), it indicates that more rounds of testing are needed. The observation of the new state St can be restarted, and action At can be executed, repeating the above process. If the required number of test rounds has been reached, the test is complete. The action sequence with the highest cumulative reward can be output as the optimal test action instruction sequence, along with an analysis report. This allows developers to locate and fix memory leaks, assess module stability, and end the test. In this exemplary test process, step 20's testing of the communication module can specifically refer to the execution of test action instruction At, or it can refer to the verification test based on the optimal test sequence after outputting the optimal test sequence, further reproducing and confirming the triggering conditions and reproduction path of memory leaks, providing developers with precise defect localization information.

[0054] This embodiment transforms the agent's decisions into actual test operations, realizing the execution phase in the reinforcement learning closed loop. This ensures that test action commands can truly affect the communication module and its operating environment, obtaining the module's real state feedback after action execution. This provides necessary data support for the agent's strategy optimization, effectively covering complex scenarios that are difficult to reach with traditional testing, improving the comprehensiveness and efficiency of memory leak testing. Through continuous iterative strategy optimization, it can autonomously discover deep, time-dependent memory leak paths, providing an intelligent solution for the stability testing of communication modules.

[0055] Optionally, in some embodiments, testing the communication module based on the target test action instructions output by the reinforcement learning agent includes: converting the target test action instructions output by the reinforcement learning agent into hardware operation instructions; sending the hardware operation instructions to the hardware interface of the communication module in the physical execution environment to drive the communication module to execute the corresponding test action instructions and perform state updates.

[0056] In this optional embodiment, the transformation process from abstract decision-making to physical execution of target test action commands is further clarified, ensuring that the agent's decisions can be accurately implemented into operations on the communication module and its environment. In this process, various development toolkits, automated testing frameworks, and data processing libraries are first used to convert the abstract target test action commands output by the reinforcement learning agent into specific hardware operation commands, thus realizing the transformation from abstract decision-making to physical operation (for example, using Python programming, the pySerial library can generate AT command frames with abnormal formats, the SNMP library can call the network impairment instrument's interface to generate link interruption commands, the SCPI protocol can generate jitter commands for the programmable power supply, and data processing libraries such as NumPy and Pandas can normalize and extract features from the state data to convert multi-dimensional operating state data into state vectors). Then, the hardware operation commands are sent to the hardware interfaces of the communication module or environmental devices in the physical execution environment (such as the serial physical interface for sending AT commands, the network control interface for controlling the network impairment instrument and the programmable power supply), driving the communication module and environmental devices to execute corresponding actions and triggering state updates of the communication module. This process effectively achieves deep coupling between the reinforcement learning framework and the communication module hardware, ensuring accurate implementation of the agent's decisions, obtaining real state feedback, providing reliable input for subsequent state perception and policy optimization, and improving the effectiveness and comprehensiveness of the test.

[0057] This embodiment uses a laboratory test bench as an example. In the setup and operation of this test bench, the hardware connection deployment is completed first. The communication module under test is connected to the main control PC via a serial port or universal serial bus interface. Its power supply line is connected to a programmable power supply to simulate power fluctuation scenarios, and the network access path is connected in series with a network impairment instrument to simulate a complex network environment. Next, software implementation is performed: a PPO reinforcement learning agent is built on the main control PC using Python integrated with the TensorFlow / PyTorch framework. The pySerial library controls the serial port to send abnormal AT commands, and SNMP or a dedicated API is used to control the network impairment instrument and the programmable power supply to perform scenario simulation actions. Subsequently, status monitoring is started. Through the module debugging interface or the / proc file system of the embedded Linux system, multi-dimensional operational data such as physical memory, heap memory, number of active threads, and network registration status are collected in real time and preprocessed into state vectors, thus constructing a reinforcement learning framework. Under this framework, effect verification can be carried out. The reinforcement learning agent will autonomously decide on test action commands based on the state vectors, and drive the physical device to execute and provide feedback on new states and rewards based on these commands, continuously iterating and optimizing the strategy. This test bench deeply integrates communication module hardware with a reinforcement learning framework. Through a closed loop of "state awareness - intelligent decision-making - execution feedback," it autonomously discovers deep memory leak paths that are difficult to reproduce using traditional testing methods, significantly improving testing efficiency and coverage depth. It also possesses adaptive iteration capabilities, allowing it to adjust its strategies automatically with module software version updates, reducing maintenance costs. In a real-world test, a PDU (Protocol Data Unit) parsing buffer leak issue in a certain 4G module failed to be reproduced in two weeks using traditional testing, while this system accurately triggered and located the problem within 48 hours, verifying the efficiency and reliability of the aforementioned method.

[0058] See Figure 4 , Figure 4This is a schematic diagram of an embodiment of the communication module memory leak testing system provided in this application. The communication module memory leak testing system 30 includes: an intelligent decision-making and control module 31, including a reinforcement learning agent, which is configured with a corresponding state space and action space, for observing the current state of the corresponding state space, and selecting and outputting a target test action instruction from the corresponding action space based on the current state; a state perception and evaluation module 32, including a data acquisition unit, a state processor, and a reward calculator, wherein the data acquisition unit is used to collect multi-dimensional operating state data of the communication module in the physical execution environment, the state processor is used to convert the multi-dimensional operating state data into a state vector and map it to the state space, and send the state vector corresponding to the new state to the reward calculator, the reward calculator is configured with a reward function corresponding to the reinforcement learning agent, for calculating the immediate reward corresponding to the new state using the reward function, and feeding the immediate reward back to the reinforcement learning agent so that the reinforcement learning agent can optimize the strategy based on the immediate reward; and a test execution and interaction module 33, which is used to map multiple test action instructions that induce memory leaks in the communication module to the action space, and test the communication module based on the target test action instructions output by the reinforcement learning agent.

[0059] Optionally, in some embodiments, the intelligent decision-making and control module 31 further includes: a policy network: used to output the corresponding target test action instruction At based on the current state St; a value network: used to evaluate the long-term value of the current state St and predict the total reward that can be obtained in the future; and an experience replay buffer: used to store experience tuples generated by interactions and randomly sample a batch of experiences during training for batch training of the agent; wherein, the experience tuple includes the current state St, the target test action instruction At, the immediate reward Rt, and the new state S_{t+1}. In this optional embodiment, the policy network, the value network, and the experience replay buffer are the core components of the agent in the reinforcement learning framework. The policy network and the value network share state features. The evaluation result of the value network serves as a benchmark to guide the update direction of the policy network. The experience replay buffer provides stable and efficient data support for the training of both, enabling the agent to quickly converge and learn effective test policies.

[0060] Optionally, in some embodiments, the test execution and interaction module 33 further includes: an action mapper, used to convert the target test action instructions output by the reinforcement learning agent into hardware operation instructions; and a test execution engine, used to send the hardware operation instructions to the hardware interface of the communication module in the physical execution environment, so as to drive the communication module to execute the corresponding test action instructions and perform state updates.

[0061] Optionally, in some embodiments, the physical execution environment of the communication module further includes a network impairment instrument and a programmable power supply; the test execution and interaction module 33 further includes: a serial port / USB command control unit, connected to the communication module, used to send AT commands with abnormal format and excessively long parameters to test the module's fault tolerance capability for illegal input and memory release; a network impairment instrument control unit, connected to the network impairment instrument, used to adjust network parameters, simulate cellular network switching, instantaneous signal fading and interruption, and test memory leaks during network fluctuations; and a programmable power supply control unit, connected to the programmable power supply, used to simulate module power jitter and cycling, and test memory management defects when the power supply is unstable. In this optional embodiment, a network impairment instrument and a programmable power supply are specifically introduced, as well as their corresponding network impairment instrument control unit and programmable power supply control unit. Their functions and implementation methods can be found in the scenario-level event simulation section described above, and will not be repeated here.

[0062] Optionally, in some embodiments, the reinforcement learning agent is constructed based on a preset reinforcement learning algorithm; the state space corresponding to the reinforcement learning agent is constructed based on real-time acquisition of multi-dimensional operating state data of the communication module in the physical execution environment, and the multi-dimensional operating state data is converted into a state vector; the action space corresponding to the reinforcement learning agent is constructed based on acquiring multiple test action instructions that induce memory leaks in the communication module, and the test action instructions are converted into executable instructions; the reward function corresponding to the reinforcement learning agent is constructed based on setting several preset reward items and several preset penalty items; the state space, action space, and reward function are associated with the reinforcement learning agent.

[0063] Optionally, in some embodiments, the multidimensional operational status data of the communication module includes at least one of memory-dimensional operational status data, system resource-dimensional operational status data, communication-dimensional operational status data, data flow-dimensional operational status data, and test context-dimensional operational status data.

[0064] Optionally, in some embodiments, the test action instructions that induce memory leaks in the communication module include at least one of atomic-level interface operations, scenario-level event simulations, and system-level stress operations.

[0065] Optionally, in some embodiments, the reward function includes a memory consumption reward, an abnormal state combination reward, a problem discovery reward, and an inefficiency penalty.

[0066] Since the embodiments of system 30 correspond to the embodiments of the above methods, please refer to the above method embodiments for the introduction of the communication module memory leak testing system 30 provided by the embodiments of the present invention. The embodiments of the present invention will not be described again here, and it has the same beneficial effects as the above communication module memory leak testing method.

[0067] See Figure 5 , Figure 5 This is a schematic diagram of an embodiment of the communication module memory leak testing device provided in this application. The communication module memory leak testing device 40 includes: a test system hardware platform 41, integrating the communication module memory leak testing system 30 as described above; a unit under test 42, including a communication module, used to connect the communication module to the test system hardware platform and provide a physical execution environment to the communication module; and a peripheral simulation device 43, communicatively connected to the communication module, used to simulate the communication peer of the communication module.

[0068] Optionally, in some embodiments, the test system hardware platform 41 further includes: a programmable power supply simulation unit for simulating abnormal power environment conditions; these abnormal conditions include, but are not limited to, power jitter, cyclic power outages, and voltage fluctuations; a network impairment tester simulation unit for simulating abnormal network environment conditions; these abnormal conditions include, but are not limited to, cellular network switching, instantaneous signal fading, link interruption, and bandwidth limitation; the module memory leak test system 20 controls the programmable power supply simulation unit and the network impairment tester simulation unit via Ethernet; the module memory leak test system 20 controls the communication module via a serial port or universal serial bus; the programmable power supply output of the programmable power supply simulation unit is connected to the communication module; and the network impairment tester simulation unit is connected to the communication module via a network link. This optional embodiment specifies the core simulation unit of the test system hardware platform and clarifies the connection method and function of each unit with the module memory leak test system and communication module. By controlling the scenario simulation unit via Ethernet and the communication module via serial port / USB, it realizes the hardware support of the "state perception-intelligent decision-execution feedback" closed loop. The hardware support of the "state perception-intelligent decision-execution feedback" closed loop can accurately reproduce the extreme environmental conditions in real applications, provide rich scenario exploration dimensions for reinforcement learning agents, and at the same time ensure the real-time and reliability of test command transmission, ensuring that the test action commands are accurately applied to the communication module under test to stimulate potential leakage defects.

[0069] Optionally, in some embodiments, the peripheral simulation device 43 includes: a server simulation unit, which is connected to the communication module via data connection, for simulating a data server; and a base station simulation unit, which is connected to the communication module via cellular network connection, for simulating a cellular network base station. This optional embodiment provides a simulation scheme for communication scenarios, which can effectively construct a typical interactive environment of the communication module in real applications, supporting the module to complete core business processes such as data transmission and reception, network registration, and network switching. At the same time, it can cooperate with the environment simulation unit of the test system hardware platform to trigger extreme working conditions under real business interaction backgrounds, helping reinforcement learning agents to explore memory leak problems in business-environment coupling scenarios, and improving the realism and comprehensiveness of the test scenarios.

[0070] Since the embodiments of the device 40 correspond to the embodiments of the above-described methods and systems, the description of the communication module memory leak testing device 40 provided in the embodiments of the present invention should refer to the above-described methods or systems embodiments. The embodiments of the present invention will not be described again here, as they have the same beneficial effects as the above-described communication module memory leak testing method and communication module memory leak testing system 30.

[0071] See Figure 6 , Figure 6 This is a schematic diagram of an embodiment of the storage medium provided in this application.

[0072] The storage medium 50 stores program data 51, which, when executed by the processor, implements the communication module memory leak test method described above.

[0073] The program data 51 is stored in a storage medium 50 and includes several instructions for causing a network device (such as a router, personal computer, or server) or processor to execute all or part of the steps of the methods described in the various embodiments of this application.

[0074] Optionally, the storage medium 50 can be any medium capable of storing program data 51, such as a USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), disk, or optical disc.

[0075] See Figure 7 , Figure 7 This is a schematic diagram of the structure of an embodiment of the computer device provided in this application.

[0076] The computer device 60 includes a processor 62 and a memory 61 interconnected. The memory 61 stores a computer program, and when the processor 62 executes the computer program, it implements the communication module memory leak testing method described above. The memory 61 may include a storage medium 50, or it may be another separately developed memory.

[0077] Unlike existing technologies, this application discloses a method, system, apparatus, medium, and device for testing memory leaks in communication modules. By constructing a reinforcement learning agent and configuring its corresponding state space, action space, and reward function, the communication module is tested based on the target test action instructions output by the reinforcement learning agent. Specifically, by collecting multi-dimensional operational status data of the communication module in the physical execution environment in real time and constructing a state space, the actual operating status of the communication module can be comprehensively and accurately reflected, providing richer and more realistic observation data for the reinforcement learning agent, significantly improving the comprehensiveness and accuracy of the test. By acquiring multiple test action instructions that induce memory leaks in the communication module and constructing an action space, the reinforcement learning agent is provided with a diverse range of test action instructions to choose from, enabling it to flexibly simulate various complex and extreme test scenarios. This effectively covers the blind spots of deep and time-dependent memory leaks that are difficult to reach by traditional testing, discovering deep memory leak paths. Furthermore, since the test action instructions are effective actions designed to induce memory leaks, and the selected target test action instructions are the optimal actions based on the current state, the repeated execution of invalid operations can be avoided, reducing unnecessary consumption of hardware resources, shortening the test cycle, and improving test efficiency and hardware resource utilization. By setting a reward function, a clear optimization goal is provided for the reinforcement learning agent, enabling it to continuously learn and improve the test strategy, thereby better discovering deep and time-dependent memory leak problems. This dynamic learning mechanism based on real-time interactive feedback enables reinforcement learning agents to adapt to software version iterations or protocol updates of the module. Without the need for manual redesign of test cases, they can automatically adjust target test action instructions to adapt to new functionalities or hardware drivers, reducing test maintenance costs and improving the real-time performance, compatibility, and adaptability of test strategies, thus providing a strong guarantee for the long-term stable operation of the communication module.

[0078] Therefore, by implementing this reinforcement learning-based communication module memory leak testing scheme, a comprehensive, efficient, adaptive, and continuously optimized testing process for communication module memory leaks has been achieved. The real-time interaction loop between the agent and the communication module state enables the testing process to automatically adapt to the real-time hardware state and operating environment of the communication module. This effectively solves the problems of poor adaptability, insufficient test coverage, low resource efficiency, and maintenance lag in existing testing methods, which are conducive to the long-term stable operation and iterative optimization of the communication module.

[0079] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments, device embodiments, storage medium embodiments, and computer device embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0080] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for testing memory leaks in a communication module, characterized in that, include: A reinforcement learning agent is constructed, and its corresponding state space, action space, and reward function are configured. The state space maps the state vectors corresponding to the multi-dimensional operational state data of the communication module in the physical execution environment. The action space maps multiple test action instructions that induce memory leaks in the communication module. The reinforcement learning agent observes the current state of the state space and selects and outputs a target test action instruction from the action space based on the current state. The reward function calculates an immediate reward based on the new state of the state space after executing the target test action instruction and feeds the immediate reward back to the reinforcement learning agent for policy optimization. The immediate reward is positively correlated with the probability of memory leaks in the communication module. The communication module is tested based on the target test action instructions output by the reinforcement learning agent.

2. The method for testing memory leaks in a communication module according to claim 1, characterized in that, The construction of the reinforcement learning agent and the configuration of the state space, action space, and reward function corresponding to the reinforcement learning agent include: The reinforcement learning agent is constructed based on a preset reinforcement learning algorithm; The multi-dimensional operating status data of the communication module in the physical execution environment is collected in real time, and the multi-dimensional operating status data is converted into a state vector to construct the state space; The test action instructions for inducing memory leaks in multiple communication modules are obtained, and the test action instructions are converted into executable instructions to construct the action space; Several preset reward items and several preset penalty items are set to construct the reward function; The state space, the action space, and the reward function are associated with the reinforcement learning agent.

3. The method for testing memory leaks in a communication module according to claim 1, characterized in that, The testing of the communication module based on the target test action command output by the reinforcement learning agent includes: The target test action instructions output by the reinforcement learning agent are converted into hardware operation instructions; The hardware operation instructions are sent to the hardware interface of the communication module in the physical execution environment to drive the communication module to execute the corresponding test action instructions and update the status.

4. The method for testing memory leaks in a communication module according to any one of claims 1-3, characterized in that, The multidimensional operational status data of the communication module includes at least one of the following: memory-dimensional operational status data, system resource-dimensional operational status data, communication-dimensional operational status data, data flow-dimensional operational status data, and test context-dimensional operational status data.

5. The method for testing memory leaks in a communication module according to any one of claims 1-3, characterized in that, The test action instructions for inducing memory leaks in the communication module include at least one of atomic-level interface operations, scenario-level event simulations, and system-level stress operations.

6. The method for testing memory leaks in a communication module according to any one of claims 1-3, characterized in that, The reward function includes a memory consumption reward, an abnormal state combination reward, a problem discovery reward, and an inefficiency penalty.

7. A memory leak testing system for a communication module, characterized in that, include: The intelligent decision-making and control module includes a reinforcement learning agent, which is configured with a corresponding state space and action space, for observing the current state of the corresponding state space, and selecting a target test action instruction from the corresponding action space based on the current state and outputting it. The state perception and evaluation module includes a data acquisition unit, a state processor, and a reward calculator. The data acquisition unit is used to collect multi-dimensional operating state data of the communication module in the physical execution environment. The state processor is used to convert the multi-dimensional operating state data into a state vector and map it to the state space, and send the state vector corresponding to the new state to the reward calculator. The reward calculator is configured with a reward function corresponding to the reinforcement learning agent, and is used to calculate the immediate reward corresponding to the new state using the reward function, and feed the immediate reward back to the reinforcement learning agent so that the reinforcement learning agent can optimize the policy based on the immediate reward. The test execution and interaction module is used to map multiple test action instructions that induce memory leaks in the communication module to the action space, and to test the communication module based on the target test action instructions output by the reinforcement learning agent.

8. A memory leak testing device for a communication module, characterized in that, include: The test system hardware platform integrates the communication module memory leak test system as described in claim 7; The unit under test includes a communication module, which is used to connect the communication module to the test system hardware platform and provide a physical execution environment to the communication module. An external simulation device is connected to the communication module to simulate the communication peer of the communication module.

9. A storage medium storing program data thereon, characterized in that, When the program data is executed by the processor, it implements the steps of the communication module memory leak test method as described in any one of claims 1 to 6.

10. A computer device, characterized in that, The device includes an interconnected processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the steps of the communication module memory leak testing method as described in any one of claims 1 to 6.

Images