Software refactoring method based on deep reinforcement learning

Abstract

The application provides a software reconstruction method based on deep reinforcement learning, which monitors the software running process, analyzes the software running live from two aspects of the software itself and the hardware device where the software is located, judges whether each algorithm module of the algorithm architecture platform of the software is in a normal running state or not, and performs deep learning training and reconstruction on the algorithm of the corresponding algorithm module, so as to reduce the inspection workload of the software, and improve the reliability and accuracy of the overall correction and reconstruction of the software.

Description

Technical Field

[0001] This invention relates to the field of information processing technology, and in particular to a software reconstruction method based on deep reinforcement learning. Background Technology

[0002] Software operation relies on the collaborative work of algorithm modules within its internal algorithm architecture platform, each containing different algorithms. Existing software typically integrates and encapsulates these different algorithm modules. While this ensures overall software stability, it also necessitates the verification of each algorithm model individually when errors occur. This increases the workload of verification and hinders effective correction of the corresponding algorithm models, thereby reducing the reliability and accuracy of overall software correction and reconstruction. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a software reconstruction method based on deep reinforcement learning. This method monitors the software's operation to obtain real-time operational information for both the software itself and the hardware terminal, thereby acquiring data flow processing information for the software and terminal data processing load information for the hardware terminal. Based on this information, it identifies algorithm modules within the algorithm architecture platform that are experiencing execution errors or are in a computational dead loop. Then, it retrieves the algorithms contained in the corresponding algorithm models, trains and reconstructs these algorithms using a pre-defined deep learning model, and returns the reconstructed algorithms to the corresponding algorithm modules. This method first monitors the software's operation, analyzing its operational status from both the software itself and the hardware device it resides on. It then determines the normal operating status of each algorithm module within the software's algorithm architecture platform and performs deep learning training and reconstruction on the corresponding algorithm modules. This reduces the workload of software verification and improves the reliability and accuracy of overall software correction and reconstruction.

[0004] This invention provides a software reconstruction method based on deep reinforcement learning, which includes the following steps:

[0005] Step S1: Monitor the software operation process to obtain real-time software operation information and real-time terminal operation information of the hardware terminal corresponding to the software; analyze and process the real-time software operation information and the real-time terminal operation information to obtain the software's data flow processing information and the hardware terminal's terminal data processing load information.

[0006] Step S2: Based on the data stream processing information, obtain the algorithm execution result information of the software's own algorithm architecture platform during operation; based on the algorithm execution result information, determine the algorithm modules in the algorithm architecture platform that are in an execution error state;

[0007] Step S3: Based on the terminal data processing load information, obtain the data processing traffic information of each algorithm module of the software's own algorithm architecture platform during operation; based on the data processing traffic information, determine whether each algorithm module of the algorithm architecture platform is in a computation dead loop state;

[0008] Step S4: When the algorithm module is in an execution error state or an infinite loop state, the corresponding algorithm is retrieved from the algorithm module; and the retrieved algorithm is transmitted to the preset deep learning model for training and reconstruction until the retrieved algorithm completes the training and reconstruction, and then returned to the algorithm module.

[0009] Furthermore, in step S1, monitoring the software operation process and obtaining real-time software operation information and real-time terminal operation information of the hardware terminal corresponding to the software specifically includes:

[0010] When a user initiates a software execution trigger command on a hardware terminal, the corresponding software is instructed to enter the running state according to the software flag of the software execution trigger command; wherein, the software flag refers to the installation location of the software in the system of the hardware terminal;

[0011] The software in operation is monitored in the background of the hardware terminal to obtain the software operation log information and the terminal memory usage and upload / download information of the hardware terminal. These are used as the software operation status information and the terminal operation status information, respectively.

[0012] Furthermore, in step S1, the analysis and processing of the software operation status information and the terminal operation status information to obtain the software data stream processing information and the hardware terminal's terminal data processing load information specifically include:

[0013] The software operation log information is analyzed and processed to extract the input and output data of each algorithm module of the software's own algorithm architecture platform during operation; and based on the algorithm logic relationship between all algorithm modules during software operation, all input and output data are analyzed and processed to obtain the software's data flow processing information.

[0014] The terminal memory usage and upload / download information are analyzed and processed to extract the memory usage, upload traffic value and download traffic value of each algorithm module during the operation of the software, which are used as the terminal data processing load information.

[0015] Furthermore, in step S2, obtaining the algorithm execution result information of the software's own algorithm architecture platform during operation, based on the data stream processing information, specifically includes:

[0016] The algorithm execution output results corresponding to each algorithm module of the software's own algorithm architecture platform during operation are extracted from the data stream processing information; wherein, the algorithm execution output results refer to the output data obtained by each algorithm module after completing the corresponding calculation steps on the received input data.

[0017] Furthermore, in step S2, based on the algorithm execution result information, the algorithm modules in the algorithm architecture platform that are in an execution error state specifically include:

[0018] The output data corresponding to each algorithm module is analyzed and processed to determine whether there is garbled data in the output data of each algorithm module. If there is, the corresponding algorithm module is determined to be in an execution error state; if not, the corresponding algorithm module is determined not to be in an execution error state. Furthermore, the algorithm module in the execution error state is identified on the algorithm architecture platform.

[0019] Furthermore, in step S3, obtaining the data processing traffic information of each algorithm module of the software's own algorithm architecture platform during operation, based on the terminal data processing load information, specifically includes:

[0020] The memory usage, upload traffic, and download traffic of each algorithm module are analyzed to determine the average memory usage, average upload traffic, and average download traffic of each algorithm module during its own algorithm execution process. This data processing traffic information for each algorithm module is used as the data processing traffic information for the execution of the algorithm.

[0021] Furthermore, in step S3, determining whether each algorithm module of the algorithm architecture platform is in a computational dead loop state based on the data processing traffic information specifically includes:

[0022] If the average memory usage rate is greater than or equal to a preset memory usage rate threshold, or the average upload traffic value is greater than or equal to a first traffic threshold, or the average download traffic value is greater than or equal to a second traffic threshold, then the corresponding algorithm module is determined to be in an infinite loop state; otherwise, the corresponding algorithm module is determined not to be in an infinite loop state; and the algorithm module in the infinite loop state is identified on the algorithm architecture platform.

[0023] Further, in step S4, when the algorithm module is in an execution error state or an infinite loop state, the corresponding algorithm is retrieved from the algorithm module; and the retrieved algorithm is transferred to a preset deep learning model for training and reconstruction until the retrieved algorithm completes training and reconstruction, and then returned to the algorithm module. Specifically, this includes:

[0024] Based on the identification results of the algorithm modules in the execution error state or operation dead loop state on the algorithm architecture platform, the algorithm contained in the corresponding algorithm module is retrieved; the retrieved algorithm is transmitted to the preset deep learning model for training; based on the training results, the algorithm defects existing in the retrieved algorithm are determined, and the algorithm defects are reconstructed; then the algorithm that has completed training and reconstruction is returned to the algorithm model.

[0025] Compared to existing technologies, this deep reinforcement learning-based software reconstruction method monitors the software's operation process to obtain real-time operational information for both the software itself and the hardware terminal. This information is used to obtain data flow processing information for the software and terminal data processing load information for the hardware terminal. Based on these two types of information, algorithm modules in the algorithm architecture platform that are in an execution error state or an infinite loop state are identified. Then, the algorithms contained in the corresponding algorithm models are retrieved, and the retrieved algorithms are trained and reconstructed using a preset deep learning model. The trained and reconstructed algorithms are then returned to the corresponding algorithm modules. This method first monitors the software's operation process, analyzes the software's operational status from both the software itself and the hardware device on which the software resides, judges the normal operation status of each algorithm module in the software's algorithm architecture platform, and performs deep learning training and reconstruction on the corresponding algorithm modules. This reduces the workload of software verification and improves the reliability and accuracy of overall software correction and reconstruction.

[0026] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0027] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a flowchart illustrating the software reconstruction method based on deep reinforcement learning provided by the present invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] See Figure 1 This is a flowchart illustrating the software reconstruction method based on deep reinforcement learning provided in an embodiment of the present invention. The software reconstruction method based on deep reinforcement learning includes the following steps:

[0032] Step S1: Monitor the software operation process to obtain real-time software operation information and real-time terminal operation information of the hardware terminal corresponding to the software; analyze and process the real-time software operation information and the real-time terminal operation information to obtain the software's data flow processing information and the hardware terminal's terminal data processing load information.

[0033] Step S2: Based on the data stream processing information, obtain the algorithm execution result information of the software's own algorithm architecture platform during operation; based on the algorithm execution result information, determine the algorithm modules in the algorithm architecture platform that are in an execution error state;

[0034] Step S3: Based on the terminal data processing load information, obtain the data processing flow information of each algorithm module of the software's own algorithm architecture platform during operation; based on the data processing flow information, determine whether each algorithm module of the algorithm architecture platform is in a computation dead loop state;

[0035] Step S4: When the algorithm module is in an execution error state or an infinite loop state, the corresponding algorithm is retrieved from the algorithm module; and the retrieved algorithm is transferred to the preset deep learning model for training and reconstruction until the retrieved algorithm completes the training and reconstruction, and then returned to the algorithm module.

[0036] The beneficial effects of the above technical solution are as follows: This software reconstruction method based on deep reinforcement learning monitors the software operation process to obtain the corresponding real-time operation information of the software itself and the hardware terminal, thereby obtaining the data flow processing information of the software and the terminal data processing load information of the hardware terminal; based on the above two types of information, the algorithm modules in the algorithm architecture platform that are in an execution error state or in a computational dead loop state are identified respectively; then, the algorithms contained in the corresponding algorithm model are retrieved, and the retrieved algorithms are trained and reconstructed using a preset deep learning model, and the trained and reconstructed algorithms are returned to the corresponding algorithm modules; the above method first monitors the software operation process, analyzes the real-time operation of the software from both the software itself and the hardware device where the software is located, judges whether the operation status of each algorithm module of the software's algorithm architecture platform is normal, and performs deep learning training and reconstruction on the algorithms of the corresponding algorithm modules, reducing the workload of software inspection and improving the reliability and accuracy of overall software correction and reconstruction.

[0037] Preferably, in step S1, monitoring the software operation process and obtaining real-time software operation information and real-time terminal operation information of the hardware terminal corresponding to the software specifically includes:

[0038] When a user initiates a software execution trigger command on the hardware terminal, the corresponding software is instructed to enter the running state according to the software flag of the software execution trigger command; whereby the software flag refers to the installation location of the software in the hardware terminal's system;

[0039] The software running in the background of the hardware terminal is monitored to obtain the software operation log information and the terminal memory usage and upload / download information during the operation process. These are used as the real-time information of the software operation and the real-time information of the terminal operation, respectively.

[0040] The beneficial effects of the above technical solution are as follows: by using the above method, when the software enters the running state, the software itself and the hardware terminal where the software is located can be monitored synchronously, thereby obtaining the real-time information of the software operation and the real-time information of the terminal operation, providing a comprehensive information and data foundation for subsequent analysis of whether the software is running normally.

[0041] Preferably, in step S1, the analysis and processing of the software operation status information and the terminal operation status information to obtain the software data flow processing information and the hardware terminal's terminal data processing load information specifically includes:

[0042] The software's runtime log information is analyzed and processed to extract the input and output data of each algorithm module of the software's own algorithm architecture platform during operation. Based on the algorithm logic relationship between all algorithm modules during software operation, all input and output data are analyzed and processed to obtain the software's data flow processing information.

[0043] The memory usage and upload / download information of the terminal are analyzed and processed to extract the memory usage, upload traffic value and download traffic value of each algorithm module during the operation of the software, which are used as the data processing load information of the terminal.

[0044] The beneficial effects of the above technical solution are as follows: Through this method, the software operation log information includes detailed information about the input and output data of each algorithm module in the software's algorithm architecture platform during operation. Furthermore, there are corresponding algorithmic logical relationships between different algorithm modules within the software's algorithm architecture platform. This algorithmic logical relationship means that the output data of one algorithm module serves as the input data of another algorithm module. Based on this algorithmic logical relationship, the software's data flow processing information can be accurately and comprehensively obtained from the software operation log information. This data flow processing information represents the correlation between the output and input data of different algorithm modules. In addition, when an algorithm module executes its own algorithm, it will correspondingly occupy the hardware terminal's memory space and perform data uploads and downloads. By extracting the memory usage rate, upload traffic value, and download traffic value corresponding to each algorithm module during software operation, a detailed characterization of the terminal's data processing load information can be achieved.

[0045] Preferably, in step S2, obtaining the algorithm execution result information of the software's own algorithm architecture platform during operation based on the data stream processing information specifically includes:

[0046] The algorithm execution output results corresponding to each algorithm module of the software's own algorithm architecture platform during operation are extracted from the data stream processing information; where the algorithm execution output results refer to the output data obtained by each algorithm module after completing the corresponding calculation steps on the received input data.

[0047] The beneficial effects of the above technical solution are as follows: through the above method, the data input / output of each algorithm module during the operation of the software can be effectively extracted, thereby obtaining the corresponding data stream processing information completely.

[0048] Preferably, in step S2, based on the algorithm execution result information, the algorithm modules in the algorithm architecture platform that are in an execution error state specifically include:

[0049] The output data corresponding to each algorithm module is analyzed and processed to determine whether there is garbled data in the output data of each algorithm module. If there is, the corresponding algorithm module is determined to be in an execution error state. If not, the corresponding algorithm module is determined not to be in an execution error state. Furthermore, the algorithm module in the execution error state is marked on the algorithm architecture platform.

[0050] The beneficial effects of the above technical solution are as follows: When the algorithm module has a running error while running its own algorithm, its output data will contain certain garbled data components. By analyzing and processing the data codes of the output data, it is possible to quickly and accurately determine whether there is an algorithm execution error in the algorithm model.

[0051] Preferably, in step S3, obtaining the data processing traffic information of each algorithm module of the software's own algorithm architecture platform during operation, based on the terminal's data processing load information, specifically includes:

[0052] The memory usage, upload traffic, and download traffic of each algorithm module are analyzed to determine the average memory usage, average upload traffic, and average download traffic of each algorithm module during its own algorithm execution process. This data processing traffic information for each algorithm module is used as the data processing traffic information for the execution of the algorithm.

[0053] The beneficial effects of the above technical solution are as follows: through the above method, the memory usage, upload traffic and download traffic can be averaged and measured throughout the entire process of the algorithm module executing its own algorithm. This allows for accurate calibration of the data operation traffic information of each algorithm module.

[0054] Preferably, in step S3, determining whether each algorithm module of the algorithm architecture platform is in a computational dead loop state based on the data processing traffic information specifically includes:

[0055] If the average memory usage is greater than or equal to a preset memory usage threshold, or the average upload traffic is greater than or equal to a first traffic threshold, or the average download traffic is greater than or equal to a second traffic threshold, then the corresponding algorithm module is determined to be in an infinite loop state; otherwise, the corresponding algorithm module is determined not to be in an infinite loop state; and the algorithm module in the infinite loop state is identified on the algorithm architecture platform.

[0056] The beneficial effects of the above technical solution are as follows: When the algorithm model is executing its own algorithm, if a computational dead loop occurs in a certain algorithm step, it will cause a large amount of memory space to be occupied on the hardware terminal, or cause an abnormal increase in data upload / download traffic. By comparing the average memory usage, average upload traffic value and average download traffic value with thresholds, it is possible to determine in a timely manner whether the algorithm module is in a computational dead loop state.

[0057] Preferably, in step S4, when the algorithm module is in an execution error state or an infinite loop state, the corresponding algorithm is retrieved from the algorithm module; and the retrieved algorithm is transferred to a preset deep learning model for training and reconstruction until the retrieved algorithm completes training and reconstruction, and then returned to the algorithm module. Specifically, this includes:

[0058] Based on the identification results of algorithm modules in execution error state or operation dead loop state on the algorithm architecture platform, the algorithm contained in the corresponding algorithm module is retrieved; the retrieved algorithm is transmitted to the preset deep learning model for training; based on the training results, the algorithm defects existing in the retrieved algorithm are determined, and the algorithm defects are reconstructed; then the trained and reconstructed algorithm is returned to the algorithm model.

[0059] The beneficial effects of the above technical solution are as follows: by using the above method, the retrieved algorithm is trained using a pre-constructed deep learning model, the algorithm defects are identified, and the identified algorithm defects are repaired and reconstructed, thereby maximizing the correctness of the algorithm module and ensuring the overall reliability of the software operation.

[0060] As can be seen from the above embodiments, this deep reinforcement learning-based software reconstruction method monitors the software's operation process to obtain real-time operational information for both the software itself and the hardware terminal, thereby obtaining data flow processing information for the software and terminal data processing load information for the hardware terminal. Based on these two types of information, it identifies algorithm modules in the algorithm architecture platform that are in an execution error state or an infinite loop state. Then, it retrieves the algorithms contained in the corresponding algorithm models, trains and reconstructs the retrieved algorithms using a preset deep learning model, and returns the trained and reconstructed algorithms to the corresponding algorithm modules. This method first monitors the software's operation process, analyzes the software's operational status from both the software itself and the hardware device where the software resides, judges whether the operation status of each algorithm module in the software's algorithm architecture platform is normal, and performs deep learning training and reconstruction on the corresponding algorithm modules' algorithms. This reduces the workload of software verification and improves the reliability and accuracy of overall software correction and reconstruction.

[0061] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A software reconstruction method based on deep reinforcement learning, characterized in that, It includes the following steps: Step S1: Monitor the software operation process to obtain real-time software operation information and real-time terminal operation information of the hardware terminal corresponding to the software; analyze and process the real-time software operation information and the real-time terminal operation information to obtain the software's data flow processing information and the hardware terminal's terminal data processing load information. Step S2: Based on the data stream processing information, obtain the algorithm execution result information of the software's own algorithm architecture platform during operation; based on the algorithm execution result information, determine the algorithm modules in the algorithm architecture platform that are in an execution error state; Step S3: Based on the terminal data processing load information, obtain the data processing traffic information of each algorithm module of the software's own algorithm architecture platform during operation; based on the data processing traffic information, determine whether each algorithm module of the algorithm architecture platform is in a computation dead loop state; Step S4: When the algorithm module is in an execution error state or an infinite loop state, the corresponding algorithm is retrieved from the algorithm module; and the retrieved algorithm is transmitted to the preset deep learning model for training and reconstruction until the retrieved algorithm completes the training and reconstruction, and then returned to the algorithm module. Specifically, in step S1, monitoring the software operation process and obtaining real-time software operation information and real-time hardware terminal operation information corresponding to the software includes: When a user initiates a software execution trigger command on a hardware terminal, the corresponding software is instructed to enter the running state according to the software flag of the software execution trigger command; wherein, the software flag refers to the installation location of the software in the system of the hardware terminal; The software in operation is monitored in the background of the hardware terminal to obtain the software operation log information and the terminal memory usage and upload / download information of the hardware terminal. These are used as the software operation status information and the terminal operation status information, respectively.

2. The software reconstruction method based on deep reinforcement learning as described in claim 1, characterized in that: In step S1, the analysis and processing of the software operation status information and the terminal operation status information to obtain the software data stream processing information and the hardware terminal's terminal data processing load information specifically include: The software operation log information is analyzed and processed to extract the input and output data of each algorithm module of the software's own algorithm architecture platform during operation; and based on the algorithm logic relationship between all algorithm modules during software operation, all input and output data are analyzed and processed to obtain the software's data flow processing information. The terminal memory usage and upload / download information are analyzed and processed to extract the memory usage, upload traffic value and download traffic value of each algorithm module during the operation of the software, which are used as the terminal data processing load information.

3. The software reconstruction method based on deep reinforcement learning as described in claim 2, characterized in that: In step S2, obtaining the algorithm execution result information of the software's own algorithm architecture platform during operation based on the data stream processing information specifically includes: The algorithm execution output results corresponding to each algorithm module of the software's own algorithm architecture platform during operation are extracted from the data stream processing information; wherein, the algorithm execution output results refer to the output data obtained by each algorithm module after completing the corresponding calculation steps on the received input data.

4. The software reconstruction method based on deep reinforcement learning as described in claim 3, characterized in that: In step S2, based on the algorithm execution result information, the algorithm modules in the algorithm architecture platform that are in an execution error state specifically include: The output data corresponding to each algorithm module is analyzed and processed to determine whether there is garbled data in the output data of each algorithm module. If there is, the corresponding algorithm module is determined to be in an execution error state; if not, the corresponding algorithm module is determined not to be in an execution error state. Furthermore, the algorithm module in the execution error state is identified on the algorithm architecture platform.

5. The software reconstruction method based on deep reinforcement learning as described in claim 4, characterized in that: In step S3, obtaining the data processing traffic information of each algorithm module of the software's own algorithm architecture platform during operation, based on the terminal data processing load information, specifically includes: The memory usage, upload traffic, and download traffic of each algorithm module are analyzed to determine the average memory usage, average upload traffic, and average download traffic of each algorithm module during its own algorithm execution process. This data processing traffic information for each algorithm module is used as the data processing traffic information for the execution of the algorithm.

6. The software reconstruction method based on deep reinforcement learning as described in claim 5, characterized in that: In step S3, determining whether each algorithm module of the algorithm architecture platform is in a computational dead loop state based on the data processing traffic information specifically includes: If the average memory usage rate is greater than or equal to a preset memory usage rate threshold, or the average upload traffic value is greater than or equal to a first traffic threshold, or the average download traffic value is greater than or equal to a second traffic threshold, then the corresponding algorithm module is determined to be in an infinite loop state; otherwise, the corresponding algorithm module is determined not to be in an infinite loop state; and the algorithm module in the infinite loop state is identified on the algorithm architecture platform.

7. The software reconstruction method based on deep reinforcement learning as described in claim 6, characterized in that: In step S4, when the algorithm module is in an execution error state or an infinite loop state, the corresponding algorithm is retrieved from the algorithm module; and the retrieved algorithm is transferred to a preset deep learning model for training and reconstruction until the retrieved algorithm completes training and reconstruction, and then returned to the algorithm module. Specifically, this includes: Based on the identification results of the algorithm modules that are in an execution error state or an infinite loop state on the algorithm architecture platform, the algorithm contained in the corresponding algorithm module is retrieved; the retrieved algorithm is transmitted to a preset deep learning model for training; based on the training results, the algorithm defects existing in the retrieved algorithm are determined, and the algorithm defects are reconstructed; then the algorithm that has completed training and reconstruction is returned to the algorithm module.

Images